“HUNTING CARIBOU HUNTERS BENEATH MODERN LAKE HURON 2: USING VIRTUAL WORLD MODELING AND SERIOUS...

“HUNTING CARIBOU HUNTERS BENEATH MODERN LAKE HURON 2:

USING VIRTUAL WORLD MODELING AND SERIOUS GAMES TO GUIDE THE EXPLORATION OF THE ALPENA-AMBERLEY LAND BRIDGE.”

Robert G. Reynolds

Professor Computer Science

Wayne State UniversityDetroit, Michigan 48202

andAssociate Research Scientist

Museum of AnthropologyUniversity of Michigan-Ann Arbor

Summary• The Alpena-Amberley land bridge• Goals• Influence Maps• Path-finding• Flocking• Cultural Algorithms• Implementation• Results• Graphical User Interface• Future Work

Early Human Occupation of the Upper Great Lakes

PaleoIndian (fluted point) occupation Begins: ? Ends: 10,300 BP with drop from Lake Algonquian (i.e. Lake

Stanley) Archaic 10,000-2,000 BP

Early and Middle Archaic poorly known (Lake Stanley) Late Archaic sees major explosion in known sites (Post-

Lake Stanley). Tight correspondence assumed between PaleoIndian

social organization, big game hunting, and the environment of the Lake Algonquian high water phase.

Very few known sites of Early and Middle Archaic Age.

Lake Stanley Dates and Elevations

Low Water Phase Date in 14C Years BPElevation Above Mean Sea

Level

Early Lake Stanley 9900 - 9500 55 – 80m

Mid Lake Stanley 9300-9000 85 – 100m

Late Lake Stanley 7900-7500 90 – 95m

Layered Search and Evaluation Strategy

AUV and ROV

• •

Project Goals

• Hunter behavior reflects that of their food sources, Caribou.

• Knowing where Caribou are likely to be at given times of the year can be used to predict the location of human occupants.

• Maintain a virtual world that can be used to visualize the past landscape that the surveyors are currently looking at.

Extensive (Macro)

• Can be a repository of found artifacts and structures stored in a “virtual” or cognitive

database. • Produce a variety of “influence maps” that can

contain information about present and past environments that are taken from running the “virtual world”. These can be used by surveyors to make “tactical decisions” about where to deploy the various survey devices.

Intensive Survey (Meso)

• Can be used to interpret “videos” taken by underwater devices in terms of the location of the device in the past virtual world.

• GPS positioning of underwater device can be used to locate its position in the “virtual world”.

Scale of Contribution (Micro)

• Can be used to link up the view on the lake floor taken by a survey device with the corresponding portion of the landscape in the virtual world.

• Virtual site prospecting.• Provide a virtual context for “found” artifacts

and structures.

Stages of Virtual World Development

• 1)Pristine Environment-Caribou in Tundra environment.

• 2)Add in human occupation.• 3)Adjust system to respond to long term

climate change, e.g. lake levels shifts.

Caribou Simulation Goals• Previous work focused on modeling tundra environment and

individual animal behavior – Walters, ’75 – Barren ground caribou dynamics– Bergman ’00 – Caribou movement as correlated random

walk– Bliss ’73, Price ’99, Sirois ’99 – Arctic tundra ecosystems

• Other previous work by Reynolds and students in WSU AI lab began to take a look at the holistic view– Reynolds ’09 – Design of reality games– Vitale ’09 – Integration of environment interacting with

caribou– More work ongoing

Simulation Goals

Simulation Goals

• Use a “serious game” to tie together environment and entities – Game designed for use other than entertainment (Abt,

’70)– Extensibility– Reusability– Interactivity

• Discover migratory paths and patterns– Where are the likely areas of local concentration– Can we predict behavior and actual historical sites

Alpena-Amberley Land Bridge

Simulation Goals

• Utilize a number of technologies for effective simulation and presentation– Microsoft’s XNA Framework– Influence Maps– Path-Finding– Cultural Algorithms

• Use multiple knowledge sources in tandem, which will be used and refined by Cultural Algorithms to generate realistic migratory behavior by herds.

Game Program Overview

Caribou Simulation Game

• COMPONENT FRAMEWORK:– 3D virtual world component– Basic game engine component– CAT 3.0 component

22

Pilot Study Search Areas

Environment Generation

• Combination of real-world and simulated data

Easting Northing Elevation

381571.523 4972936.15128.5

381637.29 4972934.936 27.6

… … …

Environment Generation

Environment Generation

• Data not always smooth

Environment Generation• Simple method based on models of boreal growth

– Two types of vegetation, tree and scrub.– Different levels of nutritional value.– Considered the availability of light and water.

• More thorough modeling, such as seed transport, forest fires and age, may yield different maps

Area 1

Environment Generation

Influence Maps – What They Are• The mechanism behind several knowledge sources• They are 2D or 3D cellular divisions of a world

– Tactical values associated with each cell. – Strong support and history in gaming community (Tozour,

’01)– Thorough support system built around this module.

• Dynamically alter and constrain values• Dynamic size and cell dimensions based on creation parameters• Load and save maps from and to greyscale• Merge and branch maps• Discover values at specified positions or indexes, etc

Influence Maps – What They Are

A 2D influence map Changing influences

Influence Maps - Implementation

Using multiple maps: by splitting maps, we can “save” a particular influence. Merging maps using negative values acts as deterrents.

Influence Maps – Our Usage• We base our maps on certain parameters to use as input:

– Availability of food.– Historical death rate.– Passability of terrain.

• Initialized according to those parameters• Knowledge sources for Cultural Algorithm• Other parameters for future work:

– Predator behavior• Wolves, humans, etc

– Local influence• Drive lanes

Basic AI Engine

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Can be searched using A★

heuristic search– Commonly used paths can

be stored for quick access• Cons

– Worst-case complexity is exponential in nature

– Paths may look unrealistic and require post-processing in some cases

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

35

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Partitions the environment’s

terrain into polygons– Waypoints are used to

connect points that create paths

• Cons– Works best with static

environments– Similar issues to Grid-Based

Methods

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

36

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Apply local environmental

information to generate movement

– Characters are react from intrinsic information within the landscape

– Reactions are determined on a per-application basis, most often using “potential fields”

• Cons– Bots often get stuck from

poorly designed potential fields

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

37

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Apply local environmental

information to generate movement

– Characters are react from intrinsic information within the landscape

– Reactions are determined on a per-application basis, most often using “potential fields”

• Cons– Bots often get stuck from

poorly designed potential fields

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

38

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Used often in computer

graphics– Behavior is determined by:

• a specified set of rules• social forces• particle swarm methods

• Cons– Requires quantifying,

identifying and controlling abstract knowledge and information

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

39

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Combines several

approaches in one– Can consider local and global

information• Cons

– Difficult to design and implement

– Requires in-depth knowledge of the problem

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

40

ALGORITHMS FOR STEERING SOFTBOTS IN GAME WORLDS

• Pros– Always engineered with the

optimal path solution– Direct-control of agent

steering• Cons

– Very restrictive– Unresponsive to the smallest

change– Non-autonomous

1. GRID-BASED METHODS2. NAVIGATION MESHES3. REACTIVE METHODS4. AGENT-BASED APPROACHES5. HYBRID APPROACHES6. HARDCODED SYSTEMS

41

Path Planning in Games with Cultural Algorithms: Previous Work

• Super_Mario Competition (2010).

• Car Racing Competition (2008).

Mario AI Controller & Logistics

PATH PLANNING IN GAMES WITH CA

• Reynolds & Kinnaird-Heether [2008]– WCCI 2008 Competition; – Socially motivated, agent-based approach:

• Cultural Algorithms

– 3D racing environment– Parameterizes rules for an single racecar driver

• RESULTS: Steers a car around a track

48

EXAMPLE OF CA OPTIMIZATION Reynolds & Kinnaird-Heether Applied CA to learn racing parameters Came in second at the WCCI 2008 How can this approach be be scaled up?

49

acceleration(speedX, maxSpeed)if(speedX<maxSpeed)accel= 1elseaccel= 0

THE TALE OF TWO SIMULATIONS

REYNOLDS/HEETHER RACECAR• LOCATION:

– Multiple race tracks

• CA INTERFACES WITH: – T.O.R.C.S (3rd party)

• GOAL:– Single agent path-planning

CARIBOU SIMULATION• LOCATION:

– Alpena-Amberley Ridge

• CA INTERFACES WITH:– Virtual World (1st party)

• GOAL:– Multiple agent path-planning

50

SHOW CULTURAL ALGORITHMS CAN BE ADAPTED TO PARAMETERIZE THE RULES FOR THE MOVEMENT OF CARIBOU AGENTS WITHIN A LAND BRIDGE GAME SIMULATION

OUR GOAL:

51

START

FINISH

Path-Finding – Our Usage• Each herd is an individual

– Has separate set of in-order nodes to follow– The herd generates the path, all members space out and

follow• Path is established by a path-finding algorithms

– We choose to use A*• Established, efficient, suited for graph based paths• We can treat each influence map cell as a node

• Herd is given the entire path to find– Herd moves as a single unit (local center is considered the

position)

Path-Finding (A* Pseudocode)AStarPath(AllNodes)

currentPath = 0g(S) = 0.0h(S) = hEstimatE(S, goal)f(S) = h(S)  CreateOpenList(S)CreateClosedList()While OpenList.Length > 0

node = lowest(O) if (node == goal)

construct pathreturn

open.remove(node)closed.add(node)

foreach n_node in neighbors(node) if closed.contains(n_node) continue g_temp = g(node) + dist(node, n_node)

if (!open.contains(n_node)) open.add(n_node) currentBetter = true elseif g_temp < g(n_node) currentBetter = true else currentBetter = false

if (currentBetter) if (currentPath(n_node)) if (f(node) < f(n_node)) currentPath(n_node) = x else currentPath(n_node) = x g(n_node) = g_temp; h(n_node) = hEstimate(n_node, goal) f(n_node) = g(n_node) + h(n_node)

Path-Finding - Example

f(n) = g(n) + h(n)

Flocking – What is it• Gives visual realism to members of our herds• Affects vegetation consumption by how members of our herds spread out• Expansion on three basic parameters of flocking

– Separation– Alignment– Adhesion

Cultural Algorithms – Origins and Concept

• Described by Reynolds in ‘79. • Cultural Algorithms are a type of socially motivated learning.• Evolutionary process:

– Population (micro-evolutionary)• Individual behavior

– Belief space (macro-evolutionary)• The combined knowledge of the sum of valuable individuals.

• Cultural Knowledge model derived from Flannery, “Archaeological Systems Theory and early Mesoamerica”, in Anthropological Archaeology in the Americas, Betty J. Meggars, ed., pp: 67-87, The Anthropological Society of Washington, Washington, D.C., 1968.

• Population model derived from Holland’s Genetic Algorithms– “Adaptation in Natural and Artificial Systems”, 1975.

Cultural Algorithms - Behavior

Begin t = 0 InitPop(t) InitBelief(t) Repeat EvaluatePop(t) Vote(Belief(t), Accept(Pop(t))) Adjust(Belief(t)) Evolve(Pop(t), Influence(Belief(t))) t++ Select Pop(t) from Pop(t – 1) Until (termination condition)End

Cultural Algorithms - Belief Space• Belief Space: a source of cultural knowledge. Each knowledge source represents:

– Normative knowledge• Parameter ranges which individuals can vary by set increments

– Domain knowledge• First set of influence maps: terrain, floral and faunal info drawn from research

into tundra environment.– Situational knowledge

• Chromosome of the top 10% of individuals for successful herd crossings– Temporal knowledge

• Second set of influence maps: the history of cells in which unsuccessful individuals lost members (i.e. caribou died in a herd). Used as negative values to discourage entrance into the area.

– Spatial knowledge• Influence map of current pathfinding node preferences based on unsuccessful

and successful herd movement strategies.

Cultural Algorithms - Population Space

• Utilization of genetic algorithm terminology to describe individuals in the Population Space

• Herds are our individuals represented as a GA– Each individual chromosome controls a herd– Construct a “gene” containing beliefs for those

items which the individual herds reference at runtime

– Tied together with influence maps as a combined source of knowledge

Cultural Algorithms - Performance

• The success of a particular individual herd is scored on the following guidelines:– Must successfully cross the land-bridge (herdTransTime)– The number of surviving herd members (herdCount),

average nutritional information (avgHerdCalories) and the transition time in seconds are counted.

– v = herdCount * avgHerdCalories * (maxTime / herdTransTime))

– Top 10% of those herd scores are ACCEPTED to UPDATE the Belief Space.

Cultural Algorithms - Communication

• Population Space to Belief Space– Top 10% of performers elected to update the knowledge sources

in the Belief Space.– Selected based on performance (previous slide)– Chromosome values (herds) are shifted by 50% towards latest top

performers• Belief Space to Population Space

– Top 10% of herd plans (chromosomes) are used to update the normative knowledge source

– Failed and successful herds update situational, temporal, and spatial.

– Domain knowledge source is the vegetation, which regrows

Knowledge Source Update• The influence map is a large component of our knowledge

sources.– Current and historically low cell values in influence map mean

locations will be less likely to be selected as path node– Combined with normative knowledge (particularly detection

distance and final goal weight) to select paths• Individual feeding reduces attractiveness of terrain (domain

knowledge) to others as they eat– Each animal reduces [0, 255] values by 0.5 every second,

increasing their nutritional intake– More feeding reduces cell’s values, less likely to be chosen as a

node for other herds.

Interactive GUI interface (Che 2.0)

KS Types Color

Normative O

blue

Situational O

white

Domain O

green

History O

yellow

Topographical O

Cyan

Implementation

• Microsoft .Net C#– XNA Framework for asset storage, visual display and

some game engine functionality.– Efficient, managed code.

Implementation

• Microsoft .Net C#– XNA Framework for asset storage, visual display and

some game engine functionality.– Efficient, managed code.

Results

• Box highlights the regions where the experiment take place. Herd migration can take place from north to south, or south to north.

Results• 10 herds, 25 caribou each, 100 generations.• 6 minute allowable crossing time.• Failure per individual:

– All caribou expired (starvation).– Time elapsed.

• Success per individual– Arrival at goal.

• Generation ends when all individuals have succeeded or failed.

• Highest survival rate at end of 100 generations is best individual.

Results

• Important factors emerge:– Final goal weight is important

• Too high, cross the bridge with no regard for survival• Too low, never achieve the land bridge crossing in time• Rapid crossing with low survival or nutritional intake• High intake without ever crossing

– Detection distance is important in finding best nodes– High old direction influence will on occasion allow

herds to continue into new cells and allow new possible paths to be computed

The Learning Curve

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97100

0

10

20

30

40

50

60

Pathfinding

Path nodes for 4 small herds

Pathfinding

Path nodes for 4 small herds arriving to final destination

Paths from a test run

Unsuccessful herd in lower left, successful is small trail along middle

Paths from a test run

Successful trails for 4 small herds

Results• Different patterns of crossing the land bridge emerged:

– Crossing with little regard for health or survival rates. Runs #10 and #3.

– Greedy behavior with consumption only, expiring the clock. Runs #2 and #8.

– A moderate path emerged which balanced the two, leading to decent survival and crossing time. Runs #1 and #9.

• More realistic information for vegetation and consumption might lead to different results, as input variables would change

Land Bridge GUI User’s Guide

The Land Bridge GUI is simplistically controlled with just the mouse, and the four arrow keys.

HUD Map The HUD Map is an accurate, two-dimensional display of the currently loaded UTM data file. It can show a variety of information representations, and by default displays the height data, with the darkest colors representing the deepest points, and the brightest spots representing the highest. The three flags on the map represent location, the Red Flag indicating the camera’s current location, it will also move to update itself when you use the Camera. The Green Flag can be positioned by clicking on the HUD Map, and through the Map Controls, you can jump to the location indicated. The Yellow Flag indicates the last location the camera was at prior to using the ‘Jump To’ button. Grabbing the edges of the HUD Map allows you to move the HUD Map around the screen. Compass The Compass indicates the direction the camera’s currently facing. Grabbing the Compass allows you to move it around the screen and reposition it to your liking. Camera Controls Before getting into the buttons on the side, let’s discuss the Camera. The Camera is controlled with the four arrow keys. Pressing Up and Down moves the Camera forwards and backwards, respectively, while pressing Left and Right rotate the camera.

Map Controls

On-Off Switch: Clicking on this button allows you to turn the HUD Map on and off.

Jump To: Clicking this button will jump the camera to the Green Flag’s location.

Height: Clicking this sets the HUD Map to display the height data.

Temperature: Clicking this sets the HUD Map to display temperature data.

HUD Scale: Clicking Plus or Minus will increase or decrease the size of the HUD.

Water Depth: Clicking either button will raise or lower the water depth, and the HUD will update once the buttons are released.

Animal Controls

Caribou: Displays influence map data for the Caribou. Camera Controls

Compass Switch: Clicking this button switches the Compass on and off.

Terrain Following: Clicking this will switch the Camera’s terrain-following on and off. While following terrain (which the camera begins on by default), the Camera will swoop up and down to follow the terrain. When switched off, the Camera will constantly aim towards the horizon.

Camera Speed: Clicking plus or minus increases and decreases the Camera speed.

Water Filter: Switch the Water Filter on and off. The Water Filter by default activates when underwater to let you know if the camera’s underwater, with a blue tint and bubbles.

Future Work • GUI (replace current terrain with virtual world)• Vegetation and Terrain

– More detailed and accurate generation, as well as nutritional information– Accurate modeling of movement impact of certain types– More floral variety.– More faunal variety.

• Herd variability– Different genders, roles and states for caribou.

• External factors– Integration of Paleolithic hunters– Weather and seasons

Thank You!

• A special thank you to the Land Bridge Team at WSU:

• James Fogarty• Thomas Palalazollo• Jin Jin• Gerald Larsen