Automatic Annotation of Urban Environments to Support POL Behavior Automatic Annotation of Urban Environments to Support Pattern of Life Behavior Daniel

Automatic Annotation of Urban Environments to Support POL Behavior

Automatic Annotation of Urban Environments to Support Pattern of

Life Behavior

Daniel J. Hershey

David M. McKeown

TerraSim, Inc.One Gateway Center, Suite 2050

420 Fort Duquesne Blvd.Pittsburgh, PA 15222

March 29, 2012GameTech 2012

Orlando, FL


Introduction

Pattern of Life (POL) in Urban Environments

• How can we create more compelling simulations of human activity in urban environments?

• How can we avoid manual scripting or programing the background activities that are common to urban life?

• As we develop new urban environments for simulation and training, do we have to start with a clean slate?

• In this presentation we will address these issues and show how urban environments can be automatically generated to support advanced POL activity.


What Is Pattern of Life Anyway?

• AI entity behaviors that autonomously perform daily background “life” activities within an environment.

– Driving to work and parking a car, walking from car to office, enter office building.

– Leave office building for lunch, walk to lunch place, meet with others, go into restaurant.

– Leave restaurant in group, individually return to place of work.

– Leave work, walk to car from office, drive car from parking garage.

• Using spatial cues, this part of the simulation should be autonomous and tailored to the specific environment


Information Required to Support POL

• Locations and the properties of places in the environment• Spatial organization of the urban terrain for appropriate

routing of pedestrians and vehicles• Properties need a rich set of spatial attribute descriptors.

– Types of buildings based on function:• residential, commerial, administrative, parking lots, etc.

– Types of open areas based on human use: • parks, soccer fields, stadiums, lakes, open space. etc.

• This information can be generated from:– the geospatial source data used to build the 3D environment– or automatically generated based upon urban density statistics.– or interactively added to source data prior to 3D generation.


Urban Environments

• Urban spatial organization varies widely depending on area of the world

• However, there are common themes of human geography and, in particular, daily human activity.

• Currently background POL activity is manually scripted, when it is available at all.

• AI must infer spatial relationships and content from the game geometry without benefit from other semantic information.

• Our goal is to provide a correlated attributed model that various AI systems and Game Developers can use directly.


Generation of Annotated Environments for POL

• Make minimal assumptions about available source data– Road centerline and building footprints are generally

available for most modeling and simulation tasks• Derive all other POL features from input source data

– Use a combination of computational geometry and image processing

– Minimize human interaction to post processing checks

• Make annotated urban environment generation an integral part of runtime database creation process.


Annotated Urban Environments Process Flow


Some Key Urban Annotation Primitives

Pedestrian Routing Vehicular Routing• Sidewalk areas• Crosswalk areas• Buildings• Building entrances

• Road areals• Road centerlines• Road intersection area• Intersection points• Parking areas• Parking curb cutouts


Building an Integrated Spatial Network

Pre-computed relationships vs. derived spatial relationships• crosswalk 42 connects sidewalk 37 + 61• crosswalk 42 crosses road surface 84• sidewalk 37 is adjacent to parcel 54• parcel 54 contains buildings <building list>• building 101 as entrances at <location list>• road 14 intersects road 37 at intersection 99

Support entity reasoning and navigation at a high level of spatial abstraction


Software Support for Annotated Environments

Develop an application programming interface• Use API support to abstract (hide) underlying spatial

data content and organization.

Levels of API support:• Access to each thematic layer: basic geometry and

attribution• Access to all layers: specify and limit to spatial area of

interest• Access to functional and spatial relationships between

urban objects within and across layers


Generation of an Urban Semantic Description


Complex Urban Environment (360K buildings)


Building 2D Footprint Layer (Input Data)


Building Parcel Source Layer (Input Data)


Road Centerline Source Layer (Input Data)


Road Junction Areas (Derived)


Street Islands (Derived)


Sidewalks (Derived)


Pedestrian Crosswalks (Derived)


Parking Areas (Derived)


Parking Access Points (Derived)


Building Entrances (Derived)


Example Area 1


Example Area 1 (Phototexture)


Example Area 1 (Source Data Layers)


Example Area 1 (Source and Phototexture)


Example Area 1 (POL Derived Layers)


Detailed Area 2 (Angled Road)


Phototexture Detailed Area 2 (Angled Road)


Source Data Detailed Area 2 (Angled Road)


Detailed Area 2 (Angled Road)


POL Generated Detailed Area 2 (Angled Road)


Detailed Area 2 (Angled Road) in VBS2


Detailed Area 2 (Angled Road) in VBS2


Detailed Area 3 (Median Road)


Phototexture Detailed Area 3 (Median Road)


Source Data Detailed Area 3 (Median Road)


Detailed Area 3 (Median Road)


POL Generated Detailed Area 3 (Median Road)


Detailed Area 3 in VBS2 (Median Road)


Detailed Area 3 in VBS2 (Median Road)


Parking Area


Parking Area (Zoom)


Parking Area and Cutout in VBS2


Parking Area and Cutout in VBS2


Road T-Junction


Road T-Junction Zoom


Road T-Junction in VBS2


Road T-Junction in VBS2


Preliminary Results: Annotated Environments

• Working with SimCentric and Bohemia Interactive• VBS2 1.60 with SimCentric Urban Ambience “add-ons”• Use TerraTools 4.0 to generate correlated:

– VBS2 maps and multi-tile .pbo– POL annotated data layers

• POL data layers are loaded into Urban Ambience• Current default behaviors include:

– Limiting walking to sidewalks and avoid street areas– Crossing roads within crosswalk areas– Entering and exiting buildings are door locations.– Milling activity within defined park and open areas


Entity Routing Without Annotated Environment


Entity Routing Without Annotated Environment


Entity Routing With SimCentric “Urban Ambience”


Entity Routing With SimCentric “Urban Ambience”


Crossing and Loitering Sequence






SimCentric UPOL Editor (Office Buildings - Blue)


SimCentric UPOL Editor (Restaurants - Purple)


SimCentric UPOL Editor (Residential - Pink)


Ongoing work

• Refine generation of annotated urban environments– Additional attention to structure semantic attribution (human)– Add segment definitions for roads (vehicular navigation)– Add traffic signs and lights using derived intersection areas– Add urban clutter (hydrants, garbage bins, newspaper box, etc.)– Work with additional urban datasets

• Work with Serious Game and AI middleware developers to flesh out requirements for Annotated Urban Environments


Pittsburgh Source Data – (Buildings)


Pittsburgh Source Data – (Roads)


Pittsburgh Source Data – (Parks)


Pittsburgh Source Data – (Drainage)


Pittsburgh Annotated Environment – (All Layers)


Pittsburgh POL Environment in VBS2 – Overview






Pittsburgh VBS2 – Gateway Center


Pittsburgh VBS2 – Gateway Center


Pittsburgh in VBS2 – Street View


Pittsburgh in VBS2 – Clutter Models


Pittsburgh in VBS2 – Clutter Models


Pittsburgh in VBS2 – Bus Shelter


Conclusions

• Annotated urban environments can be efficiently produced as an integral part of the serious game generation process

• They can be used to support improved AI behaviors particularly to support background urban “pattern of life”

• We believe they will have particular impact in making urban environments look less sterile and in supporting significantly more compelling training scenerios.

• Questions or comments?


TerraSim, Inc.

One Gateway Center,

Suite 2050

420 Fort Duquesne Blvd.

Pittsburgh, PA 15222

Visit us at:

http://www.terrasim.com

Documents

Automatic Annotation of Urban Environments to Support POL Behavior Automatic Annotation of Urban Environments to Support Pattern of Life Behavior Daniel