Identifying Physical Team Behaviors from Spatial Relationshipscc.ist.psu.edu/BRIMS/archives/2005/presentations/05-BRIMS-026.pdf · – range of scale factors for which the model is

14th BRIMS Conference16-19 May 2005Copyright 1998 Institute for Simulation & Training

Identifying Physical Team Behaviors from Spatial Relationships

Gita SukthankarKatia Sycara

Robotics InstituteCarnegie Mellon University


Team Behavior Recognition• Team behavior recognition is the ability to

recognize a team’s high-level intention from sequences of low-level actions executed by the team members.

• In MOUT, many team behaviors are physical in nature (e.g., patrolling an area, moving in formation) and have distinctive spatial characteristics.

• How can we efficiently and robustly identify these spatial relationships between MOUT entities?


Recognition without Spatial Landmarks

Spatial landmarks such as doors and intersections are importantcues for analyzing MOUT team behaviors.

Doorway


Problem: Scene Analysis• Team of soldiers

(blue) about to enter building

• Sniper (red) hiding in the trees

• Commonly occurring patterns can be difficult to recognize in cluttered environment






Building entry






Likely sniper position






Same behavior, different orientation






Cluttered environment


Related Work

• Spatial recognition of multi-agent behaviors– Football (Intille & Bobick, 1999)– Robocup soccer (Riley & Veloso, 2002)

• MOUT spatial team plan representations– ACT-R (Best and Lebiere, 2003)– SOAR (Pearson and Laird, 2004)


Constructing the Model

Model: Building Entry

Overhead Map

Unreal Tournament view



Model

Map

Construct point-basedversion of map withlines of visibility andconcealment


Spatial Representation• Behavior name • Spatial position (x,y) of entities • Entity type:

– person (unknown), civilian, teammate, opponent, hard cover, softcover, empty area, window, intersection, doorway, hazard, objective

• Pairwise constraints– visibility: points must be connected– occlusion: points across this barrier cannot be connected

• Scaling limitations– range of scale factors for which the model is valid

Spatial models and behaviors don’t have a one-to-one correspondence; each behavior can have multiple models.



Model

Map


Exhaustive Template Matching

Model

Map

Slide model acrossmap checking to seeif the distance metric falls beneath threshold


Model

Map





Model

Map




Model

Map




Model

Map



Model

Map

Also have to checkdifferent scales androtations



Model

Map




Model

Map




Model

Map

Time-consumingexhaustive search



Random Sampling Consensus

Model

Map

Generate and testhypothesized transforms to projectthe model to the map

Fischer & Bolles, 1981



Model

Map

Randomly selecttwo points (a minimalset from the model)



Model

Map

Randomly select 2corresponding pointsof compatible typesfrom the map.



Model

Map

Calculate T, thesimilarity transform,from the pointcorrespondences.



Model

Map

Use the transformT to project remainingmodel points to themap coordinate frame



Model

Map

Score the hypothesisT based on whetherpoints of compatibletypes fall withinthe distance threshold



Model

Map

3 potential matches(marked in green)found. Normalized modelscore=3/5



Model

Map

Iteration 2:Choosea new minimal set



Model

Map

Iteration 2:Randomly select 2corresponding pointsof compatible typesfrom the map.



Model

Map

Iteration 2:Calculate T’, thesimilarity transform,from the pointcorrespondences.



Map

Model

Iteration 2:Use the transformT’ to project remainingmodel points to themap coordinate frame



Map

Model

Iteration 2:Score the hypothesisT’ based on whetherpoints of compatibletypes fall withinthe distance threshold



Map

Model

Iteration 2:1 potential match(marked in green)found. Normalized modelscore=1/5



Model

Iteration 2:New score (1/5) is lessthan previous score(3/5). Our previouslygenerated hypothesis,T, remains the best.



Map

Model

Iteration 2:1 potential match(marked in green)found. Normalized modelscore=1/5


Model Misalignment

Model aligned with best fitting transform, T,scores highly (5/5)

Same model aligned with poor transform,T’, scores poorly (2/5)

We score the model according to the best fitting transform. To determine whether the model is a valid fit, we check the score to see if it falls above a threshold


Calculating the Similarity Transform

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

100)cos()sin(

)sin()cos(ysxs

T θθθθ

)},(),,{( 2211 yxyx)},(),,{( 2211 YXYX

1

321

321

321

321

333231

232221

131211

111111

−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡yyyxxx

YYYXXX

ttttttttt

Model

Map


Calculating the Similarity Transform

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

100)cos()sin(

)sin()cos(ysxs

T θθθθ

1213 yyxx −+=

2113 xxyy −+=

1213 YYXX −+=

2113 XXYY −+=

)},(),,{( 2211 yxyx)},(),,{( 2211 YXYX

1

321

321

321

321

333231

232221

131211

111111

−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡yyyxxx

YYYXXX

ttttttttt

Model

Map

Virtual point


Spatial Model Matching Interface

Models of team spatialrelationships (right panels)

Blue circles: teammatesGreen circles: neutralsWhite squares: entry points;Orange triangle: hazard

Annotated overhead situation map (left panel)


Example: Opponent Modeling

Model marked in right hand panel scored over threshold; the model combinedwith the highest scoring transform give us a prediction of where an enemy(marked in red) is likely to be found.


Example: Identifying Flanking





Matching fails because the spatial relationship between elementsin the map is different from the idealized model.


Advantages of RANSAC Search• Faster than exhaustive template matching• Invariant to many spatial transformations• Robust to occlusions (missing points) and

outliers• Generates an explicit hypothesis of the location

of missing scene elements based on the projection of the template

• Can be calculated in batch across multiple models

• Disadvantage: probabilistic algorithm is not guaranteed to find all possible matches


Conclusion

• Our approach is:– Efficient– Accurate

• Assuming 95% of the points are outliers, RANSAC will find the best match with 99% probability in 1840 iterations

• In 3D, approximately 40000 iterations are required.– Robust to missing points and outliers

Documents

Identifying Physical Team Behaviors from Spatial Relationshipscc.ist.psu.edu/BRIMS/archives/2005/presentations/05-BRIMS-026.pdf · – range of scale factors for which the model is