35
Effect of Information on Collusion Strategies in Single winner, multi-agent games December 2, 2010 Nick Gramsky Ken Knudsen

Effect of Information on Collusion Strategies in Single winner, multi-agent games

Embed Size (px)

DESCRIPTION

Effect of Information on Collusion Strategies in Single winner, multi-agent games. December 2, 2010 Nick Gramsky Ken Knudsen. Contents. 1. Motivation 2. Identification of Collusion 3. Classification of Coalitions 4. Implementation 5. Results 6. Conclusions. Motivation. - PowerPoint PPT Presentation

Citation preview

Page 1: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Effect of Information on Collusion Strategies in

Single winner, multi-agent games

December 2, 2010  

Nick GramskyKen Knudsen

Page 2: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Page 3: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Motivation

Explicit Collusions Alliances Survival Truces

Implicit Collusions Minimax against strongest

player Tit-for-tat

Reasons to Collude Improve position relative to other agent(s) Self-preservation / Survival

Page 4: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation   5. Results

6. Conclusions

Page 5: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Identification

Find course grained collusive behavior

1. Offensive-based collusion Multiple agents attacking a single agent for a fixed

number of rounds In our examples, we limited this to 1 round.

2. Defensive-based collusion Multiple agents not attacking each other over a fixed

number of rounds. In our examples, we limited this to 2 rounds.

Page 6: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

IdentificationOffensive based coalitions

Page 7: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

IdentificationDefensive based coalitions

Page 8: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Page 9: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

1. Socially inclined behavior For some predefined time, if target satisfies the

following, then we define the actions of the attacking players as being 'socially oriented‘

h(x) is a heuristic function for any adversary. vh(x) when dealing with different layers of fog

2. Else: Some other collusive behavior

Classification Offensive based behaviors

Page 10: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Classification Offensive based algorithm

Page 11: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Classification Defensive based algorithm

Page 12: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ClassificationMissed opportunities 

Classify a missed opportunity by finding players that: for a predefined period were not attacked

above a certain percentage and… satisfy either their power heuristic or visual

heuristic (below) threshold

Page 13: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Page 14: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Implementation

Used Warfish to play games of Risk. Free website warfish.net

  Risk is a zero-sum game where players seek (simulated) world

domination! 

Only one winner, the last remaining contestant.

Attacks are made via dice (random number generator)

Amass armies, grow in power, rule the world! Or at least the world represented on a board...

Page 15: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ImplementationEnvironment

Reduced resource strategies

Randomized players

Set card trade-in values to be constant (5)

Disabled card capture on elimination

Multiple map types Larger than original Risk board Reduces board specific strategies in analysis

Page 16: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ImplementationWorld Map

Page 17: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ImplementationEurope Map

Page 18: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ImplementationFog of War

Varied amount of information available to all agents via different levels of 'fog of war'.

6 different levels of fog available in game Level 0: No fog (perfect information) Level 1: See all occupations, neighboring units only Level 2: See all occupations (no units) Level 3: Only see neighboring occupations and units Level 4: See only neighboring occupations Level 5: Complete fog (only know about self)

Tested with 3 levels of fog {0,1,3}

Page 19: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ImplementationOracles

Participants who annotated their strategies and behaviors as games were played

Compared oracle annotations to game data Spot-check that analysis found collusion Though noisy, analysis and annotations were

inline with game history.

Page 20: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Page 21: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsCollusion vs Game length

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 per 1 turn window

Page 22: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsOffensive

1. Players all gang up on Yellow.

2. Validated by Oracle annotations.

Game: 98478150 Map: World Fog Level: 1

Page 23: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsOffensive

1. Minmax against Blue

2. Confirmed by reading through the transcript.1. Blue quickly gained

power

2. Challenged remaining players to team up against him Game: 97976903

Map: Europe Fog Level: 0

“Right now (Yellow) knows that if he does not get both you (Red) and (Green) on his side, this game will be won by me”

Page 24: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsOffensive

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 / 1 turn window

Games 98478150 (left) and 97976903 (right)

Page 25: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsOffensive & Defensive

1. Minimax against strongest player

2. Towards the end of the game, explicit truce between top 2 players

Game: 12069561 Map: Europe Fog Level: 0

Page 26: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Scatter plot of number of windows classified as defensive-oriented for all games.x-axis: number of turns y-axis: number of interesting windowsθ = 0.05

*Game: 12069561

ResultsDefensive

Page 27: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsOracle

1. Oracle self-interest annotations (Blue)

Game: 88318444 Map: World Fog Level: 1

x-axis: Number of turnsy-axis: Number of "interesting" windowsθh = 1.3 / 1 turn window

Page 28: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsFog Level 3

1. Typical of the layer 3 games.

2. Everything breaks down. Players can’t figure out who is in the lead until it is too late.

Game: 67785982 Map: Europe Fog Level: 3

Page 29: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Results

Collusion % is percentage of available windows where remaining players direct more than 75% of attacks towards target.

Social % is percentage of available windows with same criteria as above BUT the target satisfies heuristic thresholds from earlier

θh = 1.3 / 1 turn window

Target’s residual power 43.3% (4-player) 65% (3 player)

θh = 1.6 / 1 turn window

Target’s residual power 53.3% (4-player) 80% (3 player)

Page 30: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsEurope Map

θh = 1.3 θh = 1.6

Page 31: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ResultsWorld Map

θh = 1.3 θh = 1.6

Page 32: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Contents

1. Motivation

2. Identification of Collusion

3. Classification of Coalitions

4. Implementation

5. Results

6. Conclusions

Page 33: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Conclusions

Presented a basic algorithm to identify and classify collusion

Games with unusually large number of collusive behaviors tended to prolong games beyond the average.

As fog increased (information decreased), collusive behaviors diminished.

Results were consistent across maps.

Level 0 data was consistent between our volunteers and the public.

Analysis supported by Oracle annotations and in-game conversations.

Page 34: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

Conclusions

Visual heuristic does not hold well for fog games Based on a knowledge of territories and bonuses

Limited data sets Time limitation

Short time-frame for project Games averaged 20 days to complete

Require more experiments with fog levels

Data integrity Games had large variance in player abilities Players were involved in multiple simultaneous games

May have forgotten strategy Players may have a predefined disposition towards other

players (Social Value Orientation)

Page 35: Effect of Information on Collusion Strategies  in Single winner, multi-agent games

ConclusionsFuture Work

Investigate possible equilibrium in collusions versus game length.

  Lag response for social orientation.

Once the strongest player is removed from power, it can take a few rounds for the coalition to change strategies.

As information decreases, agents tend to collude less.  Why? fairness poor assessment of board

Mix socially oriented bots with human players