Agent-based modeling of cooperation in collective action

situations & diffusion of informationMarco Janssen

School of Human Evolution and Social Change&

Department of Computer Science and Engineering

Arizona State University

Games and GossipMarco Janssen

School of Human Evolution and Social Change&

Department of Computer Science and Engineering

Arizona State University

Games and Gossip

• Games: Strategic interactions

• Gossip: Diffusion of information

Agent-based modeling is a way to study the interactions of large numbers of agents and the macro-level consequences of these interactions.

Organizations of agents

Animate agents

Data

Artificial world

Observer

Inanimate agents

If <cond>

then <action1>

else <action2>

If <cond>

then <action1>

else <action2>

…..

…..

….

Content

• Games– Why do we cooperate with strangers?– Changing the rules of the game

• Gossip– Diffusion of consumer products

Why do strangers cooperate?

• Dilemma between individual and group interests– Group interest: cooperation– Individual interest: free riding on efforts of others

• Public goods and common pool resources• Expectation with rational selfish agents

– No public goods– Overharvesting of common pool resources

• Many empirical examples of self-governance

The problem of cooperation in commons dilemmas

The puzzle of eBay

• Net revenues $2.2 billion for 2003.• In eBay strangers cooperate in non-repeated interactions of

traditional dilemma of buyer and seller.• Reputation system is found to be theoretically problematic

(aggregation, unlimited memory, entry problem)• Monitoring is incomplete

– About 55% of transactions include feedback.– About 1% of this feedback is negative.

• 90% of fraud on internet occurs in auction markets.• Puzzle: Why does eBay work?

eBay reputation system

• Buyer and Seller can provide “Feedback”: • Ratings translated into points: positive = 1

point, neutral = 0 points, and negative = -1 point. Aggregate is the reputation score.

• If reputation score reaches -4 the participant is removed from the system.

Simple model on reputation and trustworthiness

• Agents play one-shot prisoner dilemma games.• Reputation scores evaluates past behavior of the

actors.• Are reputation scores alone sufficient to derive

cooperation?• Especially, when not everybody provides feedback.• They may refuse to play and decide to cooperate or

not, based on expected trustworthiness.

Monetary payoff table of the Prisoner’s Dilemma with the option to withdraw from the game.

Player B

Cooperate Defect Withdraw

PlayerA

Cooperate 1,1 -2,2 0,0

Defect 2,-2 -1,-1 0,0

Withdraw 0,0 0,0 0,0

• Experiments have shown that the subjective evaluation of monetary payoffs lead to a different order of preferred situations than monetary rewards.

• Thus, utility and monetary rewards may differ.

Utility table of the Prisoner’s Dilemma with the option towithdraw from the game.

Player B

Cooperate Defect Withdraw

PlayerA

Cooperate 1,1 -2+βA, 2-αB

0,0

Defect 2-αA,-2+βB

-1,-1 0,0

Withdraw 0,0 0,0 0,0

α and β are individual characteristics of agents

How to estimate trustiness?

• The probability to trust the opponent:

• Where

• Adjusting weightings of symbols:

MeTr

1

1]Pr[

s

iii xwwM

10

Learning rate

Symbol i

Feedback (0 or 1)

,])Pr[( ii xTrFw

When to Cooperate?

• Estimate expected utilities:

Make discrete choice decision:

)]([)]([

)]([

][DUECUE

CUE

ee

eCP

)(])Pr[1(]Pr[)]([ iSTrRTrCUE

PTrTTrDUE i ])Pr[1()(]Pr[)]([

Population dynamics

• Agent remove from the system if they do not derive positive income, or when reputation score falls to -4.

• Agent is replaced with a random new one.

• Agents provide feedback with a certain probability.

Role of feedback(history 100 interactions)

0

0.2

0.4

0.6

0.8 1

0

0.3

0.6

0.9

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

average payoff

probability feedback "positive"

probability feedback "negative"

0.9-1

0.8-0.9

0.7-0.8

0.6-0.7

0.5-0.6

0.4-0.5

0.3-0.4

0.2-0.3

0.1-0.2

0-0.1

Role of symbols

0

0.2

0.4

0.6

0.8 1

0

0.3

0.6

0.9

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

average payoff

probability feedback "positive"

probability feedback "negative"

0.9-1

0.8-0.9

0.7-0.8

0.6-0.7

0.5-0.6

0.4-0.5

0.3-0.4

0.2-0.3

0.1-0.2

0-0.1

Finding

• Reputation systems with voluntary feedback might not be sufficient to foster cooperation.

• Observed high levels of cooperation might be explained by the use of multiple other sources of indicators of trustworthiness.

Changing the rules of the game

• Earlier work has focused on behavior of individuals and groups given a particular rule set, and what happens when this rule set changes.

• I am interested in how people change the rule of the game.

Questions on rule change

• How do individuals and groups know the potential effect of a rule change?

• What affect that persons invest in a rule change?

• What is the role of experience in rule crafting?

Using different type of methods

http://www.public.asu.edu/~majansse/dor/nsfhsd.htmDynamics of Rules project:

Laboratory Experiment

-Renewable resource-Collection of green tokens- 5 subjects: self is yellow dot; and other subjects are blue dots- move yellow dot around by arrow keys

Design

• For each treatment, a practice round and then 3 rounds of about 5 minutes.• Treatments:

– no rules – vote for rule (cost 50 tokens) – no rule (22 groups, 2 groups discarded)

– No rules for three rounds (4 groups, need more done later)– Rule imposed in 2nd round (9 groups)

• Totally 174 different subjects used (one person did an experiment twice)• In communication experiment we asked 30 persons to do it a second time.

Information collected

• Everytime a subject collect a token, the time, and place are recorded.

• Every 2 seconds the location of all tokens is recorded.

• When subjects break the rule and/or are caught (place and time)

• Questionaire at end of experiment.

What happens?

Round 1

0

50

100

150

200

250

300

350

400

450

500

2 32 62 92 122 152 182 212 242seconds

reso

urc

e s

ize

no rule

imposed

YES

NO

Effect of experience

0

50

100

150

200

250

300

350

400

450

500

2 32 62 92 122 152 182 212 242seconds

reso

urce

siz

e

no rule

imposed

YES

NO

experienced

Small but significant high collection of tokens and length of time

Round 2

0

50

100

150

200

250

300

350

400

450

500

2 32 62 92 122 152 182 212 242 272 302seconds

reso

urce

siz

e

Round 1

no rule

imposed

YES

NO

How much tokens collected? (including penalties)

0

200

400

600

800

1000

1200

1400

1600

no rule imposed yes notreatment

toke

ns c

olle

cted

in r

ound

1

2

3

How fast do they destroy the resource?

0

50

100

150

200

250

300

350

no rules imposed yes no

treatment

seco

nd

s re

sou

rce

exi

st

1

2

3

Average collected earnings of individuals

0

50

100

150

200

250

300

350

voted no (no) voted no(yes)

votes yes(no)

votes yes(yes)

1

2

3

Where did they break the rules?

0

100

200

300

400

500

600

700

1 4 7 10 13 16 19cells away from property

toke

ns

sto

len

Individual collected tokens in round 2 and 3

0

100

200

300

400

500

600

0 100 200 300 400 500 600

round 2

rou

nd

3

0

100

200

300

400

500

600

0 100 200 300 400 500 600

round 2

rou

nd

3

Not elected

Elected

0

100

200

300

400

500

600

0 100 200 300 400 500 600

round 2

roun

d 3

Imposed

0

100

200

300

400

500

600

0 100 200 300 400 500 600

round 2

rou

nd

3

No rule

CommunicationExperiment for designing future experiments

• Treatment 1: All three groups could communicate within one big group

• Treatment 2: The three groups split up and could talk among themselves.

• Experienced subjects!!

Global CommunicationAgreed Rule: 20 seconds wait, 10 seconds “go

for it”

0

100

200

300

400

500

600

700

800

2 32 62 92 122 152 182 212 242 272 302

seconds

reso

urc

e s

ize

123123123

Group talk:Areas of harvest

0

100

200

300

400

500

600

700

800

2 32 62 92 122 152 182 212 242 272 302

seconds

reso

urc

e s

ize

1

2

3

1

2

3

1

2

3

Next steps

• Analysis of data

• Development of agent-based models

• New experimental designs

Fun project• Why do recreational games have the rules they

have?

• Co-evolution of agents playing games and changing the rules such that certain objectives are derived (excitement of playing?).

EvaluationAgentsPlayGames(Tournament)

Adjustment of rules

Rules of tournaments

Diffusion dynamics in various types of social networks with heterogeneous consumers

with Alessio Delre & Wander Jager (University of Groningen, the Netherlands)

- How do network structure affect diffusion of consumer products?

- How do behavioral rules of consumer behavior affect diffusion processes? (Most models assume diffusion is a kind of epidemic spreading of a disease, we use cognitive theories)

Regular network (randomness = 0)

Random network (randomness = 1)

Small-World network (0 < randomness < 1)

Watts, D.J. and Strogatz S. H. (1998). Collective Dynamics of “Small-World” Networks, Nature, 393, 440-442.

Small-World Networks

Our innovation diffusion model

iiiiij yxU )1(

ii Afx )( , jii qpfy

1

0

otherwise

hAx

iii

0

1

otherwise

qpy

jii

Individual part:Social part:

where Ai is the number of adopters in set of neighbors of agent i

hi is a personal threshold which determines when agent i adopts.

)( ,,, MINijiji UUPa

P.S. Notice that we included mass media effects. Independently on word-of mouth process, at each time step, agents adopt with probability e.

Results -the speed of diffusion-ßi =1;

hi=0.3;

T

t

T

t

tf

tD

T

0

0

)(

)(1

D(t) = cumulative number of adopters;

f(t) = adopters at time t

0.3

0.4

0.5

0.6

0.7

0.8

0.0001 0.001 0.01 0.1 1

r

rho

Results -the speed of diffusion in heterogeneous populations-

0.4

0.5

0.6

0.7

0.8

0.9

1

0.04 0.06 0.08 0.1 0.12 0.14 0.16

std dev

rho

Continuous line: <hi>=0.4;

Dashed line: <hi>=0.3;

Pointed line: <hi>=0.2.

Application: hits and flops of movies

• What makes a movie a hit? Spread of information?• Most movies have their most successful week in the

first week.• Only in rare cases there is an increase after the first

week.• Same phenomena with best seller books (Harry

Potter).• Expectations are formed by media campaign before

the product is available.• Survey data from movie-goers (challenging

fieldwork!!)