Introduction Indirect Reciprocity Proposed model Conclusions

Cooperation, Reputation & Gossiping

V.A. Traag1, P. Van Dooren1, Y.E. Nesterov2

1Department of Applied Mathematics

Universite Catholique de Louvain

2CORE

Universite Catholique de Louvain

27 October 2010

Introduction Indirect Reciprocity Proposed model Conclusions

Cooperation

Cooperation (and defection)

• Organizations (also Wikipedia, open source software, . . . )◮ Why do people contribute?

• Worker ants in colonies◮ Why do workers help without individual benefit?

• Prudents parasites in hosts◮ Why do parasites not replicate faster?

• Human body◮ Why do cells not replicate faster?

Central question

If defecting (not cooperating) is a real option, why (and how) hascooperation evolved?

Introduction Indirect Reciprocity Proposed model Conclusions

Formal cooperation (and defection)

Sort of Prisoner’s Dilemma

• The game knows two options, donating or not donating

• Donate b > 0 to someone at a cost c < b

• Agents are paired, and play a round of donating or not

• Cooperators C donate, defectors D do not donate

This can be summarized in the payoff matrix

A =

(C D

C b − c −c

D b 0

)

Defectors dominate

Whatever strategy you encounter (C or D), always better to defect

Introduction Indirect Reciprocity Proposed model Conclusions

Overview

What are possibly mechanisms to get cooperation? Payoff matrix

A =

(C D

C b − c −c

D b 0

)

Mechanisms

• Kin selection (r > cb)

Cooperate because offspring benefits from your cooperation.Basis of ‘selfish gene’, or ‘inclusive fitness’

• Direct reciprocity (w > cb)

Cooperate because of possible future payoffs

• Indirect reciprocity (q > cb)

Cooperate because someone else may cooperate with you in thefuture

Introduction Indirect Reciprocity Proposed model Conclusions

Some strategies

Example (Always)

Defect/cooperate on all rounds

Other CDDDDCC

AllD DDDDDDD

AllC CCCCCCCC

Example (Win-Stay, Lose-Shift)

Change strategy if losing, keep itotherwise

Other CDDDDCC

WSLS CCDCDCC

Example (Tit-for-tat)

Start cooperating, then imitateopponent

Other CDDDDCC

TFT CCDDDDC

Example (Generous Tit-for-tat)

As TFT, but cooperates afterdefection with probability p

Other CDDDDCC

GTFT CCDDCDC

Introduction Indirect Reciprocity Proposed model Conclusions

Insufficient explanation

Why is kin selection and direct reciprocity not sufficient?

Insufficient explanation

• Humans cooperate also with non-kin

• Humans cooperate in non-iterative situations

Indirect reciprocity

• Cooperate if cooperated with others in the past

• Brings reputation into play

• How to respond to reputation?

• How to determine new reputation?

Introduction Indirect Reciprocity Proposed model Conclusions

Indirect Reciprocity

Cooperate because others will return the favor

Simple reputation

• Cooperation increases reputation, defection decreases it

• Cooperate with those who have a good reputation

• Defect those who have a bad reputation

However. . .

• Many other possible interactions between cooperation andreputation.

• Should it be ‘bad’ or ‘good’ to cooperate with ‘bad’ agents?

• Should you cooperate only to increase your own reputation?

Introduction Indirect Reciprocity Proposed model Conclusions

Some reputation dynamics

Morals

• Defecting a defector: bad in image score

• What action should be regarded as good?

• When to cooperate, when to defect?

GG GB BG BB

C ∗ ∗ ∗ ∗

D ∗ ∗ ∗ ∗

∗ ∗ ∗ ∗

Reputation of donor and recipientAction of donor

New reputation can beeither Good or Bad

Action can be eitherCooperate or Defect

Introduction Indirect Reciprocity Proposed model Conclusions

Some reputation dynamics

GG GB BG BB

C G G G G

D B B B BImage scoring

C G G G G

D B G B BStanding

C G B G B

D B G B BJudging

C G B G B

D B B B BShunning

Introduction Indirect Reciprocity Proposed model Conclusions

Leading eight

Best strategies

• In total 2, 048 different possible strategies.

• There are 8 strategies (leading eight) that perform best (highestpayoff, and ESS)

GG GB BG BB

C G ∗ G ∗

D B G B ∗

C D C ×

Maintainance of cooperation

Mark defectors

Punish defectors

Forgive defectors

Apologize

Introduction Indirect Reciprocity Proposed model Conclusions

Subjective reputation

Subjective reputation

• Unrealistic that reputation is objective

• Introduce a subjective (private) reputation

• ‘Observe’ only a few interactions

Observing

• Probability q of observing an interaction

• Cooperation declines with lower q

• Diverging reputations cause further errors

• Good may defect bad, but not all agree on who’s bad

Introduction Indirect Reciprocity Proposed model Conclusions

Synchronize reputations

Synchronizing reputations

• Spread local information to synchronize reputations

• Players ‘gossip’ about each other to share information

• Start gossip, spread gossip and how to interpret gossip?

Lying, cheating and defecting

• Possibly ‘false’ gossips spread

• Spread rumours unconditionally allows liars to invade

• Liars cannot invade conditional rumour spreaders

Introduction Indirect Reciprocity Proposed model Conclusions

Proposed model

Basics

1 Each agent has a reputation of the other: Rij(m)

2 Everybody plays and cooperates/defects based on reputation

3 Everybody gossips the result of the interaction

4 New reputation Rij(m + 1) based on own observation and gossip

Decision to cooperate

• The decision to cooperate αij(m) (0 - Defect, 1 - Cooperate)

• Possibly logistic function (or others), but for now

αij(m) =

{0 if Rij(m) < 01 if Rij(m) ≥ 0

Introduction Indirect Reciprocity Proposed model Conclusions

Example

i j

k

Cooperate

Defect

Hmm. . .j defected

Introduction Indirect Reciprocity Proposed model Conclusions

Example

i j

kCooperate

Defect

Introduction Indirect Reciprocity Proposed model Conclusions

Example

i j

k

Cooperate

Cooperate

Introduction Indirect Reciprocity Proposed model Conclusions

Example

i j

k

Gossip D

efect

Did youknow j

defected?

So j defected k

also. And I likek! That bastard!

Introduction Indirect Reciprocity Proposed model Conclusions

Individual strategy

C D

C + −

D − +

Action of j , or αji (m).

Action of j , or αij(m)

Action is considered aseither Good or Bad

Individual strategy

• +1 for ‘good’ actions, −1 for ‘bad’ actions to reputation

• We currently study WSLS-like: Consider CC and DD as good.

∆Iij(m) = (2αij(m)− 1)(2αji (m)− 1)

Introduction Indirect Reciprocity Proposed model Conclusions

Gossiping

Consider all neighbours k when updating the reputation Sij

i j

k

The link tobe updated.

Does i ‘like’ k?

Will k gossip to i?

What actionhas j takento k?

Introduction Indirect Reciprocity Proposed model Conclusions

Social strategy

G B

C G B

D B G

Reputation of k , or αik(t).

Action of j , or αjk(t)

Action is considered aseither Good or Bad

Social strategy

• Cooperation vs. good agent and defection vs. bad agent is good

• Change in reputation due to gossiping with neighbour k

∆Sij(k ,m) = αki (m)(2αik(m)− 1)(2αjk(m)− 1)

Introduction Indirect Reciprocity Proposed model Conclusions

Reputation dynamics

Combine individual & social strategies

Combine with social influence parameter 0 ≤ λ ≤ 1

∆Rij(m) = (1− λ)

Individual strategy︷ ︸︸ ︷

(2αij(m)− 1)(2αji (m)− 1)+

λ1

n − 2

∑

k 6=i ,j

αki (m)(2αik(m)− 1)(2αjk(m)− 1)

︸ ︷︷ ︸

Social strategy

Reputation dynamics

Rij(m + 1) = Rij(m) + ∆Rij(m)

Introduction Indirect Reciprocity Proposed model Conclusions

Cooperative fixed points

Fixed point

• For which networks do we have αij(m + 1) = αij(m)?

• Positive reputation remains positive,negative reputation remains negative

Undirected case

• If αij(m) = αji (m), fixed points are groups

• Cooperate within groups, defect between groups

• Implies it is (weakly) social balanced

• Can have q groups if

λ >q

q + 1

Introduction Indirect Reciprocity Proposed model Conclusions

Evolutionary dynamics

Four different regimes (Cooperate with prob p on first round)

p < 1/2 p > 1/2

λ < 1/2 Individualistic prejudiced

• Defect vs. cooperators

• Cycles of cooperation vs.defectors

Individualistic trusting

• Cooperate vs.cooperators

• Cycles of cooperationvs. defectors

λ > 1/2 Social prejudiced

• Cooperate vs. cooperators(except second round)

• Defect vs. defectors(except second round)

Social trusting

• Cooperate vs.cooperators

• Defect vs. defectors

Introduction Indirect Reciprocity Proposed model Conclusions

Phase portraits Individual

C D

Gossipers

Individual PrejudicedC D

Gossipers

Individual Trusting

Introduction Indirect Reciprocity Proposed model Conclusions

Phase portraits Social

C D

Gossipers

Social PrejudicedC D

Gossipers

Social Trusting

Introduction Indirect Reciprocity Proposed model Conclusions

Conclusions

Proposed model

• Proposed model for gossiping and reputation dynamics

• Interesting possible cooperative network structure

• Evolutionary stable for some parameter range

• More socially oriented strategy could have developed fromindividual strategy

Shortcomings

• Actual convergence to fixed point not investigated

• Characterize directed fixed points

• Evolutionary dynamics investigated in limit of large n

• Interact all-to-all unrealistic, e.g. restrict to graph

• Gossip perhaps passed on further (cascades of gossip)