1. Meredith L. Patterson BSidesLV August 5, 2014 STRATEGIES WITHOUT FRONTIERS

2. I hate boring problems I especially hate solving tiny variations on the same boring problem over and over again The internet is full of the same boring problems over and over again Both in the cloud and in the circus Not my circus, not my monkeys MOTIVATION 3. Information theory Probability theory Formal language theory (of course) Control theory First-order logic Haskell ALSO APPEARING IN THIS TALK 4. When an unknown agent acts, how do you react? Observation of side effects Signals the agent sends Past interactions with others Formal language theory (if youre a computer) Systematic knowledge about the structure of interactions and the incentives involved in them IT IS PITCH BLACK. YOU ARE LIKELY TO BE EATEN BY A GRUE. 5. Everything You Actually Need to Know About Classical Game Theory in math and psychology Changing the Game Extensive form and signaling games Multiplayer and long-running games Reasoning Under Uncertainty, Over Real Data OUTLINE 6. EVERYTHING YOU ACTUALLY NEED TO KNOW ABOUT CLASSICAL GAME THEORY 7. Players Information available at each decision point Possible actions at each decision point Payoffs for each outcome Strategies (pure or mixed) Or behaviour, in iterated or turn-taking games Equilibria Different kinds of games have different kinds of equilibria WHATS IN A GAME? 8. a, b c, d e, f g, h A NORMAL FORM GAME Cooperate Defect Cooperate Defect 9. Pure strategy: fully specified set of moves for every situation Mixed strategy: probability assigned to each possible move, random path through game tree Behaviour strategies: probabilities assigned at information sets STRATEGIES 10. PRISONERS DILEMMA -1, -1 -3, 0 0, -3 -2, -2 Cooperate Defect Cooperate Defect d, e > a, b > g, h > c, f 11. MATCHING PENNIES 1, -1 -1, 1 -1, 1 1, -1 Heads Tails Heads Tails a = d = f = g > b = c = e = h 12. DEADLOCK 1, 1 0, 3 3, 0 2, 2 Cooperate Defect Cooperate Defect e > g > a > c and d > h > b > f 13. STAG HUNT 2, 2 0, 1 1, 0 1, 1 Stag Hare Stag Hare a = b > d = e = g = h > c = f 14. CHICKEN 0, 0 -1, 1 1, -1 -10, -10 Swerve Straight Swerve Straight e > a > c > g and d > b > f > h 15. HAWK/DOVE , 0, V V, 0 2 , 2 Share Fight Share Fight e > a > c > g and d > b > f > h 16. BATTLE OF THE SEXES 3, 2 0, 0 0, 0 2, 3 Opera Football Opera Football (a > g and h > b) > c = d = e = f 17. Games can be zero-sum or non-zero-sum Games can be about conflict or cooperation Actions are not inherently morally valenced Payoffs determine type of game, strategy WHAT HAVE WE SEEN SO FAR? 18. Cournot equilibrium: each actors output maximizes its profit given the outputs of other actors Nash equilibrium: each actor is making the best decision they can, given what they know about each others decisions Subgame perfect equilibrium: eliminates non- credible threats Trembling hand equilibrium: considers the possibility that a player might make an unintended move EQUILIBRIUM 19. TRANSACTIONAL ANALYSIS: GAMES PEOPLE PLAY 20. MIND GAMES As far as the theory of games is concerned, the principle which emerges here is that any social intercourse whatsoever has a biological advantage over no intercourse at all. 21. Procedures Operations Rituals Pastimes (Predatory) Games TYPES OF INTERACTIONS 22. Hands or roles = players Extensive form; players move in response to each other Advantages Existential advantage: confirmation of existing beliefs Internal psychological advantage: direct emotional payoff External psychological advantage: avoiding a feared situation Internal social advantage: structure/position with respect to other players External social advantage: as above, wrt non-players BERNES GAMES: STRUCTURE 23. Kick Me Goal: Sympathy Find someone to beat on you, then whine about it My misfortunes are better than yours Aint It Awful Can be a pastime, but also manifests as a game Player displays distress; payoff is sympathy and help Why Dont You Yes, But Player claims to want advice. Player doesnt really want it. Goal: Reassurance BERNES GAMES: EXAMPLES 24. Now Ive Got You, You Son Of A Bitch Goal: Justification (or just money) Three-handed version is the badger game Roles Victim Aggressor Confederate Moves Provocation Accusation Defence Accusation Defence Punishment THE BADGER GAME 25. Schlemiel, in Bernes glossary Moves: Provocation resentment (repeat) If B responds with anger, A appears justified in more anger If B keeps their cool, A still keeps pushing TROLLING 26. Social media Organic responses against predatory games Predator Alert Tool /r/TumblrInAction known trolls wiki Those just happen to be ones I know about A truly generic reputation system is probably a pipe dream Wikipedia eBay But for these, we have to extend the basic mathematical model. OTHER MONKEY GAMEBOARDS 27. DISSECTING A SIGNALING GAME 28. THE SETUP 29. THE TYPE Split Steal 1 30. BOTH SPLIT 31. BOTH SPLIT Split Steal 1 1 1 A B Split Split 2 2 6800, 6800 6800, 6800 32. ONE SPLITS, ONE STEALS 33. ONE SPLITS, ONE STEALS Split Steal 1 1 1 A B Split Split 6800, 6800 6800, 6800 2 2 A Split 2 Steal Steal B Split 2 0, 13600 0, 13600 13600, 0 13600, 0 34. BOTH STEAL 35. BOTH STEAL Split Steal 1 1 1 A B Split Split 6800, 6800 6800, 6800 2 2 A Split 2 Steal Steal B Split 2 0, 13600 0, 13600 13600, 0 13600, 0 Steal Steal 0, 0 0, 0 36. NORMAL FORM Also known as the Friend-or-Foe game. 1, 1 0, 2 2, 0 0, 0 Split Steal Split Steal d = e > a = b > c = f = g = h 37. OBSERVATION 38. FIRST MOVE: NICKS CHOICE Split Steal 1 1 1 Im likely to split Im likely to steal Split Split 6800, 6800 6800, 6800 2 SplitSteal Steal Im likely to steal Split 0, 13600 0, 13600 13600, 0 13600, 0 Steal Steal 0, 0 0, 0 Im likely to split 2 39. SIGNALING 40. SECOND MOVE: NICKS SIGNAL Split Steal 1 1 1 Im likely to split Im likely to steal Split Split 6800, 6800 6800, 6800 2 SplitSteal Steal Im likely to steal Split 0, 13600 0, 13600 13600, 0 13600, 0 Steal Steal 0, 0 0, 0 Im likely to split 2 41. THE BIG REVEAL 42. THE COMPLETE PATH Split Steal 1 1 1 Im likely to split Im likely to steal Split Split 6800, 6800 6800, 6800 2 SplitSteal Steal Im likely to steal Split 0, 13600 0, 13600 13600, 0 13600, 0 Steal Steal 0, 0 0, 0 Im likely to split 2 43. GAMES IN THE TRANSPARENT SOCIETY 44. Strategies now depend on payoff matrix and history Axelrod, 1981: how well do these strategies perform against each other over time? Ecological tournaments: players abandon bad strategies Rapoport: if the only information you have is how player X interacted with you last time, the best you can do is Tit-for-Tat TFT cannot score higher than its opponent Axelrod: Dont be envious Against TFT, no one can do better than cooperate Axelrod: Dont be too clever ITERATED GAMES 45. Nice: S is a nice strategy iff it will not defect on someone who has not defected on it Retaliatory: S is a retaliatory strategy iff it will defect on someone who defects on it Forgiving: S is a forgiving strategy iff it will stop defecting on someone who stops defecting on it PROPERTIES 46. Ord/Blair, 2002: what happens when strategies can take into account all past interactions? We can express strategies in convenient first-order logic, as it turns out Tit-for-Tat: D(c, r, p) Tit-for-Two-Tats: D(c, r, p) D(c, r, b(p)) Grim: t D(c, r, t) Bully: t D(c, r, t) Spiteful-Bully: t D(c, r, t) s (D(c, r, s) D(c, r, b(s)) D(c, r, b(b(s)))) Vigilante: j D(c, j, p) Police: D(c, r, p) j (D(c, j, p) k(D(j, k, b(p))) SOCIETAL ITERATED GAME THEORY 47. EVOLUTION IS A HARSH MISTRESS Tit-for-Tat All-Cooperate Spiteful-Bully 48. PEACEKEEPING Police All-Cooperate Spiteful-Bully 49. In a society, niceness is more nuanced Individually nice: will not defect on someone who has not defected on it Meta-individually nice: will not defect on individually nice Communally nice: will not defect on someone who has not defected at all Meta-communally nice: will not defect on communally nice Same applies to forgiveness and retaliation Loyalty: will not defect on the same strategy as itself NICENESS AND LOYALTY 50. Peacekeepers dont always agree Police will defect on Vigilantes and vice versa Peacekeepers protect non-peacekeeping strategies at their own expense META-PEACEKEEPING Police All-Cooperate Spiteful-Bully Tit-for-Tat 51. REDUCTIO AD ABSURDUM: ABSOLUTIST t j D(r, j, t) D(c, j, t) Tit-for-Tat All-Cooperate Spiteful-Bully Absolutist 52. ABSOLUTISM UBER ALLES Tit-for-Tat All-Cooperate Spiteful-Bully Absolutist 53. REASONING UNDER UNCERTAINTY 54. Frequentist: probability is the long-term frequency of events Reasoning from absolute probabilities What happens if an event only happens once? Returns an estimate Bayesian: probability is a measure of confidence that an event will occur Reasoning from relative probabilities Returns a probability distribution over outcomes Update beliefs (confidence) as new evidence arrives TWO INTERPRETATIONS OF PROBABILITY P(A|X) = P X A P(A) P(X) 55. Probability distribution function: assigns probabilities to outcomes Discrete: a finite set of values (enumeration) Function also called a probability mass function Poisson, binomial, Bernoulli, discrete uniform Continuous: arbitrary-precision values Function also called a probability density function Exponential, Gaussian (normal), chi-squared, continuous uniform Mixed: both discrete and continuous Narrower distribution = greater certainty DISTRIBUTIONS = = 1 56. Game theory is great when you know the payoffs What can you do if you dont know the payoffs? Or what the game tree looks like? Well You usually have some educated guesses about who the players are You have some idea what your possible actions are, as well as the other players You can look at past interactions and make inferences Which of these can be random variables? All of them. Deterministic: if all inputs are known, value is known Stochastic: even if all inputs are known, still random YOU DONT KNOW WHAT YOU DONT KNOW 57. Figure out what distribution to use Figure out what parameter you need to estimate Figure out a distribution for it, and any parameters Observing data tells you what your priors are Fixing values for stochastic variables Markov Chain Monte Carlo: sampling the posterior distribution thousands of times DONT WAIT SIMULATE 58. Prerequisites: A Markov chain with an equilibrium distribution A function f proportional to the density of the distribution you care about Choose some initial set of values for all variables (state, S) Modify S according to Markov chain state transitions If f(S)/f(S) 1, S is more likely than S, so accept Otherwise, accept S with probability f(S)/f(S) Repeat CONVERGING ON EXPECTED VALUES 59. A GAME WITHOUT PAYOFFS type Outcome = Measure (Bool, Bool) type Trust = Double type Strategy = Trust -> Bool -> Bool -> Measure Bool tit :: Trust -> Bool -> Bool -> Measure Bool tit me True _ = conditioned $ bern 0.9 tit me False _ = conditioned $ bern me 60. CHOOSING WHICH HOLE TO FILL IN play :: Strategy -> Strategy -> (Bool, Bool) -> (Trust, Trust) -> Outcome play strat_a strat_b (last_a,last_b) (a,b) = do a_action Measure Bool allDefect _ _ _ = conditioned $ bern 0.9 grimTrigger :: Trust -> Bool -> Bool -> Measure Bool grimTrigger me True False = conditioned $ bern 0.9 grimTrigger me False False = conditioned $ bern 0.1 grimTrigger me _ True = conditioned $ bern 0.9 64. STRATEGY AS A RANDOM VARIABLE data SChoice = Tit | GrimTrigger | AllDefect | AllCooperate deriving (Eq, Ord, Enum, Typeable, Show) chooseStrategy :: SChoice -> Strategy chooseStrategy Tit = tit chooseStrategy AllDefect = allDefect chooseStrategy AllCooperate = allCooperate chooseStrategy GrimTrigger = grimTrigger strat :: Measure SChoice strat = unconditioned $ categorical [(AllCooperate, 0.25), (AllDefect, 0.25), (GrimTrigger, 0.25), (Tit, 0.25)] 65. LETS PLAY ANOTHER GAME iterated_game2 :: Measure (SChoice, SChoice) iterated_game2 = do let a_initial = False let b_initial = False a