Embed Size (px)
Citation preview
© 2011 CTG, Inc.
Playing around with
Risks
Jurgen Cleuren
Nov. 24th 2011
Introduction
• Projects are done in a probabilistic environment
– Incomplete information
– Parameters change over time
– What is true in the beginning of the project, can be false some time
later
• Games in a probabilistic environment
– Incomplete information (cardgames,…)
– Random element (dice, cards,…)
Which games ?
• Poker
• Monopoly
• Backgammon
Succes:
First learn the rules, then play better than everyone else
- Albert Einstein -
• Rules -> requirements
• Every game starts with learning and agreeing on the rules -> every
project starts with defining and agreeing on the requirements
TEXAS HOLD‟EM POKER
Rules of Texas Hold‟em
• 2 Blind cards
• Betting round
• 3 Open Community cards (flop)
• Betting round
• 1 Open Community card (Turn)
• Betting Round
• 1 Open Community Card (River)
• Betting Round
• Best hand of 5 cards wins
Hand Ranking
• Highest Card
• Pair
• 2 Pair
• 3 of a kind
• Straight (5 consecutive cards)
• Flush (5 cards of the same suit)
• Full House (3 and 2 )
• 4 of kind
• Straight Flush (5 consecutive cards of the same suit)
General test flow vs Poker round
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
Similarities / Differences
3 Recurring phases
1 big phase (Complete Functional testing vs Flop)
2 small phases (retest + regression vs Turn + River)
Determine Risk before 1st test run (Risk Matrix or FTT vs 1st bet)
Adapt Risk Matrix or FTT after each test run
Should the FTT or Risk Matrix be adapted ?
• Every Test run gives new information
• Likelihood of risks change
– Failed test cases get a higher likelihood
– Passed test cases in unchanged code have a lower likelihood
• Closer to deadline => Risks can change
• Reporting is more clear
– Each Risk Matrix/FTT tree is a snapshot of the project at a certain time
Who ?
• Test manager should take the lead
• Preferably Project Board
• Test manager can do it by himself, but the board should at least be aware
that this activity is done
Justification
• A Poker hand needs justification at any point in the game
– Having the best hand in the beginning doesn‟t imply that you are going
to win
– You must be prepared to fold when you are not winning anymore
– The money you‟ve already bet doesn‟t count
• It is in the pot so it is not your money anymore
• Don‟t put more money in a losing hand
– No justification anymore: get out of the hand
– Cut your losses
Project justification
• A project needs justification at any point in time
• During testing: justification after each test run
– New information is given
– Risks are updated
• No Justification means project should be closed
• Previous investments do not count
– Don‟t put more money in a failing project
• Cut your losses
Test process justification
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Justification
Result Oriented Thinking
• A decision can be right even if the outcome is not favourable
• Focus more on the decision and the „why‟ instead of the result
– Tester A completes a test set in 4 days by skipping tests
– Tester B completes the same test set in 6 days through thoroughly
testing
– No defects were discovered in production
– Who would you reward the most ?
Thought process
• In poker, to anticipate and understand your opponents actions, you need
to think as your opponent and not as you in his situation
• What motivates you, does not necessarily motivate another person
• Successful people ask better questions
– WHY? is more important than HOW? or WHO?
“Someone who knows HOW will always have a job
Someone who knows WHY will always be his boss”
Luck
“ Luck is when preparation meets opportunity”
- Seneca –
• Be prepared to get lucky
– In poker, sometimes you need to get lucky. When you get lucky, be sure
to take a big pot out of it.
– Sometimes a best case scenario happens, but we need to take
advantage of it.
– Being ahead of planning is more valuable if you can actually do
something with this situation
MONOPOLY
Aspects of the game
• Investing in houses
• The higher the investment, the bigger the payoff
• Some spaces have a higher probability
• Random element: dice
Analogy to Risk Based Testing
• Different probability of landing on spaces = Likelihood
• Higher Payoff = Impact
• What can monopoly teach us ?
– What is more important: Impact or Likelihood ?
Impact and likelihood on the board
Winning Monopoly strategies
• Complete Orange Colour group and invest as soon as possible
– Why Orange ? It has the biggest probability of other players landing on
it
– Be prepared to even trade down to get this colour group
• Complete Red Colour group as a second priority
– Why Red ? It has the second biggest probability of other players
landing on it
What lessons are to be learned
• Likelihood >>>>>> impact
– The weight of Likelihood should be > 50%
– The weight of Impact should be < 50%
BACKGAMMON
Backgammon
Important rules of Backgammon
• Goal: get all your tiles from one end to the other
• Only tiles that are standing alone on a pillar can be captured
• A captured tile has to be brought back in play at the beginning of the
board
• Random element: Dice
Translation to risks
• Your own checkers: Risks
• Opponents checkers: Causes
• Whenever one of your checkers is captured it‟s in fact a cause hitting a
risk
• A risk is mitigated when the checker cannot be captured (2 or more on
one pillar)
Translation of risk priority
• Low risk: checker in the first quadrant
• Medium risk: checker in the second or third quadrant
• High risk: checker in the fourth quadrant
What can Backgammon teach us?
• Which risks should be mitigated and which are low priority ?
• There might be a possibility to remove a cause, but in the same way
creating a new risk. Should we do it ?
– Eg: Back-up testmanager vs full time testing
Backgammon Strategies
• Checkers in the 1st quadrant don‟t have to be protected. Moving forward
is the better play.
Risks with low priority don‟t have to be mitigated. The correcting cost is
usually way less than the mitigation costs
• Checkers in the 4th quadrant need to be protected at all costs.
Risk with high priority need to be mitigated at all costs
• For checkers in the 2nd and 3rd following rule applies: Always take a
chance to capture
If you can remove a cause and therefore create a medium of low risk,
do so
CONCLUSIONS
Conclusions
• What Poker taught us:
– Adapt FTT/Risk tree after each test run
– Priorities of test items can change
– Justification has to be done after each test run
– Don‟t focus on results, focus on decisions
– Be prepared to get lucky!
Conclusions
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Adapt FTT/Risk Matrix
• Justification
Conclusions
• Monopoly
– Likelihood >>>>> Impact
• Backgammon
– Don‟t mitigate low priority risk
– Always mitigate high priority risks
– Removing a cause and creating a medium or low risk is the way to play
