Markov Analysis

Markov Analysis

Overview• A probabilistic decision analysis• Does not provide a recommended decision• Provides probabilistic information about a

decision situation that can aid the DM• Applicable to systems that exhibit probabilistic

movement from one state to another, over time– Probability that a machine will be running one day and

broken down the next– Probability that a customer will change her

department store to the next, called brand switching

Brand Switching Example• Customers are usually royal to a particular brand or store, or

supplier• Two gas stations in a community , P and N• Study indicates customers are not royal to either one• Willing to change based on advertisement factors• If a customer bought gas from P in any given month, there was 0.6

probability that the customer would buy from P and 0.4 probability from N the next month

• If a customer traded with N in any given month, there was 0.8 probability that the customer would buy from N and 0.2 probability from N the next month

Next MonthThis month P N

P 0.6 0.4N 0.8 0.2

Terminology

• Gas station that a customer trades at a given month is called state of the system (two states of system)

• Probabilities of various states are called transition probabilities– Transition probability sum to one– Probabilities apply to all participants– Probabilities are constant over time– States are independent over time

What Information MA Provides?

• Answers the probability of being in a state at some future time period

• Determining the probability that a customer would trade with them in month 3 given that the customer trades with them this month

• Use the following decision tree 1– The probability of a customer’s purchasing gas from P in month 3 given

that the customer traded with P in month1 =0.36+0.08=0.44

– The probability of a customer’s purchasing gas from N in month 3 given that the customer traded with N in month1 =0.24+0.32=0.56

• Use the following decision tree 1– Given that N is the starting state in month1, the probability of a

customer’s purchasing gas from N in month3: 0.08+0.64=0.72

– Given that N is the starting state in month1, the probability of a customer’s purchasing gas from P in month3: 0.12+0.16=0.28

Month 3-Result

Month 3

This month P N

P 0.44 0.56N 0.28 0.72

• Easy for month 3, but not for month 10 or 15• Follow the notes in class