Handling Uncertainty

COM362 Knowledge EngineeringHandling Uncertainty

1

Handling Uncertainty

John MacIntyre0191 515 3778

[email protected]


2

Reasoning with Uncertainty

Why Uncertain answers Confidence Factors Probabilities Fuzzy Logic


3

Why?

82% of all statistics are made up on the

spot!

Vic Reeves


4

How do Humans Cope with Uncertainty?

Default/assumed reasoning (guess work) non monotonic reasoning

Missing data gradual degradation in performance

Uncertain information Uncertain reasoning Not always one correct solution Ranking possible answers


5

Uncertain answers

One very simple method of dealing with uncertainty

Allow Yes / No / Unknown answers Process ‘Unknown’ by triggering extra

inferencing to determine answer when user cannot Automatic depending upon sourcing

sequence within Aion DS


6

Confidence/Certainty Factors

Confidence expressed as a number between 0 and 1

Allows uncertainty to be expressed in information and in reasoning

Not necessarily based on any real evidence!


7

CFs in Practice Assume that if A is true, then B is true But we are only 80% certain that A is true Clearly we can only be 80% certain that B is true!

0.8

A => B


8

CFs continued... What if we were only 80% certain that

A=>B, and only 60% certain that B=>C?

A => B => C0.8 0.8 0.6


9

CFs continued...

How certain can we be that B and C are true?

Each uncertain step in the reasoning process must make us less certain in the result - this is mimicked by multiplying CFs together

Multiplying fractions reduces the total at each step


10

CFs continued...

Thus, given we are only 80% certain of A, we can only be 64% certain that B is true (0.8 * 0.8 = 0.64)

and only 38% certain that C is true (0.8*0.8*0.6 = 0.38)


11

CFs continued... However, two independent pieces of

corroborating evidence must make us more certain of the result - so how does this work?

Given A => C AND B => C0.8 0.8

Given A and B as true how confident are we of C? (clearly the answer should be higher than 0.8 but less

than 1


12

CFs continued...

Thus A => not (C) and B =>not (C)

0.2 0.2

0.04 A+B => not (C)

A+B => C 0.96

Then multiply 0.2 by 0.2 to give 0.04 - thus:

And invert this again to give:

To calculate the answer we invert the rules:


13

Disadvantages of CFs

CF’s come from the opinions of one or more experts and thus have very little basis in fact

People are unreliable at assigning confidence or certainty values

Two people will assign very different numbers and will often themselves be inconsistent from day to day


14

Advantages of CFs

CFs do allow people to express varying degrees of confidence

Easy to manipulate Can work well when trying to rank

several possible solutions Must be careful not to place too much

emphasis on the actual numbers generated


15

Probabilities

Advantages Much stronger mathematical

foundation for resultsBased on precise data

DisadvantagesOnly work when required statistical

data is availableSlightly more mathematical


16

Bayes’ Theorem

P(H:E) =

P(E:H) P(H)

P(E:H) P(H) + P(E:not H) P(not H)

And also

P(H:not E) =

(1 - P(E:H)) P(H)(1 - P(E:H)) P(H) + (1- P(E:not H)) P(not

H)


17

Bayes’ Theorem

Thankfully this is not as complex as it looks !!!

Must understand the notation and the principle of two types of probability: prior probabilities - our previous

assumptions or based on previous evidence

posterior probabilities - amended based on new evidence


18

The NotationP(H) The prior probability of a hypothesis (H)

being trueP(E:H) The probability of an event (E) being true

given the hypothesis (H) is trueP(H:E) The probability of the hypothesis (H)

being true given that the event (E) is trueP(E:not H) The probability of an event (E) being true

given that the hypothesis (H) is known to be false


19

Working with an Example

Imagine we are trying to determine if a patient has the flu

The hypothesis (H) is that they have the flu

The events (E) are the symptoms they present

There are multiple events (symptoms)


20

Example continued...

In this case the patient has a temperature, a runny nose and is sneezing but they do not have a headache or other symptoms that indicate flu.

How do we determine the specific probability of flu given this particular set of symptoms?


21

Easy for One Symptom!We can tell from past experience that P(flu) = 0.3 This is easy to measure!Given 1 symptom we can also determine the probability of the hypothesis:-When symptom is true Flu not (Flu)Temperature .4 .6Runny nose .4 .6Hot flushes .5 .5Thus P(flu : Temperature) = 0.4P(flu : Temperature & Runny nose) = ???P(flu : Temperature and Hot flushes) = ???


22

What do we Know? We CAN NOT measure

P(H:E1 & E2 & not E3)

We CAN measure P(H), P(E1:H), P(E2:H) P(E3,H)

P(E1:not H), P(E2:not H), P(E3:not H)

Using Bayes Theorem we can then calculate P(H:E1 & E2 & not E3)


23

A Different Approach

Need to change the way we approach the problem

Intuitively we fix the symptom and then determine the hypothesis

Instead, lets first fix the hypothesis and then determine the probabilities of the symptoms


24

Evidence from Data

Two separate populations of patients 100 people with flu 100 people without flu

% symptoms in each population

Symptom when Flu is true not (Flu)

Temperature .7 .5Runny nose .6 .2Hot flushes .7 .1


25

Using the Evidence

Notice this data is NOT the same as shown previously and the numbers do not necessarily add up to 1

Collecting this data is easy We can now repeatedly use the

equations given to calculate the probability of flu given a range of symptoms


26

Applying Bayes

to calculate P(flu : temperature)P(H:E) =

0.7 * 0.3

0.7*0.3 + 0.5 *0.7= 0.375

P(H:E) =

P(E:H) P(H)

P(E:H) P(H) + P(E:not H) P(not H)

Given P(Flu) = 0.3 and the table above If a patient has a temperature we can

use


27

Applying Bayes

Given this result we can now calculate P(flu:temperature but not runny nose)

We now use 0.375 as P(H) and the second equation

P(H:not E) =

(1 - P(E:H)) P(H)(1 - P(E:H)) P(H) + (1- P(E:not H)) P(not

H)


28

Applying Bayes

P(H:not E) =

(1 - 0.6) * 0.375

(1 - 0.6)* 0.375 + (1- 0.2)*(1-0.375)

= 0.23


29

Bayes’ Theorem

Bayes’ theorem allows us to calculate the probability of a hypothesis given any combination of symptoms (or events)

The raw data is easy to obtain though we must make sure that the symptoms are independent of each other

Normal practice would assume independence unless common sense suggests otherwise


30

Bayes’ Theorem

The equations can also be used to determine which symptom has the most significant effect on all potential hypothesis (diagnosis) and also when all significant events have been determined


31

Advantages/Disadvantages

Bayes theorem is mathematically sound therefore results using this method can be justified

However, this method needs data collected from previous results and will only work where this is available

This is in itself a strong point as the data can be proven where CFs are only expressions of opinion


32

Fuzzy Logic

Attempts to move away from “hard” values in reasoning

More like the way humans think Example: what is ‘room temperature’?

21oC....? 20, 21, 22oC....? 19 - 23oC....? 17 - 24oC....?


33

Fuzzy Set Theory

Fuzzy set theory: Define a set “room temperature” Define the range of values in the set (17-

24oC) Define the strength of membership of each

value in the set, using a membership function


34

Fuzzy Set MembershipS

tren

gth

of

Mem

ber

ship

17 18 19 20 21 22 23 24

Set Values for “Room Temperature”

1

0


35

Using Natural Terms

Allows use of intuitive terms in constructing rule base, such as: temperature is ‘high’ vibration is ‘low’ load is ‘medium’

Outputs can also be in these terms, such as: bearing damage is ‘moderate’ unbalance is ‘very high’


36

Fuzzy Logic Strengths

Strengths of fuzzy logic: less rules required since single set

membership function can cover a large range of values

membership function can be used to represent intuitive knowledge from experts

outputs can be in familiar terms


37

Fuzzy Logic Limitations

Limitations of fuzzy logic: still requires writing of many rules knowledge acquisition and representation

problems apply not adaptive in its pure form difficult to maintain and upgrade


38

Applications of Fuzzy Logic

Many applications, especially in domestic appliances: auto-focusing in cameras washing machine controllers microwave controllers etc etc etc

Development of ‘Neuro-Fuzzy Systems’ - to give some adaptive ability


39

Conclusions

Humans deal intuitively with uncertainty all the time

Knowledge-based systems can benefit from being able to mimic this

Three different ways of implementing uncertainty CFs, Bayesian Mechanics, Fuzzy logic

There are other approaches!!

Documents

Handling Uncertainty