Upload
ryan-cammack
View
216
Download
1
Embed Size (px)
Citation preview
NOISY OR GATEJavier Turek and Eyal Regev
Several causes sharing a common effect.
YZ
Ui
e+
e-
X
Polytrees
U1 Un
… …
Like a tree, but with multiple parents.
Link matrix Tx|u
Tx|u contains the conditional probabilities:P(X=x | U1=u1,…,Un=un)
1. The table is huge (contains 2n entries).
2. Who can know such information anyway?You cannot expect to find the P(X | U1,…,Un) table.
However, you may know how every Ui influences X separately.
The OR-gateHaven’t done
homework
Caught cheating
Failed an exam More likely!
OR
Inhibitors
Failed an exam
Caught cheating
OR
AND
Paid the TA
Haven’t done
homework
AND
Re-doing the
course
Inhibitors
U1
X
U2
Un
I1
I2
In
OR
AND
AND
AND
Inhibitors are independent
Associate probability to an inhibitor.
Noisy OR-Gate
U1
X
U2
Un
I1
I2
In
OR
AND
AND
AND
| , :i k iP x u u k i q
Noisy OR-Gate
U1
X
U2
Un
I1
I2
In
OR
AND
AND
AND
:u iT i U True
Tu
if
|1 if +
u
u
ii T
ii T
q x
P x uq x
Message Passing – updating X
YZ
U
e+
e-
X
| , | |Belief x P x e e P e x P x e D x A x
Y ZD x U x U x |x u Xu
A x T C u
YU x
y
XC u
ZU x
u
Message Passing – updating X
YZ
U
e+
W
e-
X
| , | | ABelief x P x e e P e x P x e D x x
YU x
y
XC u u
ZU x
XC w
Y ZD x U x U x | ,,
x u Xw
Xwu
T C CuA x w
| , | |Belief x P x e e P e x P x e D x A x
|x u Xu
A x T C u
Message Passing – updating U,Y,Z
YZ
U
e+
e-
X
Y ZD x U x U x |x u Xu
A x T C u
XU u
YC x
ZC x
|xXx
uTU u D x ZY A xC x x U
| ,x u wX Xx w
T CU D x wu
YZ
U
e+
W
e-
X
XU u
YC x
ZC x
Message Passing – updating W,U,Y,Z
XU w
Y ZD x U x U x | ,,
x u Xw
Xwu
T C CuA x w
ZY A xC x x U ZY A xC x x U
|x u Xu
A x T C u
|xXx
uTU u D x
YZ
Ui
e+
e-
X
X iU u
YC x
ZC x
Message Passing – many parents
X nU u
|:k
i u ku k i k i
X Xx
xU D T uCu x
ZY A xC x x U
| X ku k
x uT CA x u Y ZD x U x U x
Link matrix
U1 Un
… …
1XU u
Belief Update – Noisy OR-Gate
if
1 if +
u
u
i X ku i T k
i X ku i T k
D x q C u x
Bel xD x q C u x
u u
i X i X ku i T k T
qC u C u
u u
i X i X ku i T k T
qC u C u
1u u
i X i X ku i T k T
qC u C u
Belief Update – Noisy OR-Gate
1 1 if
1 1 1 if +
i X ii
i X ii
D x q C u x
Bel xD x q C u x
1u u
i X i X ku i T k T
qC u C u
1i X i X ii
qC u C u 1 1 i X i
i
q C u
Update messages
Where
The message to the child is the same:
1 if +
1 if
i i i i i
X i
i i i
D x q D x q uU u
D x D x u
1 1i k X kk i
q C u
kY
ii
kXC Ay U x x
Example
Windows Vista
No electricity
Virus 2K bug
Does not start
Wrong date
Stolen Password
Lost data
D1 D2 D3 D4
M1 M2 M3 M4
Example
D1 D2 D3 D4
M1 M2 M3 M4
0.01\0.99 0.1\0.9 0.2\0.8 0.2\0.8
ijq0.8
0.1 0.9
0.2 0.5
0.1 0.7
0.80.2
|W , forhere j iij kP m d d k iq
\ (1 )i ip p
Conjunction query
Conjunction query q: find the belief that several events happen simultaneously.
Applying the chain rule:
Product of m belief updates
Q
i ii I
q X x
1 2 2 1
1 2 1 1 2 1
1 2 2 1
, , , , ,
| | , ,
| , , , ,
m m m
m m
m m m
P q P x x x x x
P x P x x P x x x
P x x x x x
Answering a query
In our example:
Computing P(q):
1 1 2 3 4q d m m m m
1 1 1 2 1 1
3 2 1 1
4 3 2 1 1
| | ,
| , ,
| , , ,
P q P d P m d P m m d
P m m m d
P m m m m d
Example Update the belief on M1
Update likelihoods and priors:
1 11 21 1 2| 1 1 1 1 1 1 0.272P m q qd p
1 1p 2p
1 2MU d
4 2MC d
1 2 11 21 111 ,1 0.92,0.2MU d q q q
4 1 12 2 2 2 2, 1 0.338,0.662M M MC d U d p U d p
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M2
Update likelihoods and priors:
2 2 121 421 41| | 1 1 1 1 1 0.81,m d dP m P m q q p
1 1p
2 4 12 42 12, 0.45,0.9MU d q q q
4 2 24 4 4 4 4, 1 0.111,0.889M M MC d U d p U d p
2 4MU d
4p
4 4MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M3
Update likelihoods and priors:
3 32 1 1
13 33 3
1| |
1 1 1 1 1 1 0.836
, ,P m P m
q q p
m m d d
1 1p
3 3 13 33 131 ,1 0.98,0.8MU d q q q
4 3 33 3 3 3 3, 1 0.23,0.77M M MC d U d p U d p
3 3MU d
3p 4 3MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example Update the belief on M4
4
3 2 1 14
42,3,4
|
1 1 0
,
.
,
7
,
81i M ii
m m m dP m
q C d
4 3MC d
4 4MC d
4 2MC d
D1 D2 D3 D4
M1 M2 M3 M4
Example
And the final solution is…
Example
And the final solution is…
1 1 1 2 1 1
3 2 1 1
4 3 2 1 1
3
| | ,
| , ,
| , , ,
0.01 0.272 0.81 0.836 0.781 1.439 10
P q P d P m d P m m d
P m m m d
P m m m m d
Thank You!