Updating Directed Belief Network

8/13/2019 Updating Directed Belief Network

1/12


2/12

44 Boutheina Ben Yaghlane and Khaled M ellouliRe c e n t ly , a num be r o f te c hn ique s fo r ha nd l ing c ond i t i ona l be l i e f func t ions ha ve be e nde ve lope d . Ca no e t a l . [1992] have presented an axiomat ic sys tem for propaga t inguncer ta in ty ( inc luding be l ie f func t ions) in d i rec ted acyc l ic graphs . Smets [1993] hasge ne ra l i z e d t he Ba ye s t he ore m fo r t he c a se o f be l i e f func t ions a nd p re se n t e d t hed i s junc t ive ru le o f c ombina t ion fo r tw o d i s t inc t p i e c es o f e v ide nc e , w h ic h ma k e s i tposs ib l e t o re p re se n t know le dge by c ond i t i ona l be l i e f func t ions a nd to use t he mdi re c t l y fo r r e a son ing in be l i e f ne tw orks . She noy [1993] ha s e xp lo re d t he use o fg ra ph ic a l r e p re se n t a t i on o f va lua t i on -ba se d sys t e ms (V BS) , c a l l e d va lua t i onne tw orks , for represent ing cond i t iona l re la t ions . Xu an d Sm ets [1996] have presenteda n a l t e rna t i ve f ra me w ork to t he ge ne ra l V BS, c a l l e d e v ide n t i a l ne tw ork w i thc ond i t i ona l be l i e f func t ions (EN C ) a nd ha ve p ropose d a p ropa ga t ion a lgori thm fo rsuc h ne tw orks ha v ing on ly b ina ry re l a ti ons be tw e e n the va r ia b l e s .In t h i s pa pe r , w e w i ll be c on c e rne d w i th d i re c t e d be l i e f ne tw ork s w he n unc e r t a in ty ise xpre s sed i n fo rm of be l i e f func t ions . For t h is pu rpose , w e a dop t P e a r l s s t ruc tu re(di rec ted acy c l ic graph) , bu t ins tead of probab i l i ty func t ions , w e use b e l ie f func t ions .In order to eva lua te these be l ie f ne tworks , we apply the d i s junc t ive ru le ofc ombina t ion a nd the ge ne ra l i z e d Ba ye s i a n t he ore m (Sme t s [1993] ) . Ba se d on t hea x iom a t i c fra me w ork , ca l le d va lua t ion -ba se d sys t e m , p ropose d b y Sha re r a nd She n oy[1988] fo r und i re c t e d g ra phs (hype rg ra ph) a nd e x t e nde d by Ca no e t a l . [1992] ford i re c te d g ra phs (D A G ) , w e c a n p ropa ga t e b e l i e f func t ions i n t he ne tw ork us ing l oc alc om puta t i on t e c hn ique .The remainder of th i s paper i s organized as fo l lows. In Sec t ion 2 , we reca l l thene c e ssa ry ba c kground ma te r i a l a nd w e g ive t he c o r re sponde nc e be tw e e n be l i e ffun c t ion representa t ion and va lua t ion-ba sed sys tem . In S ec t ion 3 , w e present the loca lc omputa t i on a x ioms fo r t he p ropa ga t ion o f be l i e f func t ions a nd w e re c a l l t he tw oSm e t s s ru l e s . Se c t ion 4 sh ow s ho w to re p re se n t be l i e f func t ions i n d i rec t ed a c yc l icg ra phs (D A G ) a nd how to c ompute ma rg ina l s . F ina l l y , t he p ropa ga t ion a lgor i t hma pp l i e d on a be l i e f func t ion ne tw ork i s p re se n te d i n Se c t ion 5.

Background M ater ia lV a lua t ion-ba se d sys t e m (V BS) i s a f ra me w ork fo r ma na g ing unc e r t a in ty i n e xpe r tsys t e ms . In a V B S, kno w le dg e i s re p re se n t e d by fun c t ion c a l le d v a l u a t i o n w h i c h c a nbe c ons ide re d a s t he m a the ma t i c a l r ep re se n t a ti on o f a p i e c e o f i n fo rma t ion . In fe re nc ei s ma de by tw o ope ra t i ons c a l l e d c o m b i n a t i o n a n d m a r g i n a l i z a t i o n tha t opera te onva lua t i ons . Combina t ion c o r re sponds t o a ggre ga t ion o f know le dge . Ma rg ina l i z a t i onc or re sponds t o c oa rse n ing o f know le dge .In th i s pa pe r, w e w i l l pa r t ic u l a r iz e on a be l i e f - func t ion t he ory a nd w e w i l l show howth i s t he ory f i ts i n t he f ra m e w o rk o f V -BS. For t h i s pu rpose , va lua t ion , c om bina t ionand margina t iza t ion wi l l be t rans la ted to the i r in te rpre ta t ion in the be l ie f- func t ionthe ory .


3/12

Updat ing Directed Bel ief Ne tworks 452 .1 V a L u a tio n -B a s ed S y s t e m F r a m e w o r kT h e f r a m e w o r k o f V B S w a s f ir s t d e v e l o p e d b y S h e n o y [ 1 9 8 9 ] as a u ni f ie d l a n g u a g ef o r u n c e r t a i n ty r e p r e s e n t a t i o n a n d r e a s o n i n g i n e x p e r t s y s t e m s . I t i s g e n e r a l e n o u g h t om o d e l d i f f e r e n t u n c e r t a i n t y t h e o r i e s , s u c h a s p r o b a b i l i t y t h e o r y , p o s s i b i l i t y t h e o r y ,b e l i e f - f u n c t i o n t h e o r y . I n t h i s s e c t i o n , w e r e m i n d t h e f o r m a l d e f i n i t i o n s o f t h e b a s i cc o n c e p t s o f a V B S .Variables. C o n s i d e r a fi n it e s e t X o f v a r i a b l e s . E a c h v a r i a b l e X~ m a y r a n g e o v e r a

f in i te s e t o f p o s s i b l e v a l u e s Ox , o r s i m p l y O , , c a l l e d t h e frame f o r X r G i v e n an o n - e m p t y s e t X , o f v a r i a b l e s , ~ h e r e I C { 1 , .. ,n } , w e d e n o t e b y O z t h e C a r t e s i a np r o d u c t o f t h e f r a m e s o f t h e v a r i a b l e s , i .e . 1 9, = x { 19 , I i E I } . T h e e l e m e n t s o f 19, a r ec a l l e d configurationso f X rValuations. G i v e n a s e t o f v a r i a b l e s X r F o r e a c h I C_ { 1 , .. ,n } , t h e r e i s a s e t V r T h ee l e m e n t s o f V t a r e c a l l e d v a l u a t i o n s o n 1 9,. I n t u i t iv e l y , v a l u a t i o n i s t h e p r i m i t i v eo b j e c t t h a t r e p r e s e n t s u n c e r t a i n t y a b o u t a s e t o f v a r i a b l e s .Combination. I f V 1 a n d V 2 a r e t w o v a l u a t i o n s o n ~ a n d O : , r e s p e c t i v e l y , t h e n V ~ V 2i s a v a l u a t i o n o n 19~ x 1 92 . I n t u i t i v e l y , c o m b i n a t i o n i s a n o p e r a t i o n t o a g g r e g a t e t h ei n f o r m a t i o n o f t w o v a l u a t i o n s i n a s i n g l e v a l u a t i o n .Marginalization. I f V i s a v a l u a t i o n o n O , a n d J C I , th e n V *~ i s a v a l u a t i o n o n O rI n t u i ti v e l y , m a r g i n a l i z a t i o n i s a n o p e r a t i o n t o n a r r o w t h e f o c u s o f a v a l u a t io n .

2 . 2 B e l i e f F u n c t i o n R e p r e s e n t a t i o nT h e t h e o r y o f b e l i e f f u n c t i o n s , a l s o c a l l e d Dempster-Shafer theory, a i m s t o m o d e ls o m e o n e s b e li e fs . T h e t h e o r y w a s f i rs t d e v e l o p e d b y S h a f e r [ 1 9 7 6 ] . I t is r e g a r d e d a s ag e n e r a l i z a ti o n o f p ro b a b i l it y t h e o r y ( B a y e s i a n a p p r o a c h ) .D e f m i t i o n L A b e l i e f f u n c t i o n m o d e l i s d e f i n e d b y a s e t O c a l l e d a frame ofdiscernment a n d a basic probab ility assignm ent C o . p . a ) f u n c t i o n . A b . p . a i s a f u n c t i o nt h a t a s s i g n s t o e v e r y s u b s e t A o f 0 , a n u m b e r m : 2 --~ [ 0 , 1 ] s u c h t h a t : r e ( A ) > 0 f o ra l l A C _ , r e O ) = 0, an d ~ { m A ) I A c _ O} 1D e f i n i t i o n 2 . L e t t 9 b e t h e f r a m e o f d i s c e r n m e n t . T h e m a p p i n g b e l : 2 --* [0 ,1 ] i s abelief func tion i f f t h e r e e x i s t s a b a s i c p r o b a b i l i t y a s s i g n m e n t m s u c h t h at : V A o f 19,b e l ( A ) = ~ . { m ( B ) I B C _ A } .Interpretation. A n u m b e r m ( A ) m e a s u r e s t h e d e g r e e o f b e l i e f t h a t i s e x a c t l yc o m m i t t e d t o A . D u e t o th e l a c k o f i n f o r m a t i o n , m ( A ) c a n n o t s u p p o r t a n y m o r es p e c i f i c e v e n t . T h e v a l u e b e l ( A ) q u a n t i f i c s t h e s t r e n g t h o f t h e b c l i c f t h a t t h c e v e n t Ao c c u r s . A s u b s e t A o f 19 s u c h t h a t m ( A ) > 0 is c a l l e d focal element o f b e l . b e l i svacuous i f t h e o n l y f o c a l e l e m e n t i s O . I n D e m p s t e r - S h a f e r p r e s e nt a ti o n , w e a s s u m et h a t o n e a n d o n l y o n e e l e m e n t o f is tr u e closed-world). H o w e v e r , i n S m e t sd e f in i ti o n , w e a c c e p t t h a t n o n e o f th e e l e m e n t s c o u l d b e t ru e open-world), s o m ( O )c a n b e p o s i t i v e . I n t h i s c a s e , b e l i s c a l l e d a n unnorm alized belie f unction.


4/12

46 Boutheina Ben Yaghlane and Khaled MellouliT h e c o r r e s p o n d e n c e b e t w e e n b e l i e f - f u n c t i o n t h e o r y a n d V B S f r a m e w o r k c o u l d b ee xpr e s se d a s f o l lows :Valuat ions . Va lua t ion i s a ba s i c p r oba b i l i t y a s s ignm e n t .Combina t ion . T o c o m b i n e i n d e p e n d e n t p i e ce s o f e v id e n c e , w e u s e Dem ps ter s rule o fcombination. S uppose tha t be l l a nd be l2 a r e two be l i e f f unc t ions ove r a f r a m e o f

d i sc e r nm e n t O , l e t m l a nd m 2 be the i r r e spe c t ive b .p . a f unc t ions . The d i r e ct sum o fbe l~ and bel2, deno ted b y bei=be l~bel2 , i s g iven by i t s b .p .a func t ion m (m~m2):m ( A) = k ~ { m ~ ( A~ ) m : ( Az) [ A,C_O, A :CO , A~ nA z=A } for a l l O~A__.O, wh erek~=l-Y.{mj(Al)m2(A z) [A,C O , A~___O, AtO A2 =O }, k is a no rm aliz at io n factor .Margina l i za t ion . I f J C I and m is a bas ic p roba bi l i ty as s ignm ent on O~, then them a r g ina l o f m f o r J is t he ba si c p r oba b i l i t y a s s ignm e n t on O , de f ine d by :m~J(A) = ~ {m (B ) ] B C O t such tha t B l ' = A} for a l l A_CO

A x i o m a t i c F r a m e w o r k f o r t h e P r o p a g at io n P r o ce ssGive n a n e v ide n t i a l sy s t e m de f ine d by a s e t o f va r i a b le s X={ X~ . . .. . X ,} a ndv a l u a t io n s e x p r e s s e d b y b e l i e f f u n c t i o n s b e l L. . . . be l o we m a k e in f e r e nc e bycom pu t ing , for each va r iab le X, , the m argina l o f jo in t b e l ie f fun c t ion Cod,. . be l ) :~H o w e v e r , w h e n t h e r e a re m a n y v a r i a b l e s to m a n i p u l a te , t h e c o m p u t a t io n o f th e jo i n tbe l i e f f unc t ions be c o m e s n o t f e a s ib l e s inc e the jo in t f r a m e o f va r i a b le s i s t oo l a rge .M a ny r e se a r c he r s ha ve s tud ie d th i s p r ob le m in o r de r t o p r opose t e c hn ique s f o rc om pu t ing m a r g ina l s w i thou t e x p l i c i t ly c a l c u la ting the g loba l be l i e f f unc tion . A w e l l -k n o w n m e t h o d i s th e l o c a l c o m p u t a t i o n a m o n g t h e in i ti a l b e l i e f f u n c t io n s i n M a r k o vt r e es ( S ha r e r et al. [ 1 9 8 7] , S h e n o y a n d S h a r e r [ 1 9 9 0 ]) . T h e i d e a p r o p o s e d b y S h e n o ya nd S ha f e r [ 1990] i s t ha t i f c om bina t ion a nd m a r g ina l i z a t ion op e r a t ions ve r i f y th r e ea x i o m s , t h e n t h e l o c a l c o m p u t a t i o n s c a n b e d o n e .I n t h is s e c t io n , w e s h o w h o w t o d o c o m p u t a t i o n s w i t h b e l i e f fu n c t io n s u s in g l o c a lc o m p u t a t i o n t e c h n i q u e s i n u n d i r e c t e d a n d d i r e c t e d g r a p h s . T h e n , w e g i v e a m e t h o da n a l o g o u s t o B a y e s t h e o r e m p r o p o s e d b y S m e t s [ 1 9 9 3 ] i n o r d e r t o u s e i t i n t h ep r opa ga t ion p r oc e ss .

3 .1 A x i o m s f o r L o c a l C o m p u t a t i o n i n U n d i r e c t e d G r a p h sS h e n o y a n d S h a r e r [ 1 9 90 ] p r o p o s e a s e t o f a x i o m s i n w h i c h e x a c t l o c a l c o m p u t a t io no f m a r g ina l s i s poss ib l e . The se a x iom s p r ov ide ne c e ssa r y c ond i t i ons f o r t hed e v e l o p m e n t o f p r o p a g a t i o n a l g o r i t h m i n h y p e r g r a p h s . T h e y a r e o p e r a t e d o n b a s i cp r oba b i l i t y a s s ignm e n t s .Axiom A 1 . ( C o m m u t a t i v i ty a n d a s s o c i a t iv i t y o f c o m b i n a t i o n )

mL m~ = m~ m~ and (m~ m~) m e =- m~ ( m : m~).Axiom A 2 . ( C o n s o n a n c e o f m a r g in a l i za t io n )I l l _ J C K a nd m i s a ba s i c p r oba b i l i t y a s s ignm e n t o n O z , t he n (rn J )~ = m ~.Axiom A 3 . ( D i s t ri b u t i v it y o f m a r g i n a li z a t io n o v e r c o m b i n a t i o n )L e t m La nd m 2 be two b .p . a f unc t ions o n O~ a nd O ,, r e sp e c t ive ly , t he n(m, m,) ~= mL m~~'n~.


5/12

U p d a t i n g D i r e c te d B e l i e f N e t w o r k s 4 7

I n t e r p r e t a t i o n A x i o m A 2 t e l l s u s t h a t t h e o r d e r i n w h i c h t h e v a r i a b l e s a r e d e l e t e dd o e s n o t m a t te r . A x i o m A 3 is o f p a r t ic u l a r i m p o r t a n c e t o t h e d e v e l o p m e n t o fp r o p a g a t i o n a l g o r i t h m i n h y p e r g r a p h s . I n d e e d , i t a l l o w s u s t o c o m p u t e ( m , m : .) ; 'w i t h o u t e x p l i c i t ly c a l c u l a t i n g ( m ~ m 2 ).

3 2 A x i o m s f o r L o c a l C o m p u t a t i o n i n D i r e ct e d G r a p h sT h r e e n e w a x i o m s a r e a d d e d b y C a n o e t a l [ 1 9 9 2 ] t o S h e n o y - S h a f e r ' s a x i o m a t i cf r a m e w o r k , f o r t h e p r o p a g a t i o n i n d i r e c t e d g r a p h s . T h e s e a x i o m s a l l o w t h e d e f i n it i o no f s o m e a s p e c t s r e la t e d w i t h d i r e c t e d g r a p h s .A x i o m A 4 . ( N e u t r a l E l e m e n t )

T h e r e e x i s t s o n e a n d o n l y o n e b . p . a m 0 d e f i n e d o n ( g lX . . .x ( 9 , s u c h t h a t f o r e v e r yb . p . a m d e f i n e d o n O j , V J C _ I , m o ~j m = m .A x i o m A S . ( C o n t r a d i c t i o n )T h e r e e x i s t s o n e a n d o n l y o n e b . p . a m o d e f i n e d o n O ~ . . . O , s u c h t h at fo r e v e r yb . p . a m , m o m = m .A x i o m A 6F o r e v e r y m d e f i n e d o n t h e f r a m e c o r r e s p o n d i n g to t h e e m p t y s e t o f v a r i a b l e 0 2 ;, i fm ~, m ~ ~ , t h e n m = m 0 ~ e

I n t e r p r e t a t i o n T h e n e u t ra l e l e m e n t i s t h e v a c u o u s b e l i e f f u n c t i o n d e f i n e d b y :m 0 ( A ) = l i f A = O x x . . . x O , a n d r no (A ) = 0 o t h e rw i s e .T h e c o n t r a d i c t io n i s a b e l i e f s u c h t h a t i f i t i s c o m b i n e d w i t h a n y o t h e r b e l i e f, i tp r o d u c e s t h e c o n t ra d i c t io n . W h e n w e d o n o t n o r m a l i z e , t h e c o n t r a d i c t i o n i s t h e z e r o -v a l u e d m a s s a s s i g n m e n t d e f i n e d b y : m e ( A ) = 0 V ~ O t x .. . x ( 9,. T h e m e a n i n g o fa x i o m A 6 i s re l a t e d to c o n d it io n a l i n d e p e n d e n c e c o n c e p t ( S e e C a n o e t a l [ 1 9 9 2 ] ) .D e f in i t ion 3 . ( C o n d i t i o n a l b e l i e f f u n c t io n )A c o n d i t i o n a l b e l i e f f u n c t i o n o n O s g i v e n (9~ i s d e f i n e d a s a b e I i e f f u n c t i o n o n O t (g js u c h t h a t m a r g i n a l i z i n g i t o n O l g i v e s t h e n e u t r a l e l e m e n t .E x a m p l e 1 . A c o n d i t i o n a l b e l i e f f u n c t i o n o n 0 2 g i v e n O 1 i s g i v e n b y a b a s i c p r o b a b i [ ita s s i g n m e n t o n O t x O 2 s u c h t h a t if i t i s m a r g i n a l i z e d o n ( 9 1 , t h e n w e g e t th e n e u t r ae l e m e n t , l i k e i n t h i s e x a m p l e :

m ( { ( 0 ,, ,0 z z ) , ( 0 ~ , 0 2 z )} ) = 0 . 7 1m ({ (0 ,,021) ,(0~2,0~) }) = 0 .2 ~ ~ m t m ( (91) = 1m ( { 0 , ,0 ~ 2 } ) = m ( ~ ) = 0 . 1

Def in i t ion 4 . ( A b s o r b e n t b e l i e f f u n c t i o n )A b s o r b e n t b e l i e f f u n c t i o n r e p r e s e n ts p e r f e c t i n f o r m a t i o n r e p r e s e n t e d b y m ( A ) = l ,w h e r e A i s a s i n g l e t o n .A n e x a m p l e o f a b s o r b e n t b e l ie f f u n c t i o n i s w h e n w e c o m b i n e t w o c o n t r a d ic t o r y b e l i e ff u n c t i o n s .


6/12

48 Boutheina Ben Yaghlane and Khaled M elloul i3 .3 T h e G e n e r a l iz e d B a y e s ia n T h e o r e mL e t u s c o n s i d e r t w o s p a c e s O x a n d O v . W e s u p p o s e t h a t t h e a d d i t io n a l i n f o rm a t i o n i sg i v e n b y b el.~ (. I x ) r e p r e se n t i n g c o n d i t i o n a l b e l i e f f u n c t i o n i n d u c e d o n t h e s p a c e O yg i v e n x e l e m e n t o f O x . W e w a n t t o c o m p u t e b e l x (x [ y ) f o r a n y x __ .Ox and y__ .O Th isb e l i e f i s d e ri v e d f r o m t h e g e n e ra l iz e d B a y e s i a n t h e o r e m ( G B T ) .F o r g i v e n b e l i e f f u n c t i o n b e l: 2 - - ,[ 0 , 1 ] , S m e t s [ 1 9 9 3 ] d e f i n e s a f u n c t i o n b : 2 -- , [0 , 1 ]s u c h t h at : b ( A ) = b e l ( A ) m ( O ) .F o r a ny x_C O a nd yC _O r, t he G B T pe r m i t s t o bu i l t t he b e l i e f f unc t i on be l ( x [ y ) by

belx(x I Y) = b x x I Y ) - b x O I Y ) ( 1 )b x ( x l Y ) = I - I b y ( y Ix ~)

x i ~I f e a c h b e ly ( . [ x ) h a p p e n s t o b e a p r o b a b i l i t y f u n c t i o n P ( . I x ) o n O y a n d t h e p r io rb e l i e f o n O x i s a l s o a p r o b a b i l i ty f u n c t i o n P o (x ), t h e n t h e n o r m a l i z e d G B T i s r e d u c e dt o B a y e s t h e o r e m :

P o x i ) P ( y x i) : P x , [ y ) 2 )re(x, l y ) = y ~ e 0 ( x i f x P ( y lx i )xi~3Ox

S i m u l t a n e o u s l y , i f w e w a n t t o c o m p u t e b e l y ( y l x ) f o r a n y x C O X a n d y C_ O ,o w e u s ea n o t h e r r u l e p r o p o s e d b y S m e t s [ 1 9 9 3 ] c a l le d t h e d i s j u n c t i v e r u le o f c o m b i n a t i o n( D R C ) . I n d e e d , t h e D R C p e r m i t s t o b u i l t th e b e l i e f fu n c t i o n b e l r ( y [ x ) b y :be l~ (Y I x ) = b~ (Y I x ) - by (O [ x ) (3)b y ( y ] x ) = Y I b y ( y ] x )xi~:x

B e l i e f F u n c t i o n s a n d D i r ec t ed A c y c l i c G r a p h sB e l i e f n e t w o r k s d e s c r i b e d b y S h a f e r et al [ 1 9 8 7 ] a r e undirected hypergraphs w h e r eh y p e r - n o d e s r e p r e s e n t s e t s o f v a r i a b l e s a n d h y p e r - e d g e s a r e w e i g h t e d w i t h b e l i e ff u n c t i o n s o n t h e produc t space o f t h e v a r i a b l e s . I n o r d e r t o e v a l u a t e t h e s e b e l i e fn e t w o r k s , S h a f e r et aL [ 1 9 8 7 ] h a v e p r o p o s e d a m e s s a g e - p a s s i n g s c h e m e f o rp r o p a g a t i n g b e l i e f f u n c t i o n s u s i n g l o c a l c o m p u t a t i o n .I n P e a r l s a p p r o a c h ( u s i n g p r o b a b i l i t y f u n c t i o n s ) [ 1 9 8 8 ] , t h e e d g e s a r e directed a n dw e i g h t e d b y t h e conditional probabili t ies o v e r t h e c h i l d n o d e g i v e n t h e p a r e n t n o d e s .T h e g r a p h i c a l s t r u c t u r e u s e d t o r e p r e s e n t r e l a t i o n s h i p s a m o n g v a r i a b l e s a r e c a u s a ln e t w o r k s , c a l l e d directed acycl ic graphs ( D A G ) i n w h i c h t h e v e r t i c e s r e p r e s e n tv a r i a b l e s , t h e a r c s s h o w t h e e x i s t e n c e o f d i r e c t i n f l u e n c e b e t w e e n t h e v a r i a b l e s , a n dt h e s tr e n g t h s a r e e x p r e s s e d b y c o n d i t i o n a l p r o b a b i l i ti e s .


7/12

Updating Directed Belief Networks 49L i k e i n o u r p r e v io u s w o r k ( B e n Y a g h l a n e e t a l . [1998] , Khal faUah e t a L [1998]) , weadop t P ear l ' s s t ruc tu re , bu t i n s t ead o f p robab i l i t y func t i ons , we use d i r ec t ed be l i e fne tworks t ha t a re we igh t ed by cond i t i ona l be l i e f func t i ons . Indeed , cons ide r t hes i m p l e st d i r ec t e d b e l i e f n e t w o r k w i t h tw o n o d e s X a n d Y d e f i n e d o n f r a m e s O x andO v , r e spec t i ve ly . S uppose t ha t t he re ex i s ts so m e a p r i o ri i n fo rmat ion ov er O x g iven bybe l i e f fun c t i on be lox (mox i s t he co r respond ing ma ss func t i on ) an d som e a p r io r ii n fo rm at ion over ~ g iven b y be l i e f func t i on be lov (m0v). W e as sume tha t we a l so havecon di t iona l bel ief funct ion s {belv( . [ x ) : x ,~O x}.F or each n ode i n t he ne twork , i ts marg ina l is com pu ted by com bin ing a l l t he messagesrece ived f rom i t s ne ighbo rs and i ts own p r io r be l ie f . S o , i f we wa n t t o compu te bd o fthe node X , wh ich is t he pa ren t o f Y , we com bine its p r io r be l i e f be l0x wi th t hem e s s a g e c o m i n g f r o m Y , i .e .

belx = belox be ly- x (4)w h e r e b e lv ~x i s a be l i e f func t ion on X , and i s com pu ted by :V xC_O, bet~_x(X = ~ , ,~ m0,(y ) bel(x[ y) (5)

such t ha t be l(x [y ) i s a pos t e r i o r be l i e f func t ion g iven by t he genera l i zed B ayes i antheo rem (F orm ula 2 ).In t he o the r hand , i f we wa n t t o com pu te be l v o f t he node Y , wh ich i s t he ch il d o f X ,we com bine i ts p r i o r be l i e f be loy wi th t he m essage com ing f rom X, i . e.

belv = belov belx-,w (6)wh ere be lx_ v i s a b e l i e f func t i on on Y , and i s com pu ted by :V yC_O~, bel _y(y ) = ~__, m0x(X bel,c(Y lX (7)

such t ha t be lv (y I x ) i s a cond i t i ona l b e l i e f func t i on r ep resen t i ng t he r e l a ti on be tw eenX and Y, and g iven by t he d i s junc t i ve ru l e o f comb ina t i on (F ormula 1 ).

Th i s compu ta t i on i s s im i l a r t o t he one p roposed by S hafe r e t a l . s algori thm [1987],bu t t he p ropaga t ion be twe en node s i s fa s t e r because t he s t o rage a t t he edge i s smal l e r(w i th con di t iona l bel ief func t ions , we s tore on ly [ O I x2 I'~ I valu es a t w ors t case) .

Propagat ion of Bel ief Funct ions in Directed Ac ycl ic G raphsIn t h i s s ec t ion , we p ropose t he p ropaga t ion a lgo r i t hm fo r d ir ec t ed be l i e f ne tworks t ha ta re quan t i f i ed by cond i t i ona l be l i e f func t i ons . The main i dea o f t he p roposeda lgo r i t hm i s to ex t end P ea r l ' s a l go r i t hm fo r be l i e f func t i ons and t hen t o u se t he l oca lp ropaga t i on p roces s fo r t he com pu ta t i on o f ma rg ina l be l i e f func t i ons ra the r t han t heca l cu l a t i on o f g loba l be l i e f func t i ons . As i n P ear l ' s s t ruc tu re , we suppose t ha t t hen e t w o r k s w e a r e w o r k i n g w i t h d o n o t h a v e l o o p s.


8/12

50 Bou theina Ben Ya ghlane and Khaled Mel lou liF o r t h e n e e d o f t h e p r o p a g a t i o n a l g o r i t h m , l e t U b e a fi n i te s e t o f n o d e s o f a g i v e nn e t w o r k . F o r e a c h X @ U , l e t P ( X )C _ U b e t h e s e t o f p a r e n t s o f X , a n d C (X )C __ U b e t h es e t o f c h i l d r e n o f X . F o r e a c h n o d e X , w e s t o r e t h e a v a i l a b l e a p r i o ri b e l i e f f u n c t i o nb e l 0 x . W e s t o r e a l s o t h e c o n d i t i o n a l b e l i e f f u n c t i o n s f o r e a c h n o d e g i v e n i t s p a r e n t s{ b el x( . [ Y ) : Y E P ( X ) } . W e a s s u m e t h a t i f a n o d e X i s i n s ta n t i a t e d t h e n w e h a v e a n e wo b s e r v a t i o n ( O x ), e l s e O x i s t h e v a c u o u s b e l i e f f u n c t i o n .L i k e i n P e a r l s n o t a t io n s , e a c h v a r i a b l e o f th e n e t w o r k h a s a X v a l u e a n d = v a l u ea s s o c i a t e d w i t h it. I n a d d i ti o n , e a c h v a r i a b l e p a s s e s a X m e s s a g e t o e a c h o f i ts p a r e n t sa n d a r{ m e s s a g e t o e a c h o f it s c h il d r e n . I n t h e f o l l o w i n g s e c t i o n , w e p r e s e n t t h ep r o p a g a t i o n a l g o r i t h m .

5 . 1 I n i t i a l i z a t i o n P r o c e s sI n t h is s t e p , w e s h o w h o w t o c o m p u t e t h e a p r i o ri b e l i e f f u n c t i o n s o f e a c h v a r i a b l e( n o d e ) o f th e n e t w o r k u s i n g p r o p a g a t i o n m e t h o d .Procedure INIT1 . S e t a l l X v a l u e t o v a c u o u s b e l i e f f u n c t i o n2 . F o r a l l r o o t s X ,

set r,.x = be l x- s e n d a n e w n x ~ Ym e s s a g e f o r al l c h i l d re n Y o f X u s i n g f o r m u l a ( 7)

3 . W h e n a v a r i a b le Y r e c e i v e s a n e w m e s s a g e f r o m a p ar e n t, t h e n c o m p u t ethe ne w 7,~ va lue u s i ng th i s fo rm u l a n,~ = be l v ( 7~x~ v )x c P ( v )t h e m a r g i n a l b e l i e f b e l v = ~ . X ys e n d a n e w ~ m e s s a g e f o r a l l i t s c h i l d r e n u s i n g f o r m u l a ( 7 ) .

xample 2 . L e t u s c o n s i d e r t h e f o l l o w i n g d i r e c t e d b e l i e f n e t w o r k ( F i g . 1 . ) c o n s t i t u t e db y 6 n o d e s U = { A , B , C , D , E , F } r e p r e s e n t in g t h e v a r i a b l e s o f th e p r o b le m . F o r th e s a keo f c o m p u t a t i o n a l s i m p l i c i t y , a ll t h e v a r i a b l e s u s e d i n t h is e x a m p l e a r e b i n a r y .

m(a) = .4m(a .) = .5m (e ,) = .1 fola = .5 m (b l ~) = .85 re(e) = .7rn c lb )= .3 m E I b ) = . 8 , , , , ~ 2 - ~ 0 , , [ a ) = . 5 m e ~ l a )= . 1 5 ~r n e ) = . lm(O,lb = ,7 m(Oc Ib ) = . ~ . . . ~ k ~ . _ ~ J _ _ . ~ , . .. . .. _ ~ m (t~ d = . ~~ . _ ~ m ( d l b ) = .75 m (71 b ) --- . g , ~ - - ~ ~ ,( d l~ ) - - .'~ m( d[ e) = .6

~ ~ 9 ~ l b ) : .25 re O J ; ) : .5 ~ m O 0 l e ) .1 m(O~ ~ = .4r n f t d ) = .8 m f [d ) - = - . 5

~ ~ e , . l d ) - . 2 m O ~ I d _ . s

Fig . 1 . Th e state o f the bel ief netwo rk before a ny variables a re instantiated


9/12

Upda t ing Directed Be l ief Netw orks 51B e f o r e a n y v a r i a b l e s a r e i n st a n ti a te d , w e p e r f o r m t h e i n i ti a l iz a t i o n s t e p i n w h i c h w ea p p l y t h e p r o p a g a t i o n m e t h o d d e s c r i b e d a b o v e .N o d e A . r o o O rc,~= r e ( A )C o m p u t e a n d s e n d n , , _ , m e s s a g e t o B u s i n g f o r m u l a ( 7 ). S o , f o r a n y bC __ O,, t h e

b e l i e f f u n c t i o n b e l ^ _ ~ (b ) i s g i v e n b y i ts b . p . a m ^ ~ . ( b ) a s f o l l o w s :m , , _ , ( b ) = 0 . a x 0 . 5 = 0 . 2 m A ~ ( b ) = 0 . 4 2 5 m A ~ ( O O = 0 .3 7 5N o d e E . A s n o d e E i s a ls o a n a n o t h e r r o o t, w e d o t h e s a m e t h i n g f o r it.N o d e B . W h e n B r e c e i v e s a n e w r e m e s s a g e f r o m i ts p a r e n t A

C o m p u t e its : t , v a l u e a n d i ts n e w m a r g i n a l i n th is m a n n e r : n . = m ( B ) = u , , ~m , ( b ) = 0 . 2 m , ( b ) = 0 . 4 2 5 m B ( O 0 = 0 . 3 7 5C o m p u t e a n d s e n d ~ B ~c m e s s a g e t o C u s i n g f o r m u l a ( 7 ) . S o , f o r a n y c.C _O t h eb e l i e f f u n c t i o n b e l ~ c ( C ) i s g i v e n b y it s b . p . a m r c ( c ) a s f o l l o w s :m ~ c ( c ) = 0 . 2 x 0 . 3 = 0 . 0 6 m , ~ c ( E ) = 0 . 3 4 m , ~ c( O c) = 0 .6S i m i l a r l y , c o m p u t e a n d s e n d r% ~ o m e s s a g e t o D u s i n g f o r m u l a ( 7) .

N o d e D . W h e n D r e c e i v e s n e w re m e s s a g e s f r o m i ts p a r e n t s B a n d DC o m p u t e i ts JtDv a l u e a n d i ts n e w m a r g i n a l : ~ t, = ~ t ~ o ~ D

0 . 0 6 0 . 3 1 a : - : ' i i ~ } r t : i : .

0 . ~.,dd0.0945 0.009

:'ii d

i d0.210 . 6 4

d0.0465

0.03840.1323 0.0126 0.0651

d0.4032 ~ D0.1984

A f t e r n o r m a l i z a ti o n , w e o b t a i nax,,(d ) = m o (d ) - 0 .5 44 2 / 0 .85 87 = 0 .63~ D ( d ) = m r, ( d ) = 0 . 1 1 6 1 / 0 . 8 5 8 7 = 0 .1 4rCD(OD) = m D O D) = 0 .19 84 /0 .8 5 87 = 0 .23w h e r e k = 0 . 8 5 8 7 i s a n o r m a l i z a t i o n f a c t o r .

C o m p u t e a n d s e n d n t ,- F m e s s a g e t o F u s i n g f o r m u l a ( 7 ).N o d e C . a n d F . S i m i l a rl y , w e d o t he s a m e c o m p u t a t i o n s f o r n o d e C a n d F .F i n a l l y , t h e r e s u l t s o f i n i t ia l i z a t i o n a r e i l lu s t r a t e d i n ( F i g . 2 . ) :re(A ) ~t L^re(B) ~ KB ~ i 0.4 }f'0.4 } [ 1.2 0.2 1 0.5 0.5 1

~ 3 7 5 J ~ 3 7 5 j E l j . c n V : L J U J Lm ( C ) ~ ( : k c f ~ O ) ~ f 0 _ ~ ..5 _ .0 _ ]~ I.06 0.0 1 0.07 0.07 1 ,d

t O . 6 0 j L O 6 0 J I. i J L J L J l . J t ,, , . _ _ ~ _ . .. )

m (E ) ~ ~'-,0,7 1

Fig. 2 . T he state of the be l ief netwo rk after init ial izat ion


10/12

52 Boutheina Ben Yaghlane and Khaled Mellouli5 2 Updat ing P rocessW h e n n e w o b s e r v a ti o n i n f o r m a t io n ) is i n t ro d u c e d a t a g i v e n n o d e , t h e u p d a t i n ga lgo r i thm wi l l be pe r f o r m e d . The ge ne r a l i de a o f t h i s a lgo r i thm i s ba se d on the f a c tt h a t w h e n w e a r r i v e a t a n o d e X , w e c o m p u t e a l l t h e i n c o m i n g m e s s a g es , a n d t h e n w ecalc ula te r~x va lue , Xx va lue , ne w m a r g ina l be lx , a nd a l l ou tgo ing m e ssa ge s .C o m p u t i n g v a l u e sF o r e a c h n o d e X o f U , w e c o m p u t e t h e m a r g i n al b e l x v a l u e r e p r e s e n ti n g t h e m a r g i n a lbe l i e f o f node X . I t i s ob ta ine d b y c o m b in ing i ts i n i ti a l va lue w i th the ne w obse r va t ionOr.) a nd w i th the m e ssa ge s c om ing f r o m a ll i t s pa r e n t s a nd a l l i ts c h i ld r e n :

be lx = x w h e r e ~ x = b elx v ~P x ) ~ t v _ x ) a nd L x = O x z c < x ~ k z - x ) 8 )

C o m p u t i n g m e s s a g e sAs the node X i s upda te d , t he n i t w i l l s e nd ne w m e ssa ge s to a l l i t s ne ighbor s no t ye tb e e n u p d a t e d . T h e s e m e s s a g e s a r e c o m p u t e d a s f o ll o w s : r Lx~ r e p r e se n t ing the m e ssa ge se n t f r om a nod e X to i ts c h i ld r e n Y f o r m ula 7 ) :

~x~ r = be lx~v w her e be lx~v y = ~_ .~ m~ x) be l~ y I x ) 9)suc h tha t be l r y ] x ) is g ive n by the d i s junc t ive ru l e o f c om b ina t ion F o r m ula 1 ) .Xv_x r e p r e se n t ing the m e ssa ge se n t f r om a no de Y to it s pa r e n t s X f o r m u la 5 ) :~ x = be lv--x w here be lv~x X ) = ~_ .~ mv y be lx x I Y) 10)suc h tha t be lx x I Y) i s a pos t e r io r be l i e f f unc t io n g ive n by the G B T F or m ula 2 ) .

P r o p a g a t i o n a l g o r i t h mW h e n a n e w o b s e r v a t io n O x ) i s in t r o d u c e d at n o d e X , w e p r o p o s e th e f o l l o w i n gr e c u r s ive a lgo r i thm to pe r f o r m the upd a t ing :

X c o m p u t e s i ts n e w v a l u e b e lx , u s ing f o r m u la 8 ) . F o r e v e r y c h i ld n o d e Z E C X ) , w e c a l c u l at e a n d s e n d t h e n e w m e s s a g e r tx ~ Yus ing f o r m ula 9 ) to a l l c h i ld r e n no t ye t be e n upd a te d . F o r e v e r y p a re n t n o d e Y E P X ) , w e c o m p u t e an d s en d t h e n e w m e s s a g e

L x _Yus ing f o r m ula 10 ) t o a l l pa r e n t s no t ye t be e n upda te d . T h e n , w e s e le c t a n e w n o d e X o f U .Th i s p r opa g a t ion a lgo r i thm e nds w he n the r e a r e no no de s to upda te .E x a m p l e 3 S u p p o s e t h a t w e h a v e t h e f o l l o w i n g s i m p l e b e l i e f n e t w o r k c o n s t i t u te d b ythe no de s A , B , a nd C . A f t e r pe r f o r m in g in i ti a l iz a t ion , t he s t a te o f t he ne twor k i s a sf o l lows :


11/12

U pdat ing D irec ted B el ie f N etworks 53m(B) ~ ~ m(C) kc[ 0 7 ] 0 7 1l r l. . - ~ ~ 1'3 0"~I 1.1 0. I 1 0.2 0.2

( I ) . m ( ' ~ l ~ ) = . 2 b . ~ . 7 ' l ; B - - ^ 7 1 ; C - - ^ l d m , ( a l c ) '- 9 m ( a l ~ ) - , 6m(O,lb)= .5 re(O, b ) = 8 \ 7 m (e,l) . t re(e^ c) = 4m ( a . k ,{ z l F I I

Fig. 3. Th e state of the b elief network after initializationS u p p o s e n o w t h a t n e w o b s e r v a t i o n O ^ i s i n t r o d u c e d a t a n o d e A , w h e r e O , = 1 , 0 , 0 ) .S o , nod e A i s i n s ta n t i a te d a nd f o r upda t i ng t he be l i e f ne t w or k , t he p r op a ga t i ona l g o r i t h m d e s c r i b e d a b o v e w i l l b e p e r f o r m e d .N o d e A A is instantiatedC o m p u t e t h e n e w v a l u e s f o r t h e n o d e A . A h a s n o c h i l d re nCo m pu t e a nd s e nd ~ .^ ~s t o i ts pa r e n t B us i ng f o r m u l a 10 ) . F o r a ny b C O B, bel^ ~B b ) i sg i v e n b y i t s b . p .a : m ^ _ B b ) = l x 0 . 2 = 0 . 2 mA~,~ b=0x0.5 = 0 m^ _B O b =0 . 8S i m i l a r ly , c o m p u t e a n d s e n d k ^ _ c to C u s i n g f o r m u l a 1 0 )N o d e B . When B receives a new A message from AC o m p u t e it s n e w m a r g i n a l u s i n g fo r m u l a 8 ) .

0.7i ; b~ 0 . 2 ~ i i b ; ; : 0 .1 4 b

0 6

0 1 0 2b :e|0.02 0 0.04 b

0.08 b 0.16 0 xA f t e r no r m a l i z a t i on , w e ob t a i n : m B b ) = 0 . 74 0 .98=0 . 75 m ~ b )=0.08 m a O = 0 . 1 7w h e r e k = l - 0 . 0 2 = 0 . 9 8 i s a n o r m a l i z a ti o n f a c t o r.T h e n o d e B h a s n o ch i ld r e n o t h e r t h a n A ) a n d n o p a r e n t.N o d e C . W e d o t h e s a m e t h in g a s n o d e B .T h e f i na l s t a te o f t he be l i e f ne t w or k i s g i ve n by F i g . 4. :frg.7~75~ [ 0.~71 i0 ~ 21 @ @ m (C _ ) [ 0 ~o.68l f o~ 3l1 0 .0 8 1 ~ :~ J / 0 O l 0 =/.0 .17,/ L0 SJ 0 .5 0 .2 I X~ k^--H ~',,,--c~ r b.9 0 -~ [ 0 .2 2 j L 0 .5 J t, 0 .4 jb , o J o4 ..

L o jLO 4j L o jFig. 4. Th e state of the be lief network after A is instantiated


12/12

54 Bouthe ina Ben Yagh lane and Kha led M el lou l i

C o n c l u s i o nI n th is p a p e r, w e h a v e p r e s e n t e d t h e p r o p a g a t i o n a l g o r i th m f o r d ir e c t ed a c y c l i c b e l i e fn e t w o r k s t h a t a r e w e i g h t e d b y c o n d i t i o n a l b e l i e f f u n c t i o n s . U s i n g t h e g e n e r a l i z e dB a y e s i a n t h e o r e m , t h e p r o p o s e d a l g o r i t h m a l l o w s u s t o p e r f o r m u p d a t i n g w h e n n e wi n f o r m a t i o n i s i n t r o d u c e d i n t h e n e t w o r k . T h e r e p r e s e n t a t i o n u s e d f o r t h e s e n e t w o r k si s s i m i l a r t o P e a r l s s t r u c t u r e . T h i s a l g o r i t h m c a n b e g e n e r a l i z e d b y u s i n g o t h e rg r a p h i c a l r e p r e s e n t a ti o n s , s u c h t h a t v a l u a t i o n n e t w o r k s .cknowledgments

T h e a u t h o r s a r e g r a t e f u l t o P r o f e s s o r P h i l i p p e S m e t s a n d a n o n y m o u s r e v i e w e r s f o rt h ei r c a r e fu l r e a d i n g a n d v a l u a b l e c o m m e n t s o n t h is p a p e r .References1. Ben Yagh lane B. , Khalfallah F. , and Mellouli K. (199 8) Propagating Condit ional BeliefFunctions in Directed Acyclic Networks CE S A 98 , IEEE S ys t ems , M an and Cyberne ti c s ,Tunisia, 2, pp 232-238.2. Ca no J. , Delgado M. and Mo ral S. (199 2) An Axiomat ic Fram ework for Propagat ingUncertainty in Directed Ac yclic Networks Int. J . of Ap proxim ate R easoning, 8 : 253-280.3. Da wid A.P . (1979) Conditional Independence in Statistical Theory Journal o f the Roy alStatistical Soc iety, Series B , 41, 1-31.4. Kha l fa l lah F ., Ben Ya ghlane B. and Mel louI i K. (1998) Propagating Multi-Observations

in Directed Belief Networks IEE E-S M C 98 , IEE E S ys t ems , M an and Cyberne ti cs , S anDiego (US A ) , pp. 2472-2477 .5. Lau r i tzen S. L . , Daw id A . P . , Larsen B. N. and Leime r H. G. (1990), IndependenceProperties of Directed M ark ov Fields Netw orks , 20(5) , 491-505.6. M ellouli K. (1987) On the Propagation o f Beliefs in Ne two rk using Dempster-ShaferTheory o f Evidence Do ctora l Dissertation, U nive rsi ty o f Kansas.7. Pearl J. (1988) Probabilistic Reasoning in Intel ligent System s : Ne twor ks o f PlausibleInference Morgan Kaufmann, San Marco, CA.8. Sharer G. (1976) A M athematical Theory of Evidence Princeton Univ. Press, NJ.9. Sha rer G. , Sh eno y P.P., and Mellouli K. (198 7) Propagating Be lief Functions in

Qualitative Markov Trees Inter . J . of Appro xim ate R easoning, 1: 349-400.10. Shafer G. and She noy P .P . (1988) Local Computation in Hypertrees W orking paper N o201, Sch ool of Business, Un ivers i ty of Kansas, Law rence, KS.11. Shenoy P .P . (1989) A Valuation-based Language for Expert Systems Inter. J. ofApp rox imate R eason ing , 3 (5) , pp 383-411 .12. Sh eno y P . P . (1993) Valuation Netwo rks an d Condit ional Independence in Heekerman D.A nd M am dani A . eds . P roc . 9 h UA I , M organ K aufmann , S an M ateo , CA , 191-199 .13. Shenoy P .P . and Sharer G. (1990) Axioms for Probabil i ty and Belief-FunctionPropagation in Shachter R.D., Levitt T.S., Kana l L.N. and Lemm er J.F. ed s U A I, 4, 169-19 8.14. Sm ets Ph. (1988) Bel i e f Func t ions in (Ph. Smets, A. Mamdani, D. Dubois, and H. Prade,Eds) , Nonstanda rd Log ics for Autom ated Reasoning, Ac ade m ic Press , Lond on, 253-286.15. Smets Ph. (1993), Be lief Functions : The Disjunctive Rule o f Combination and theGeneralized Bayesian Theorem Int. J . o f A pp rox im ate R easoning., 9 : 1-35.16. Xu H. and Smets Ph. (1996 ) Reasoning in Evidential Networks with Condit ional BeliefFunctions Int. Journal of Ap proxim ate Rea soning, 14: 155-185.

Documents

Updating Directed Belief Network