33

B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

� � ��� ����

� � � ���������� � � � ���� � ���������� �! � " �#� $ � % ���&��� �'� "� �! �(������)���&� ! �'*+�

,.-0/21436587:9;5+<>=?-0/214363;@BAC=?-0D;EGFH5IFJ/K5L790AMEN90AG@81POQ90AR<>=N9;S./2ST=?-0D;EGF#UV9;1WACE?7X-C-Y3[Z

\^]B_V`Yacb.dLefehgRiW\Rikj

l�m#npofqsr)ntvuxwNy ]Bz{_ yX|;}:w `Yas~�`���`VzN_Yac]B��` y bC~��fa |;}�y ]�_ |;��|;u �x_Ybf� u[w `Yac� | _ w � |;�^� b2�:` � zd��K`Y_ w ]Bb u z#�:iW� |;u �h�2i4��j�_ |;u _Yb � ��] u ` w bH` � ]B_V] w _ | ��z |;� ] u �Lbfa �6|�w ]Bb u �LaNb � ] ���} `Yac�L`V_ w `G� } `VaN] � ` u�w zY��`J� } `VaN] � ` u[w z w�y�|�w �:`V�K] |�w `{�Lacb �'w�y `{]B��` |;��} acb w bK_Yb �b;�Ta |;u �:b � ]��Y`V�Q_Vb u[w aNb � i¡ ¢_Yb �Q� b u ��`V�K] |�w ]�b u b2_V_Y��aczY£��Lbfa¤`G� |;�h}�� `�£�~ y ` uzN�:�K¥c`Y_ w z¤] us| a |;u �:b � ]��Y`V�s_ � ] u ]�_ |;�[w aN] |�� �:b u b w �L� �B��¦ _Vb �h}���¦ ~�] w�yswNy `Y]�a | z �zN]�� u `V� w ac` |�wN� ` u[w £ wNy ��z¡_Vb �Q} aNb � ]Bzc] u � wNy `�]���` u[w ]�§X_ |�w ]�b u b���_ | ��z |;� `V¨©`Y_ w zViª�y ` u _Yb u ��] w ]Bb u z«�Lbfa�]���` u[w ]4§X_ |0w ]Bb u.| ac` u b wR� ` w £ w�y `¬�©`Vz w b u `¬_ |;u ��b6]�z��`VaN]��;`{­ -0/2S�30A �Lbfa w�y `)®�� |�u�w ] w ]B`Yz#b��T] u�w `Yac`Yz w°¯�uX|�� ` ��¦ £ | a |;u �f`�b;� } b;zNzN]�� � `� |;� �:`Yz w�yX|�w aN` } aN`VzN` u�w z«bf��a�]B� u bfa |;u _Y` | �©b;� w¤w�y `±� |�w?|�� �f` u `Ya |�w ] u � } aNbK_Y`Yzcz|;u � w�yX|�w _ |�u�u b w �©`{] �h} aNbC�f`V�²~�] w�y ] u _Yac` | zc] u �Qz |;�h}�� `{zc]B�Y`�i�³ y ` | ] � b;�w�y ]Bz)_ yX|;}:w `YaR]�z w b6��` � b u z w a |�w `´dµ]¶j w�yX|�w zN��_ y �©bf� u ��zR_ |;u ��`s��`VaN]4�f`Y�.� ¦zN] �Q}:� ` |�� �f`V��a | ]�_ � ` wNy b2�:z |;u ��dµ]�]8j w�yX|�w £���`Vz } ] w ` w�y `H] �h} `Vac�L`Y_ w ]�b u b;� w�y ``G� } `Yac] � ` u[w zV£ wNy `´��`VaN]��;`Y�·��bf� u ��zs_ |;u�¦ ]B` � ��zc]B� u ]4§X_ |�u�wM|;u ��zNb � ` w ] � `Yz

�f¸[�

Page 2: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f¸f¸�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ| _V_Y��a |�w `Q] u �Lbfa �6|0w ]Bb u b u�wNy `6] �Q}X| _ w b;� |Ç} b � ]B_ ¦ b uÑw�y `Q` u[w ]�aN` } b } � �¶|0�w ]�b uh| z#~^` �B��| z¤b uQ|¬}�| a w ]B_Y� �¶| a#] u ��]��K]���� |;� ~ y b }X| a w ]�_Y] }X|�w `V��] uMw�y `)z w ��� ¦ i

Page 3: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ4ÕfÂÁðÉM½�¿HÍ«ÏÖÎ^¹)½ÔÃGÍ«É �f¸;×ØÖÙCÚ ÛÝÜ�Þ m¤ßÖàâáâr Þ#ã ß Üä«åcæ�åcæ ç�èêé{ëÔìXíJëÝîTï�ð¤ñpòóç�ñpòÒôÅìXëÝîTï�õÇöRésëÝì÷ôcèøëÔñ^ï�ù� w�|;u � | ac��`G� } `YaN] � ` u[w?|;� z w ����]B`Vz>] u�w�y `��:]Bb � bf�f]B_ |;� £ � `Y�:]B_ |�� £ |�u ����` yX| � �]Bb;a |;� zN_Y]�` u _V`Yz¬] u � | aN] | � �4¦ ] u �fbfú�` wNy `�] u z w ac� � ` u[w b;��a |;u �:b � ]��Y`V�Ñ_Vb u[w acb ��ûw�y�|�w ]BzV£#zN����¥c`Y_ w z | aN` | zNzN]�� u `V� |0w a |;u ��b �üw b � | aN]�bf��z{�faNbf� } zÇdµbfa w aN` |�wc�� ` u[w z)bfa } aNbf�;a |;� z?jJ£ |;u � wNy ` � ` |;u ��]4¨T`VaN` u _Y`VzR��` w ~�`Y` uâ}�| a w ]B_Y] }X|;u[w z�] u��]4¨T`VaN` u[w �faNb;� } z | aN`)aN`V� | aN�:`Y� | z � ` | zc��aN`Vz^b;� w�y `)`Výh_ | _Y]�`Yz^b�� w�y ` | zczNbK_Y] �|�w `Y� } aNbf�;a |;� zYi¤þp`G�2] |�w ]�b u z��Lacb �êwNy ]Bz«]���` |;� zc` w � }.�Q|ÿ¦6w?| ú;` }��¶| _V`H`Y] w�y `Va� ¦ � | ] � �:aN` w b � `Y` wM|;u[¦ b�� w�y `6`G� } `VaN] � ` u�w�|;� ac`Y®���]�aN` � ` u[w z>bfas� ¦ ��` � ]B� �`Ya |�w ` |0wNw ` �Q}2w z w bâac` �B| � wNy `Yzc`ÖaN`Y®���]�aN` � ` u�w zYi <JS�3;@8F�E?=G5{E����XEGFJ@��6EGSX5LAs| ac`z w ����]�`Yz^] u ~ y ]�_ y a |;u ��b � ]B�V`Y�Q_Yb u[w acb � ]Bz�`Y] w�y `Va�� u �L` | zc]B� � `±bfa�� u �:`YzN]�a | � � `;itvu zc��_ y `G� } `YaN] � ` u[w zY£¡zN���K¥c`V_ w z | aN`hz w ] �B��| zczN]�� u `Y� |�w a |;u ��b � w b.� | ac]Bbf��z�facbf� } zV£K�:� w�� ` � �©`VaNz#b;�©` | _ y �facbf� }Q| aN`«zc] �h}���¦ ` u _Ybf��a | �f`V� dLa |�wNy `Ya w�y�|;u�Lbfac_Y`Y�Xj w b }X| a w ]�_Y] }X|�w `M] uâw�y ` } aNb;�fa |;� | zNzcb2_V] |�w `Y� ~�] w�y w�y `Q�facbf� } û ] w ]�z� }Öw b w�y `¬] u ��]4�2]���� |;� z w b�zc` � `V_ w±|;� b u � wNy ` } aNbf�;a |;� zYi� `V_Y` u�wN��¦ £���zN`>b���z w ac]B_ w a |;u ��b � ]B� |0w ]Bb u ] u zNbK_Y] |;�+|;u � � `V��]B_ |;� `G� } `VaN] �� ` u[w?|�w ]Bb u´y�| z��©`V` u ®���`Vz w ]Bb u `V�Ç�Lb;a w�y ac`Y` �6| ¥cb;a�aN` | zNb u zYi�fi��Ô`Yac�L`V_ w _Vb u[w aNb � ]Bz yX| aN� w b | _ y ]�`V�f`Mb;a | zc_Y`Ya w?| ] u i{� w �:��]B`Vzp] u ~ y ]B_ yw aN` |0w�� ` u[w ]Bz | zNzc� � `Y� w bp��`«a |�u ��b � ]B�V`Y� �Q|ÿ¦ ��` �Q| aNac`Y�>� ¦ � u _Yb uK�w aNb ��� `Y� @���XEGFLUÿE?=G5�=?-���1P@µ9;ST=?E i��Xbfa«`J� |;�Q}:� `;£2zN����¥c`Y_ w zR`J� } `Yac]B` u _Y] u �| �2�f`YaczN`Mac` | _ w ]Bb u z w b |;u `G� } `Yac] � ` u[w?|;� ��aN�:� �6|ÿ¦ ��`V_Y]B�:` w b´ac`Y����_V`wNy ` | zNzN]�� u `V�²��b;z | �f`�i^  ��w `Va uX|0w ]��;` �4¦ £X]�� w�y `{`J� } `Yac] � ` u[w ]Bz w `Vz w ] u � |��ac���Ö�Lbfa |´w `Ya � ] uX|;� ��]�zN` | zN`�£ | zN����¥c`Y_ w zN��z } `V_ w ] u � w�yX|�w¬y `�bfaÒz y `]�zp] u wNy `M_Yb u[w aNb � �;aNbf� }â�Q|ÿ¦ bf� w?| ] u.w�y `M�:aN���´�LaNb � b wNy `Yapzcbf��ac_Y`YzVi�2�:_ y ] �h} `Vac�L`Y_ w _Vb �h}�� ] |;u _Y`�ac` u ��`VaNz wNy `�`J� } `Yac] � ` u[w ] u ��]�aN`Y_ w>|�u �] u�w aNbK����_V`Yzs�:] | z{] u[w b w�y `Q_Yb u _ � ��zc]Bb u z w�y�|�w ac`YzN` | aN_ y `YaNz���a | ~ �LaNb �wNy `±� |�w?| i¡³ y ]Bz���] | z�_ |;u�u b w ��`±_VbfaNac`Y_ w `Y�6� u�� `VzNz���` w�| ] � `Y� � bK��` � z#b;�_Vb �Q}:� ] |�u _Y` | aN`H_Yb u z w aN��_ w `Y��d� +�LaNb uÇ|;u ���X` � � �Q|;u �C×f×:�CjGi

�2i)þp` u�¦ ] u ��zN�:�K¥c`Y_ w z | zNzc]B� u `Y� w b´_V`Ya w?| ] u _Yb u[w aNb � �faNbf� } z w�y `M�©` u `V§ w zb;� w�y `±�©`Vz w«| � | ] �¶| � � ` w ac` |�wN� ` u�w�yX| z � b;a |;�÷|;u � � `V� |;� a |�� ]4§X_ |�w ]Bb u zYi�Xbfa�`G� |;�h}�� `;£^] u   t þÒ��ac`YzN` | aN_ y ] w ]Bzh��]�ýh_V� ��wMw bÖ¥c��z w ]�� ¦·}:�¶| _V`Y��b} acbf�fa |;� z{�©`V_ | �:zN` w�y bfzN` }X|0w ]B` u[w z | zczN]�� u `Y� w b w�y ` }:�¶| _V`Y��b �faNb;� }~�bf� � �M�©`«��` u ]B`V� | _Y_V`YzNz w b } b w ` u[w ] |;�B�4¦¬� ]��L` � z | �K] u � w aN` |0w�� ` u[w d�� |;� _ |�C×;¸f×[jGi

Page 4: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f×���¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ�:i �)|;u ��b � ]�� |�w ]Bb u £[� ¦ ] w z��;`Ya ¦�} ac`YzN` u _Y`�£ �Q|C¦ ] u�� ��` u _Y` }�| a w ]B_Y] }X|�w ]Bb u| zH~�` ���¤| zÒ�©` yX| �2]�bfaQd��)`Y_?ú �6|�u �C×f×f�fjGi��XbfaH`J� |��Q}�� `;£Ý` � ]B�f]�� � `{_ |;uK���]�� |�w `Yz �Q|C¦ �©`M~ | a ¦ b;� |�}�}���¦ ] u � w b | zN_ y bKb � b u _Y` w�y ` ¦ ��]Bzc_YbC�f`VawNyX|�w ] w ��` � ]B��`Ya |�w ` ��¦ a |;u ��b � ]��Y`Yz¬] w z | � � ]BzNzc]Bb u _Yac] w `YaN] | i���]�ú;`V~�]�zN`�£| z��Òa |;� `Ya |�u �Ö� yX|�} ]BacbÖdÅ�C×;¸;e�j u b w `V� £�zN�:�K¥c`Y_ w z«] u ��aN��� w aN] |;� z �6|ÿ¦��` � `Yzcz � ]Bú�` ��¦²w b }X| a w ]B_V] }X|�w `M] u a |�u ��b � ]B�V`Y� w aN] |;� z w�yX|;u ] u u b u `G� �} `Yac] � ` u[w?|;� z w ���:]B`YzV£�`V�;` u ~ y ` u�wNy ` w aN` |�w�� ` u[w z | ac`Ç`V®�� |;�B�4¦�u b uK�wNy aN` |�w ` u ] u �:i

  �4w bf�;` w�y `YaY£¤ac`YzN` | aN_ y `YaNz | ac`´�©`V�f] u�u ] u � w b | _?ú u bC~ � `Y�:�f` w�yX|�w>�6|;u � |�w `V�a |�u ��b � ]B� |�w ]�b u��Q|C¦ � u ��`Va � ] u ` wNy `ÒaN` � ] | �:] � ] wv¦ b;��`G� } `VaN] � ` u�w�|;� `V�K]���` u _V`|;u � w�yX|�w `G� } `Yac] � ` u[w?|�w ]Bb u ~�] wNy y � �Q|;u zc���K¥c`V_ w zâb�� w ` u ] u �fb � �f`Vz ¯�|;u �zNb � ` w ] � `Yz A?7X-0/K143 ] u �fb � �f` ¯¢|;u ` � ` � ` u[w b;��zN` � � � zc` � `Y_ w ]Bb u i

³ y ]Bz²_ yX|;}:w `Ya²_Yb u _V`Ya u z w�y `���a | ~�] u �Áb;�M] u �L`YaN` u _Y`Vz²�Lacb � z w ����]B`Vz.] u~ y ]�_ y zc���K¥c`V_ w z yX| �f`Ò§ uX|�� _ y bf]�_Y`Hb;� } acbf�fa |��²ûKw�y `Òa |�u ��b � ]B� |�w ]�b u ]�z�_Yb uK�§ u `V� w b |;u ] u ��]�aN`V_ w)@8S:AG5IFJ/��6EGSX5 dµbfa 90A?AG@��;S��6EGSX5 j wNyX|�w�� `YaN` ��¦ ` u _Ybf�:a | �f`Vzbfap�:]BzN_Vbf��a | �f`Yz }X| a w ]B_V] }X|�w ]Bb u ] u²w�y `>� | ac]Bbf��z } aNb;�fa |;� zYi��÷b;aR`G� |;�h}�� `;£ ] u`V� |;� � |�w ] u � wNy `�`Výh_ | _ ¦ b;� | �f]4�f` u�w a | ] u ] u � } aNbf�fa |;� £ u b w ]�_Y`Vz�b��T` � ]B�;]B��] � ] wv¦�Q|C¦ ��`pzN` u[w�w b | a |;u ��b �Q�4¦ zN` � `Y_ w `Y�´�;aNbf� } b;�Ýz w ����` u[w z)b;aY£ |;��w `Ya u�|�w ]4�f` �4¦ £` � ]B�f]�� � `s_ |;u �:]B� |�w `Yz �6|ÿ¦ ��`hzN` � `Y_ w `Y� |�w a |;u �:b � w bÇaN`V_Y`Y]4�f`QzN_ y b �B| aNz y ] } z�Lbfa }X| a w ]�_Y] }X|�w ] u �R] uMw�y ` } acbf�fa |�� i¡�2] � ] �B| a �4¦ £�] u ��ac��� w ac] |;� zY£fzc���K¥c`Y_ w z �6|ÿ¦��`«�f]��;` u a |;u �:b �Q�4¦ _ y bfzN` u�| �:�K]�_Y`«b u aN`V_Yb �h� ` u ��`V�Q�:bfz | �;` � `V�;` � £ ¦ ` w�w�y `§ uX|�� _ y bf]�_Y`Öb;�±��bfz | �f`�~�] ��� ��`´��` w `Ya � ] u `Y�·� ¦�w�y `Özc���K¥c`Y_ w z w b⧠wMwNy `Y]�a] u ��]��K]B�:� |;�Tu `Y`Y�:zYi

tv�Q} `YaÅ�L`Y_ w _Vb �h}�� ] |;u _Y` } bfzc`Yz |Ñ} aNbf� � ` � �©`V_ | ��zc`²zc] �h}���¦ _Yb �h}X| ac] u �w�y `>�La | _ w ]Bb u zH] u wNy ` w ac` |�wN� ` u�w¬|�u � _Yb u[w acb � �facbf� } z �Q|ÿ¦ } acbC�2]���` |´� ]�z �� ` | ��] u � `Yz w ] �Q|�w `��Lb;a y b�~'`V¨©`Y_ w ]��;` w�y ` w aN` |0w�� ` u[w ~^bf� � ���©`Ç]�� |;}�}�� ]B`V�� u ]4�Lbfa �h��¦.w b w�y ` } b } � �¶|0w ]Bb u i��Xbfa{`G� |;�h}�� `�£+]�� w�y bfzc`�zN�:�K¥c`Y_ w zM~ y bâ��` �_ � ] u `V� w b w?| ú�` w�y `>��aN�:� | aN` } aN`V_Y]�zN` �4¦²wNy bfzc`{~ y b�~�bf� � � yX| �f`Mac`Yz } b u �:`Y�| �:�;`YaczN` �4¦ £ wNy `>`G� } `Yac] � ` u[wH� ]B� y[w _Yb u _ � ���:` w�yX|0wpw�y `s��aN���6]Bz � bfaN`{`V¨©`Y_ �w ]4�f` wNyX|;u ] wH| _ w � |;����¦ ]�zYi tvu \ yX|;}2w `YaÒ� dµzN`V`h�2`Y_ w ]�b u �:i �2£!�¡]��f��ac`>�:iW��dµ�XjNjG£~�`Hz y bC~�`Y� w�yX|0w�w ac` |�wN� ` u�w `V¨©`Y_ w ]��;` u `VzNz)] u zc��_ y z w ����]B`Vz�]Bz | _ w � |;����¦ÇST-0S�"@µ3KEGSX5µ@ #�9 ­ 1BE iÔ³ y�|�w ]BzY£�] uÒwNy ` | ��zc` u _V`�b;� | ���:] w ]�b uX|;��� bK��` � ] u � | zNzc� �h}:w ]�b u zY£w ac` |�wN� ` u[w `G¨T`V_ w ]4�f` u `Yzcz+_ |;u:u b w �©`¤`Yz w ] �6|�w `Y�Ò�LaNb � wNy `�� |�w?| ~�] wNy bf� w ��] | zY£`V�;` u ~ y ` uÇw�y ` u � � ��`Ya«b;��zN�:�K¥c`Y_ w zR] u´w�y `Ò`G� } `Yac] � ` u[w)|;}�} acb | _ y `Vz±] u § u2�] wv¦â|;u �Ñ`V�f` u ~ y ` u�| ac`Y_VbfaN��]Bz | � | ] �¶| � � `{b;� wNy ` | _ w ]�b uÑ|;u ��aN`Yz } b u zN`hb;�` | _ y zc���K¥c`Y_ w i

Page 5: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ4ÕfÂÁðÉM½�¿HÍ«ÏÖÎ^¹)½ÔÃGÍ«É �f×2�³ y `6®���`Vz w ]Bb u ~^` |�wNw ` �h}:wÒw b |;u zc~�`Ya{] uâw�y ]Bz¬_ yX|;}:w `Ya¬]BzH~ y ` wNy `Ya{] u2�

��]�aN`Y_ w a |�u ��b � ]B� |�w ]�b u _ |;uÖ} aNbC�K]B��`R] u �Lbfa �6|0w ]Bb uhwNyX|�w)|;��� bC~�z |;}:} aNbÿ�2] �Q|�w `| zczN`Yzcz � ` u[w b;� w�y `{] u[w ac] u zc]B_ � `Yac] w b;� |h} aNb;�fa |;� £ | z�~^b;� � �²�©` � ` | zN�:aN`Y�T£�Lbfa{`G� |;�h}�� `�£+]�� wNy ` } acbf�fa |�� ~�`Yac` w b ��`�`J� w ` u �:`Y� |;u � �Q|;u � |0w `Y�x� u ] ��Lbfa �Q�4¦Çw b w�y ` } b } � �¶|�w ]Bb u iÒ³ y ` |�uX|;��¦ zc]Bz } ac`Yzc` u[w `V�xz y b�~�z wNyX|�w £��;]��f` u |� ] u ] �6|;� zN` w b;� | zczN� �h}:w ]Bb u zV£[zN��_ y ] u �L`VaN` u _Y`Yz | ac`)] u ��`V`Y� } bfzczN]�� � ` ¯¢|;� �©`V] w] uÒw�y `��Lbfa � b;�Ý­ -0/KS�30A £�a |�w�y `Ya wNyX|;u{} aN`V_Y]Bzc` } bf] u[w `Yz w ] �6|�w `YzV£0�Lbfa w�y `�_ | ��z |��`V¨©`Y_ w b;� wNy ` } acbf�fa |;� bfa w aN` |0w�� ` u[w i±³ y `Yzc`M�©bf� u ��zp_ |;u �©`{��zN`V�â� ¦.w�y `|;uX|���¦ z w�w bM�f� | a |;u[w `V` w�yX|�w^w�y `H_ | ��z |�� `V¨©`Y_ w b;� | �f]��;` u´} acbf�fa |��ê� �:z w ��`y ]�� y `Ya wNyX|;u b u ` � ` | zc��a | � � `p®�� |;u[w ] wv¦Ç|;u � � b�~�`Ya wNyX|;u²|;u b wNy `YaVi$p��a � bfz w _Yac��_Y] |;�Ô| zNzc� �h}:w ]�b u ]�z wNyX|�w £÷�Lbfa |;u[¦ �f]��;` u.} `YaczNb u £ w�y `s` uK�_Yb;��a | �;] u �Ç] u z w ac� � ` u[w ] u%� ��` u _Y`Yz w�y ` w ac` |�wN� ` u�w _ y bfzc` u � ¦ w�yX|�w{} `YaNzcb u��� wpyX| z u b´`V¨©`Y_ w b u y bC~ w�yX|0wH} `VaNzcb u ~^b;� � � ac`Yz } b u � w b wNy ` w ac` |�wN� ` u[w_ y bfzc` u dµzc`Y` w�y `´�:`V§ u ] w ]�b u b;�)] u z w ac� � ` u[w?|;� � | ac] | � � `Yz{] u �2`V_ w ]Bb u �2i e�i �fjGi³ y `�zN`Y_Vb u � | zNzN� �Q}:w ]Bb u £pb u ` ~ y ]B_ y ]Bz |;� ~ |ÿ¦ z �Q| ��` ] u `G� } `Yac] � ` u[w?|;�z w ����]�`YzY£±]Bz wNyX|�w zc���K¥c`Y_ w zÇaN`Vz } b u � w b w ac` |�wN� ` u�w ] u ��` } ` u ��` u[wN��¦ b��sb u `b wNy `YaVi&$ w�y `Va wNyX|;u�w�y `YzN` w ~�b | zNzN� �Q}:w ]Bb u zY£¤bf��a � b2�:` ��}��B| _Y`Vz u b _Yb uK�z w a | ] u[w z�b u6y bC~ w ` u ��` u _Y]�`Yz w b>aN`Vz } b u � w b w aN` |�w�� ` u[w z �6|ÿ¦ ] u[w `Va | _ w ~�] wNy_ y bf]�_Y`Vz |�� b u � w aN` |�w�� ` u[w zVi

ä«åcæ�å(' )+*^ñpî,*�è�é.-cô°ð¤ñpî�ë�ð¤ñpò�ç�ñ�ï:ëÔñ^ï�ï�*0/QìXëÝð¡ïtvu´|s} b } � �¶| a�_Yb �h} acb � ]�zN] u � |�}�} aNb | _ y6w b w�y ` } aNbf� � ` � b;�Ý] �Q} `YaÅ�L`Y_ w _Vb ���}�� ] |;u _V`;£Xac`YzN` | aN_ y `YaNz } `Vac�Lbfa � |;u21 ] u[w ` u[w±w b w aN` |0w43Q|;uX|;�4¦ zc]Bz�] u ~ y ]�_ y²w�y `_Yb u�w aNb ��|;u � w aN` |�w�� ` u[w �faNbf� } | ac`s_Yb �Q}X| aN`Y�Ç~�] wNy bf� w aN`Y� | aN� w b6~ y ` w�y `Vaw�y ` w aN` |�w�� ` u[w ~ | z | _ w � |;�B�4¦ ac`Y_Y`V]��;`Y� i65�³ y `ÒaN`VzN� �4w b;�¡zc��_ y²|;u.|�uX|;��¦ zc]Bz�]�z|M� ` | zN�:aN`Hb;� y bC~ ~�` ���Tw�y ` w aN` |�w�� ` u[w{90A?AG@��;S��6EGSX5T| ¨T`V_ w z wNy `H��]�zN` | zN`;£ | zb }�} bfzc`Y� w b wNy `H��`Yzc]Bac`Y� � ` | zN�:aN`Hb;� y bC~ ~�` ���©wNy ` w ac` |�wN� ` u�wÒ@858ACEG1 U�| ¨©`Y_ w zw�y `{��]�zN` | zN`�i7 #z w ] �Q|�w `Vz�� | zN`V�.b u ] u[w ` u[wc��w b ��w aN` |�w)|;uX|���¦ zN`Vz | ac`H� |�� ]B�´b u���¦| z � b u � | z w�y `M`J� } `Yac] � ` u[w?|�� _Yb u ��] w ]Bb u z } `Yac�L`V_ wN��¦ � ] � ]B_ wNy `M_Yb u ��] w ]Bb u z} ac`V� | ] � ] u �²] u�wNy `Ö`V�f` u[w � |;� ��z | �f`´b;� wNy ` w aN` |�w�� ` u[w i tvu·}X| a w ]B_V� �¶| aY£ w�y ``G� } `Yac] � ` u[w z y bf� � � � ] � ]�_HzN���K¥c`V_ w z98�] u _Y` u[w ]��f`VzR�Lbfa±aN`Y_V`Y]4�2] u ��` | _ yâw aN` |�wc�� ` u[w i tvu zc] w � |�w ]�b u z>~ y `VaN`ǧX` � ��] u _Y` u[w ]��;`Yz | aN` � bfaN`´_Vb �Q} ` ��� ] u � w�y�|;u`G� } `Yac] � ` u[w?|;� ] u _V` u[w ]4�f`YzV£ | zH]BzR��zN� |��B��¦Çw�y `M_ | zc`>~ y ` u ��aN�:�fzHaN`V_Y`Y]4�f` w�y `|;}�} aNbC� |;� b;� | �fbC�f`Va u�� ` u[w«| �f` u _ ¦ £ w aN` |�w�� ` u[w `V¨©`Y_ w ]4�f` u `YzNz �Q|ÿ¦ � | a ¦ zc]B� �

:<;!=?>A@CBED?DF�G9BIHJ=�>A@,H<KF�FMLON9P�Q RSK@�L�TVUWRSPM=?LYX[ZC\�P�G]BEDD?F�G_^�L`NLOabTcF�Kd�@�e

Page 6: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f×[�:¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉu ]4§X_ |;u[wN��¦ �LaNb ��| zczN]�� u�� ` u[w `V¨©`Y_ w ]4�f` u `YzNzVi7�Xbfa�`G� |;�h}�� `�£2] �6| �f] u ` | z w ��� ¦] u ~ y ]B_ y d | j wNy `���aN�:� yX| z |;uÑ| �:�;`YaNzc`�`V¨©`Y_ w b uÑ|´�¶| aN�f`>zc`Y� � ` u[w b;� w�y `} b } � �B|�w ]�b u�|;u �xdL�÷j�b u:��¦hw�y bfzc` � ` � �©`VaNz�b�� wNy `HzN`V� � ` u[w ~ y b���aNb } �LaNb �w�y ` w aN` |0w�� ` u[w.1c| a �f3 dLzN��� } b } � �¶|�w ]Bb u jTaN`Y_VbC�f`YaVi#³ y `^] u[w ` u[wÅ� w b � w ac` |�w�|;u�|;�4�¦ zN]�z�~�] �B�T|�wNw aN]B�:� w ` wNy `Yzc`{_ | zN`Vz±b;��aN`V_YbC�f`Va ¦Öw b wNy `Ò��aN���h��`Y_ | ��zN` w�y ` ¦Ç| ac`}X| a w b;� w�y `�] u[w ` u[wÅ� w b � w ac` |�w¡| a � £ |;��wNy bf��� y ] u aN` |;� ] wv¦Hw�y `VzN`�_ | zN`Vz#aN`Y_VbC�f`Yac`Y�� ¦Ñ9;D;-0@µ3;@8S��pw�y ` w ac` |�wN� ` u[w i

  u b w�y `Va |�}�} aNb | _ yhw b w�y ` } acbf� � ` � ]Bz w bÒ��zN` | _YbfacaN`Y_ w ]�b u � | _ w bfa#� | zc`Y�b ux|;u ] u z w aN� � ` u[w?|�� � | aN] | � � `VzÒ�Lb;a � � �B| dµ  u �faN]�z w ` ws|;� i#�ÿ×f×�g[jJ£ | _V_Ybfac��] u �w bM~ y ]�_ yÇwNy `H] u[w ` u[wc��w b ��w aN` |�w«� ` | zc��aN`Òz y bf� � �Ç��`Ò��]��K]���`Y�´� ¦�wNy `H�La | _ w ]Bb ub;�¡zN���K¥c`V_ w zR~ y b6_Vb �h}���¦ ~�] w�yÖw�y ` w ac` |�wN� ` u�w±| zczN]�� u `Y� w b w�y ` � i�  u �fac]Bz w` wH|;� i¡dÅ�C×;×�g[j)z y bC~^`V� wNyX|�w £ � u ��`VaH_Y`Va w?| ] u _Yb u ��] w ]Bb u zY£ wNy `M_Yb;aNaN`V_ w `Y�â�Lbfa �� � �B| ]Bzp� |;� ]B� �Lbfa wNy `�zN��� } b } � �¶|�w ]Bb u b;� 1 aN`Vz } b u zc]��f` 3 zc���K¥c`V_ w z ¯ wNyX|�w ]BzY£zN�:�K¥c`Y_ w z6~ y bÑ~^b;� � � yX| �f`._ y�|;u �f`V� w ac` |�wN� ` u�w z w?|�w ��z6]4�p�f]4�f` u�| �:]�¨©`Ya �` u[w^| zNzc]B� u:� ` u�w iih u �Lbfa w � uX|�w ` �4¦ £ w�y ]�z#zN��� } b } � �¶|�w ]Bb u _ |;u�u b w ��`�]B��` u�w ]�§X`V�|;u �T£ � bfac`²zc`Yac]Bbf��z ��¦ £«] w _ |;u:u b w zc`YaÅ�f` | z | � | zc]Bzh�Lbfa } b � ]B_Y]�`Yz�] u �;b � �K] u �w�y `h` u[w ]Bac` } b } � �¶|�w ]Bb u ��`Y_ | ��zc`Q] w ]BzH] u z w ac� � ` u[wc� ��` } ` u �:` u[w �¬] u �:]��K]B��� |;� z~ y b | aN`�ac`Yz } b u zc]��;`±] u�wNy `)z w �:� ¦��6|ÿ¦su b w aN` �6| ] u aN`Yz } b u zN]4�f`)] uMw�y `«§X` � � £~ y `VaN` wNy `>] u _Y` u[w ]��f`VzR�Lbfa±bf� w?| ] u ] u � w aN` |�w�� ` u[w �:]�¨©`Ya±�Lacb � w�y b;zN`s�:zN`Y� ] uw�y `sz w ��� ¦ i ª ` w�y `VaN`G�LbfaN`s�LbK_Y�:zRbf��a |;uX|���¦ zN]�zRb uÇw�y `>z w?| � � ` | z } `Y_ w b;� w�y `w ac` |�wN� ` u[wů wNy ` | z } `V_ w�wNyX|�w ~�bf� � ��aN` �6| ] u ] u � | aN] |�u�w�w b>_ y�|;u �f`Vz«] u _Vb ���}�� ] |;u _V`H�©` yX| �K]BbfaVi

Page 7: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê �f×;�ØÖÙEm n ßhá Ü à ãÿÜbo p qsárq¡qtsvuxw�nHr Þ qä«å('�åcæ y�ì�*{z.-°ë�è |�*�ì÷è~}.-°ð¡ïXô<*�ñ³ y `�� | zN]�_6`G� } `Yac] � ` u[w?|;� zN` wNw ] u � | zNzNbK_Y] |�w `V�x~�] wNy ] u ��]Bac`Y_ w `G� } `Yac] � ` u[w?|0�w ]�b u ]Bz)z y bC~ u ] u �¡]��f��aN`¬¸:i4�f£X~ y ]�_ y ]Bz�]Bzcb � bfa }�y ]B_ w b��¡]B�;��aN`VzR�:iW��dL�÷j |�u ��2i ×:i·³�b �LbK_Y��z w�y `Ç�:]BzN_V��zNzc]Bb u £�~�`Ç~�] �B� _Vb u zc]B��`Va |â} acb w b wv¦2} ]B_ |�� _ � ] u ]B_ |;�w ac] |;� ~�] wNy´}X| a w ] |;� _Yb �h}�� ] |;u _V`;£ |;��wNy bf��� y ] u �f` u `Ya |��Tw�y ` � b2�:` � |�}�}�� ]�`Yz w b|;u[¦ z w ��� ¦ ] u ~ y ]B_ yÁ| a |;u ��b � ]B�V`Y�·] u z w ac� � ` u[w ` u _Vbf��a | �f`YzhzN�:�K¥c`Y_ w z w b_ y bKbfzN`Òb u ` } aNbf�fa |;� bC�;`Ya |;u b wNy `YaVi

������

������������

������

��

� � ��� ��� �� � ��� �

³�ac` |�wN� ` u�w RzczN]B� u `Y� � |0w ` u[w� | _ w bfaNz³�ac` |�wN� ` u�w� `V_Y`V]��f`V�

$p��zN`Vac�f`V�� `Vz } b u zN`�¡]��f��aN`>¸:i��;���Òa |;}�y ]�_ |;� aN` } aN`Yzc` u[w?|0w ]Bb u b;��_ | ��z |;� �:` } ` u ��` u _Y]�`Yz{] u | a |;u2���b � ]B�V`Y�Ö_ � ] u ]�_ |;�÷w ac] |;� ~�] w�yÖ}X| a w ] |;� _Vb �h}�� ] |;u _Y`�i

ª ` | zczN� � ` wNyX|�w��V�.�b��� | aN`hbf��zc`Yac�;`Y���:] uX| a ¦ � | aN] | � � `YzY£�~ y `VaN` �aN` } aN`VzN` u�w z w�y `>dµa |�u ��b � ]B�V`Y�÷j w aN` |�w�� ` u[w�| zNzc]B� u�� ` u[w £ � ]�z wNy ` w ac` |�wN� ` u[w| _ w � |;����¦ aN`V_Y`V]��f`V� £ |;u � � ]Bz w�y `´bf�:zN`YaÅ�f`V�Áac`Yz } b u zc`;i�³ y `�� w `Va � aN` }K�aN`VzN` u�w z |;�B� � | _ w bfaNzV£���b wNy bf��zN`Vac�;`Y� |;u �·� u bf�:zN`YaÅ�f`V� £ wNyX|�w ] u�� ��` u _Y` w�y `~ |C¦ | zN����¥c`Y_ w ac`Yz } b u �:z w b w ac` |�wN� ` u[w z û�y ` u _V`;£ |;u | acaNbC~]BzR�:a | ~ u �LaNb �� w b � i�³ y ` | acaNbC~ �LaNb � � w b � ��` u b w `Yz wNyX|�w)w�y `��� | _ w bfacz �Q|ÿ¦²|;� zcb] u%� ��` u _Y` w�y ` zN����¥c`Y_ w 8 zÖ_ y b;]B_Y`âb�� w aN` |�w�� ` u[w���ûRw�y ]�z���` } ` u ��` u _Y` �6|ÿ¦aN` } aN`VzN` u�wR| _Yb �h}�� `G�´�:`Y_Y]�zN]�b u´} aNbK_Y`VzNz)z w�|;u ��] u �M�©` w ~^`V` u.wNy ` | zczN]B� u�� ` u[wd � j |;u � w�y ` | _ w � |;� w aN` |�w�� ` u[w d � jGi

³�bH� | _V] � ] w?|0w ` w�y ` u b w?|�w ]Bb u £�~�` � ` w����i�C�Y� ac` } ac`YzN` u[w dLaN`Vz } `Y_ w ]��;` ��¦ j w�y `� |;� �:`Yz w�| ú;` u � ¦²w�y `>� | ac] | � � `Yz �V���b��� £T~�] wNy w�y `{�Lb �B� b�~�] u �h] u[w `Va } ac` w?|��w ]�b u �

������W�c�_� 5_� �{� 5 | zNzN`Va w z w�y�|�w¤w aN` |�w�� ` u[w�yX| z¤��`Y` u�| zczN]B� u `Y� d �W� £�] w z u `Y� |��w ]Bb u j û

Page 8: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f×;eX¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ�����c� �c�¡� 5_� �7� 5 | zNzc`Ya w z w�yX|0w�w aN` |0w�� ` u[w�yX| z��©`V` u´| � � ] u ]Bz w `VaN`Y� d � � £K] w z

u `V� |�w ]Bb u j ûX|;u ����¢�?�£�9�¡� 5_� �¤� 5 | zNzc`Ya w z |Ò} bfzN] w ]��;`�bf��zN`Vac�;`Y��ac`Yz } b u zc`Md �£� £�] w z u `Y� |0w ]Bb u jGi³ y `)��b �Q| ] u b��`�¢aN` �6| ] u z#� u z } `Y_Y]4§X`Y� |;u � �6|ÿ¦ £K] u �f` u `Va |;� £�_Yb � �:] u ` w�y `z }�| _Y`Vz±b;��zN`G�f`Va |;� a |;u ��b � � | ac] | � � `YzV£���b w�y ��]�zN_Yac` w ` |�u �²_Vb u[w ] u �:bf��zYi

³ y `¬�fa |;}:y ]B_ |;� � b2��` � ac` � `V_ w z w ~^b | zczN� �h}:w ]�b u zVi�fi�³ y ` | zczN]B� u `Y� w ac` |�wN� ` u�wi� ��b2`Vz u b w ] u�� ��` u _V` � ��]�aN`Y_ w��4¦ ��� w a |0w�y `VawNy aNbf�:� yÇwNy ` | _ w � |;� w aN` |0w�� ` u[wS� i tvuÇ} a | _ w ]B_Y`�£ |;u[¦ ��]Bac`Y_ w `G¨T`V_ w��� ]B� y[w´y�| �;`�b u¥� ~�bf� � ����` | �0¥c��z w `Y�¢�Lbfa w�y aNbf��� y�wNy ` ��zN`Ñb;� |}��B| _Y`V�©b:i

�2i�³ y `¤� | aN] | � � `Yz ��|;u �¦� | ac` �6| ac�f] u�|;�B�4¦ ] u ��` } ` u ��` u�wYûfw�y ]�z�]Bz�` u zN��ac`Y�wNy aNbf�:� yMw�y `«a |;u ��b � ]B� |�w ]�b u b�� � £f~ y ]B_ y aN� � `Yz#bf� w�| _Yb �h� b u _ | ��zc`�Lbfa«��b wNy�� |;u �b�{i

³ y `Yzc` | zNzc� �Q}2w ]Bb u z^] �Q} bfzN`)b u6wNy `�¥cbf] u[w ��]�z w ac]B��� w ]Bb uhwNy `p��`V_Yb �h} b;zN] �w ]�b u §

d �¨���C�_���4© j&ª §d �¬« �C�4© j § d ��«­�%�4© j § d � j § d © j � dµ¸:i��Cj

~ y ]�_ y £:b;�Ý_Vbf��aczN`;£2_ |;u:u b w ��`pbf�:zN`YaÅ�f`V�Ç�:]BaN`V_ wN��¦ ��`Y_ | ��zN`®� ]Bz�� u b;��zN`Vac�f`V� i�±bC~�`V�;`YaY£ w�y ` �Q| aN�;] uX|;� ��]Bz w aN]���� w ]Bb u §d �¨���C�_� j |;u � £^] u�}�| a w ]B_Y� �¶| aV£ w�y `

_Yb u ��] w ]Bb u�|;� ��]�z w aN]B�:� w ]�b u z§d �¨�¡��« � jiª°¯�± §

d �C« �C�4© j § d ��« ���4© j § d © j ��������W�9�_� 5E� � dµ¸:ik�;j

| ac`6bf��zc`YaÅ�f`Y�T£k² |�u � w�y `�_ yX|;��� ` u �f`Q]�z w b | zNzc`Yzcz>�LaNb �ów�y `YzN`6��]Bz w aN]���� w ]Bb u zw�y ` | �;`Ya | �f` =?7:9;S��KE ] u³� ���:` w b w aN` |�w�� ` u[w i

³�ac` |�wN� ` u[w `V¨©`Y_ w z | aN`¢�fbC�f`Ya u `Y� � ¦øw�y ` ��]�z w ac]B��� w ]Bb u§d �C«­´¶µ d � jcjG£

~ y ]�_ yK¯ ��zc] u � w�y ` w aN� u _ |�w `Y�&� | _ w bfac]B� |�w ]�b u �Lbfa � � �¶| b;�QdI�:i4�c�[j ¯ ]�zh�;]��f` u� ¦ §

d �C«­´¶µ d � jcj·ª ¯ ± §d �¬« �C�4© j § d © j û dµ¸:iW�fj

¸O¹�N{DFMB�H(P�>AH<L�º_GE»�H<GIKFM@�LIº_G�N?QkRYB¬¼N?>kP�L,@MBE½�D?Q L!GE»¿¾]À�Á£Â�Ã�Ä Å9Æ%a,> QkQcU£L!GIU@�LOFM^IL¡TÇe�ÈÉKcP�GIKFPMB�@ËÊi>A@�G�N?LÌGI»?> T?LON9P�>k¼¿HOBEP�> GIN¶º_NGEP%L¡@ÍPM>k½]BEP�> GIN¶ºE@�GCaÉL̽]BEÊ�L P�=LÌQABEFMdIL<Î�@MBE½�D?Q LÌBI@M@ËK½7D?P�> GINBEN¿TVHOGIN@�> T?LOFC¾]ÀÏÁ�Â�Ã�Ä Å9ÆÉBI@!d�>k^�LON¶e

Page 9: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê �f×��y `VaN`;£ w�y `Ö� | _ w bfaNz

§d �C« �C�4© j |;u � §

d © j | aN` w�y `Çz |�� ` | z w�y b;zN`Ç] u dI¸:iW�fjGi³ y `VaN`V�Lb;aN`;£:]�� ~�` | ac`R] u[w `YaN`Vz w `Y�´] uhw�y ` | �;`Ya | �f`p_ yX|�u �f`p] u�� �:��` w b w aN` |�wc�� ` u[w�w�y ` u ~^`Hz y bf� � �´_Yb �h} � w ` w�y ` 9;D;EGFc9c�KEs=N9;/[AY9;1ÔEÍÐpE?=G5 £: )\� ±d �ÒÑ � jd��)b �B�B|;u �â�C×;¸f¸[jG£K~ y ]B_ y ]Bz��f]4�f` u � ¦

 )\� ±d �ÓÑ � jÔª §d � 5 « ´¶µ d � 5 jNj·Õ §

d � 5 «­´¶µ d � � jcjª ¯ ±³Ö § d � 5 « � 5 �¡© j¬Õ §d � 5 « � �c�4© j § d © j<×�Ø dµ¸:i e[j

$p��a w?| zcú6]�z w�y ` u´w bM`Yz w ] �6|�w `pbfa���bf� u � w�y `H`J� } aN`YzczN]�b u ] u dI¸:i e�j��;]��f` uw�y `Ñb;��zN`Vac�f`V� } acbf� | �:] � ] w ]�`Yz

§d �¨�¡��« �W� j |;u � §

d �Ì�¡��«­� 5 jG£ | zÖ`G� } aN`VzNzc`Y� ] udI¸2ik�fjJiê³ y ]Bz w?| zcú |;� b;� u[w z w b | _Vb u z w a | ] u `V� b }:w ] � ]B� |�w ]�b u `G�2`VaN_Y]�zN`Ñb;�§ u �:] u � wNy ` y ]�� y `Vz w�|;u � � bC~�`Yz w � |�� ��`Vz>b;�{dI¸2i e�jÒzN����¥c`Y_ wMw b w�y `�`Y®�� |;� ] wv¦_Yb u z w a | ] u�w ] u dµ¸:ik�;jG£�~ y `Yac` w�y ` �Q| �2] � ]B� |�w ]�b u a |;u �;`Yz�bC�f`Va |;�B�)} b;zNzN]�� � `�L� u _ w ]�b u z§

d © j � §d � 5 « � �c�4© j � §

d � 5 « � 5 �4© j � §d � 5 «­�W�W�4© j �±|�u � §

d � 5 «­� 5 �4© jw�y�|�w z |�w ]BzÅ� ¦�w�y b;zN`Ò_Yb u z w a | ] u[w zYi

ä«å('�å(' /ÚÙpë õ³Û{*7-O}±ïXô<*�ñ *�íÜy³*¤ï:ëÔñ�ï�ô°ðY-JÝ¿Þ ëÝù�é®*^ñpù2ë ß�ð#ì÷ôMÝðYz.-ÅëÝù

³ y `Q�©bf� u ��] u �Ç`J�2`YaN_V]Bzc`6��`VzN_Yac]B��`Y��] u �K`Y_ w ]Bb u ¸:iW�2i4�Q_ |;u ��`Qzcb � �;`Y����zN] u �_Yb u �;` u[w ]�b uX|;��w `Y_ y:u ]B®���`VzQb;� �Q|�w�y ` �Q|�w ]�_ |;� b }:w ] � ]B� |�w ]�b u ib�)bC~^`G�f`YaV£ w�y `_Yb u�w ] u ��bf�:z uX|�w ��aN`pb�� w�y `R�L� u _ w ]Bb u z^] u �fb � �f`V� ¯¢| z�~^` �B�T| z w�y `p� u z } `V_Y]�§�`Y���b �Q| ] u b��`� ¯��Q| ú;`Vz wNy ]Bz¤ac` } ac`YzN` u[w?|�w ]Bb u ] u _Vb u �f` u ]B` u[w �Lbfa#_Yb �h} � w�|�w ]�b u itvu z w ` | � £©~�`s_ |;u ��zc` wNy `sbf�:zN`YaÅ� |�w ]�b u²wNyX|�w �_ |;uâ|�� ~ |C¦ z)�©`{aN` }��¶| _V`Y�²� ¦| § u ] w ` � z w?|�w `¬� | ac] | � � `ÒzN��_ y wNyX|�wRwNy `sac`Yzc� ��w ] u � � bK��` � ]�zR`Y®���]4� |;� ` u�w ~�] wNyaN`Vz } `Y_ w¡w b |;�B� bf��zN`Vac� |�w ]Bb u z |;u � �6|�u ] } � �¶|�w ]Bb u zTb;� �V�C�b�`� dË��` | a � �C×f×;e | jGi

\^b u zc]B��`Va w�y `^z w aN��_ w ��a |;� `Y®�� |�w ]Bb u¬w�yX|�w _Yb u�u `Y_ w z w ~^bp�:] uX| a ¦ � | aN] | � � `VzY£�ø|;u � � £X] u²| _ | ��z |���� bK��` � �� ª°à¡d �C�4© jIØ

�Xbfa |;u[¦ �f]4�f` uf© £ w�y `�aN` �¶|�w ]Bb u z y ] } �©` w ~^`V` u��(|;u � � � ��z w ��`)b u `±b��©�Lbf�:a�L� u _ w ]�b u zY�

à � � � ª¥� � à 5 � � ª �C�à ² � �r᪠�C� à?âR� � ª���Ø dµ¸:iA�;j

Page 10: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f×�g�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ Rz © � | ac]B`Vz |�� b u �M] w z���b �Q| ] u £:ac`Y� | ac� � `VzNz�b;� y bC~ _Yb �Q}�� `G� wNy `H� | aN] |�w ]�b u £w�y `Qb u:��¦ `G¨T`V_ w ] w _ |;u�yX| �f`6b uÑwNy ` � b2�:` � ]Bz w b²zÅ~�] w _ y�w�y `haN` �¶|�w ]Bb u z y ] }��` w ~�`Y` u³� |;u � �ø|;� b u � w�y `YzN`Ò�Lbf��a��L� u _ w ]Bb u zVi�³ y ]Bz }X| a w ] w ]Bb u z w�y `¬��b ��Q| ] u b;�¤�] u[w b��Lbf��a�`Y®���]4� |;� ` u _Y`¬_ �B| zNzc`YzY£ | z)z y b�~ u ] u �¡]B�f�:aN`Ò¸:iW�2£X~ y `Yac`` | _ y _ �¶| zNz¡_Vb u[w?| ] u z wNy bfzc` } b;] u[w z ©Mw�yX|�w _YbfacaN`Vz } b u � w b w�y `^z |;� `��L� u _ w ]Bb u iª `h_ |;u w�y ��zÒaN` }��¶| _V`��ø� ¦Ñ| �Lb;��a � z w?|0w `>� | aN] | � � `;£,ã�d © jJ£ÝzN��_ yÑw�yX|�w ` | _ y

yr

yryr

yr

y=x

y=x

Domainof U

=0

=3=2

=1

y=1

=0y

�¡]��f��aN`^¸:iW�2��³ y ` }X| a w ] w ]Bb u b;�,�&] u[w bp�Lbf��a�`Y®���]4� |�� ` u _Y`�_ �¶| zczN`YzV£�` | _ y ] u ����_ �] u � | �:]Bz w ] u _ w �L� u _ w ]Bb uX|��Ô�Q|;}�} ] u ���LaNb �ä� w b � �Lbfa |;u[¦ �f]4�f` u �L� u _ w ]�b u� ªxà+d �C�4© jJiz w�|�w `Hac` } ac`Yzc` u[w z)b u `Hb;� wNy `p�Lbf��a^�L� u _ w ]Bb u zVi#³ y ` } aNb;� | ��] � ] wv¦

§d © j¤~�bf� � �

| � w b �Q|�w ]�_ |;����¦�w a |;u z �B|�w `Ò] u�w b |h} aNb;� | ��] � ] wv¦ �L� u _ w ]�b u§dËå;j � 宪+� � � � � � �:£

w�y�|�w ]BzH�f]4�f` u � ¦ w�y ` w b w?|;� ~�`Y]�� y[w>| zNzN]��f`Y� w b w�y `h`V®���]�� |;� ` u _V`h_ �¶| zczÒ_Ybfa �aN`Vz } b u ��] u � w bæåKi� �z w�|�w ` �v� ] u ] �6|�� � | aN] | � � ` � ]Bú�`�ã�]�z±_ |;�B� `Y� |Ú1 ac`Yz } b u zc`� | aN] | � � ` 3 � ¦ g |;� ú;` |�u �ç��` | a � dÅ�C×f×�e | £��÷j |;u � |Ú1c�6|�}�} ] u �Q� | aN] | � � ` 3 � ¦�±`V_?ú;`Va �Q|;u |;u � � y�| _ y[w `YasdÅ�ÿ×f×Ç�fjJi â

g^`Y_ | ��zc` �V�]� £ |;u � � | aN` |;��� ��] uX| a ¦ � | aN] | � � `YzY£ w�y `Òz w?|�w `Hz }X| _V`{b��Y���]4�2]���`Yz¬] u[w b��cg E¡èY/K@8D09;1BEGST=?E�=G1490A?ACE?A �{` | _ y _ �¶| zcz{��]�_ w�|�w `Vz w ~�b.�L� u _ w ]�b uX|;��Q|;}�} ] u �fzY£©b u `s�Lacb �Ò��w b � |;u � wNy `>b wNy `Yap�Lacb �ä� w b � i¬³Ôb´��`VzN_Yac]B��`w�y `YzN`�`V®K�:]�� |;� ` u _Y`M_ �¶| zczN`VzY£�] w ]Bzp_Vb u �f` u ]B` u[wÒw b´ac`Y� | aN� ` | _ y b;� wNy ` ��| z |éO¹�N�PM=?L·D¿GIP�LON9PM> BIQêÎëGIKcPJH<G�½7L·½�GcTcLOQ�À�@�LOL�ìWL¡H(P�> GIN�íWe îe î9Æ<º�ï.@ËPMBINT?@¨»ÏGIF,BEN�L<ðcD¿L�F�> ½�LONcÎPMBIQ[K?N?>kPiBINT�ñ�ÀÏïÇÆ`H<GIFMFML�@�D¿G�NT?@,PMG�PM=?L7D¿GIP�LON9PM> BIQ�FML�@�D£GIN@�L¤GI»ÌK?N?>kP·ï�PMG�PMF�L¡B_P�½�L�N�P·Ã�e;!=?L�BI@M@�K?½�DcP�> GIN�P�=BEP!L¡BIHJ=VL<ðcD¿L�F�> ½�LON9PMBIQ�K?N?>kP�À�LIe de º?BEN>kN¿Tc>k^W>ATcKBIQ�@�K?UcòÍL¡H(P(ÆÉD¿G9@�@�L�@M@ËL¡@BEN> N�PMF�> N@�> HIº?@ËL�LO½�> N?dIQ R�óÍ»�BEPMBIQk>A@ÍPM> H�ô¤FML�@�D£GIN@�L·»ÏK?NH(PM>kG�N�=BI@ɽ�L<PCa,>kP�=.@�GI½�LiGIUcòÍL¡H(P�> GIN¿@À�Z·B4a,>ATfõ¡öIö9íIÆ(ºÇG_a,> N?dP�GP�=L�HOGI½�D?Q L<ðc>kPÍR�BENT®> N?=?LOFMLON9P�K?N?G�U@�LOFM^4BIU?> Qk>kPÍR�GE»¨P�=?L]½]BIN9R»�BIH<P�G�FM@iPM=B_P{½7> dI=9P¤d�G_^IL�F�NtBEN¦> NTc> ^W> TcK¿BEQÌFML�@�D£GIN@�L]P�G�PMF�L¡B_PM½7L�N9P�e�;!=?LSL¡÷9K?>k^_BIQkL�NH<LOÎH<QABI@M@!»ÏG�F�½{K?Q BEP�> GIN.GI»�ñ�À�ï�Æ!½�>kP�> d�BEP�L�@CP�=?G9@ËL¤GIUcòÍL¡H(P�> GIN¿@,UWR@�=?G_a,> N?d�P�=BEP·ñ�ÀÏïÇÆ!L�^IGIQ ^IL¡@NBEP�K?FJBEQ QkR�BENT�½]B_PM=?LO½]BEP�>AHOBEQ Q R�»ÏFMGI½xBINWR®H<G�½�D?QkLOð�@�Rc@ÍPMLO½øGI»C@ÍPMGWHJ=¿BI@ËP�>AH�Q BEP�LON9PY^_BEFM>kÎBEUQkL¡@Oº¶DF�G_^W>ATcL�T®GIN?Q R.PM=B_PYaÉLSBIHJÊWN?G_a,Q L�Tcd�LYPM=?L]L<ðc>A@ÍPMLONHOL7GI»!@�KHJ=�^_BEFM> BIU?QkL¡@·P�=?FMGIKdI=P�=L.L�÷9KB_PM>kG�N�Áúù°û[À�Ã�ÂËïÇÆ(eü;!=G�@�La,=G¦>kNW^IG�ÊIL�÷9KBEN9P�K½{Îë½�L�HJ=BIN?>AHOBEQ!G�UcòÍL�H<P�> GIN@7PMGP�=L`QAB_P�P�LOF!@ËP�LODBI@¨a¨L�QkQ[ÀÏL�e d¿e?ìcBEQ ½�GIN�õ¡öIöIý9Æ(º9@Ë=?G�K?QATSFMLOd�BIFMT{PM=?L¬»ÏKNH(PM>kG�NBEQ�FMLOQAB_P�> GIN¿@Ë=?> DÁ�ùúû[ÀÏÃ%ÂËïÇÆ BI@¨BENSBIU@ÍPMFMB�H(P ½]B_PM=?LO½]BEP�>AHOBEQ£H<G�N@ÍPMF�K¿H(P�º9FMLOD?FML�@�LON9P�> N?d·P�=L¬L<ðWPMF�L�½7L¬D¿G�>kN9PM@ÀÏ^�LOF�P�>AH<L¡@MÆÉGI»%PM=?LY@ËLOPCGE»�H<G�NTc>kP�> GINBIQÇD?F�G�UBEU>kQ >êPM>kL¡@C¾]ÀÏÁ¶Ä Ã�Â�ï�Æ!@MB_PM> @Ë»ÏRW>kN?d7PM=?LYH<G�N@ÍPMFMBI>kN9PJ@GE»,À�ýekõ4ÆÉBENT�À�ýe þ�Æ(e

Page 11: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê �f×f�} bf] u[w ] uâwNy `R¥cbf] u[w z }�| _Y`�b;� w ~^bÖ�Lbf��a � � |;� �:`Y�â� | aN] | � � `Vz.ãVÿ |;u �¢ã��0i�³ y `� | aN] | � � `�ãVÿ6�:` w `Va � ] u `Vz w�y `h_Yb �h}�� ] |;u _V`M�©` yX| �K]BbfaÒb;� | zc���K¥c`Y_ wswNy aNbf�:� yw�y ` �Q|;}�} ] u �

� ª°à��Hd ��� å9ÿ0jYª���������� ���������

� � ]�� å9ÿª¥� û� � ]�� å9ÿª�� |;u � � ª �W�W�� 5 ]�� å9ÿª�� |;u � � ª � 5 û� 5 ]�� å9ÿª � |;u � � ª �W�W�� � ]�� å9ÿª � |;u � � ª � 5 û� 5 ]�� å9ÿª ��Ødµ¸:i gfj

tv� ��` u z |;u � � �:��] u dÅ�ÿ×f×[�fj+_ |��B�X| zN���K¥c`V_ w ~�] w�y _Yb �Q}�� ] |;u _Y`���` yX| �K]Bb;a{å9ÿª� � � � � � �HdLaN`Vz } `Y_ w ]��;` ��¦ j |HSTEGD;EGFE"v5 9�[EGF £ |s=?-���1P@�EGF £ |{3KEÍ#�EGF £ |;u � |;uQ9;1��9;OCAI"5�9�[EGF i��K] � ] �¶| a ��¦ £ w�y `p� | aN] | � � `Vã��p�:` w `Va � ] u `Vz w�y `HaN`Yz } b u zN`Ò��` yX| �K]Bbfa^b;� |zN�:�K¥c`Y_ w±wNy aNbf�:� yÇwNy ` �6|�}�} ] u ���

� ª¥à��d �C� å �ÿj�ª���������� ���������

�£� ]�� å ��ª¥� û�£� ]�� å ��ª�� |;u � � ª � �c�� 5 ]�� å ��ª�� |;u � � ª � 5 û� 5 ]�� å ��ª � |;u � � ª � �c��£� ]�� å ��ª � |;u � � ª � 5 û� 5 ]�� å ��ª �%Ødµ¸:ik�;j

�Xb �B� bC~�] u ���)`Y_?ú;`Va �Q|;u´|;u �´� yX| _ y�w `YaÒd°�C×f×Ç�;jG£[~^`±_ |;����w�y `)aN`Vz } b u zc`R��` yX| � �]Bb;a�å ��ª¥� � � � � � �ÖdµaN`Vz } `Y_ w ]��;` �4¦ j STEGD;EGFE"vF�E?=?-0D;EGF £ 7XEG1 �XEN3 £ 7K/KFJ5 £ |;u � 9;1��9;OCAI"F�E?=?-0D;EGF i

³ y `M_Yb;aNaN`Vz } b u ��` u _Y`M��` w ~�`Y` uÑwNy `>z w?|�w `Yzpb;�#� | ac] | � � `®ã�� |�u � w�y ` } b �w ` u[w ] |�� aN`Yz } b u zN`¤� | aN] | � � `VzY£ � ÿ�� |�u � � ÿ��?£0��`G§ u `Y�s] u �2`Y_ w ]�b u �Ki��RdIþp`G§ u ] w ]Bb u�2i4�fi e�j^]Bz | z«�Lb ��� bC~�zY�

� ÿ���ª � � 5 ]��¬å �Sª��ÒbfaSå ��ªÓ� ��£� b wNy `YaÅ~�]BzN` û� ÿ��7ª � � 5 ]4�·å���ª �Mbfa]å ��ª¢� ��¿� b w�y `Vac~�]Bzc`�Ø

tvu �f` u `Va |;� £faN`Vz } b u zc` |;u ��_Yb �h}�� ] |;u _V` �Q|ÿ¦su b w ��`«] u ��` } ` u ��` u�w £ y ` u _V`w�y `���b;��� � ` | acaNbC~&ãVÿ � �®ã��¤] u �¡]B�;��aN`¤¸:i �:iÔ³ y `Ý¥cb;] u[w ��]Bz w ac]B��� w ]Bb u bC�f`VaãVÿ��vã��.aN`V®K�:]BaN`Vzâ�?�x] u ��` } ` u �:` u[wÇ}X| a |�� ` w `YaczY£ |;u � wNy `Yzc` }�| a |;� ` w `Yacz

Page 12: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

�f×f¸�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ

Z

X

Y

2 3

0

0 1 2 3

1 R

R

R0 1 x

z

y

�¡]��f��aN`p¸:iW�2�+  z w aN��_ w ��ac`¬`V®���]�� |;� ` u[w«w b wNyX|�w b;�`�¡]��f��aN`p¸:i��H�:� w ` �h}�� b ¦ ] u �§ u ] w ` � z w�|�w `Hac`Yz } b u zN`Ò� | aN] | � � `VzSã�� � ãVÿ[£ |;u �úã���i| ac`¬zc�:ýh_Y]�` u[w �Lb;a)z } `V_Y]4� ¦ ] u � w�y ` � b2�:` � b;�·�¡]B�;��aN`Ò¸:i �:£ §

d �Ì�¡�C�_��� å9ÿ � å��ÿjYª§d �C« �C� å �Vj § d ��« å9ÿ �_� j § d � j § dËå9ÿ � å �YjG£���`Y_ | ��zN` �¢|�u � � z w?|;u �{] u §:�2`Y�¬�L� u _ �

w ]�b uX|;� ac` �¶|0w ]Bb u z w b w�y `Y]Ba }X| ac` u[w zH] u²wNy `>�fa |;}�y i±³ y `>_ | ��z |;� `V¨©`Y_ w b;� w�y `w ac` |�wN� ` u[w _ |;uÖu bC~ ��`Hbf� w?| ] u `V�Ç�:]BaN`V_ wN��¦ �Lacb � dµ¸:ik�;jG£��f]4�K] u �§

d � 5 «­´¶µ d � 5 jcj ª §dËå �¡ªR��j�� §

dÍå ��ª��[j � dµ¸:iW¸fj§d � 5 «­´¶µ d � � jcj ª §

dËå �¡ª«�fj�� §dÍå ��ª��[j � dµ¸:iW×fj

|;u � )\� Rd � Ñ � jYª §

dËå���ªR��j·Õ §dÍå ��ª«�fjEØ dI¸:i4�c�[j

ä«å('�å�� �hôcñpëÝð#ì�yÑì�* ��ìXð¤èêè�ôÅñ!� |�*�ì÷è~}.-°ð¡ïXô<*�ñg ¦ `G� }�� ]�_ |�w ] u � w�y `âac` �¶|0w ]Bb u z y ] } ��` w ~�`Y` uÓw�y ` }X| a |�� ` w `Yacz�b;�

§d �Ì�¡��«­� j

|;u � w�y b;zN`¬b;�§dËå9ÿ � å �YjG£X~�`{bf� w?| ] u.| zN` w b;� � ] u ` | a«_Vb u z w a | ] u[w z u `Y`V��`Y�.�Lb;a

� ] u ] � ]��Y] u �{bfa �Q| �2] � ]B�V] u �{ )\� Rd � Ñ � j��f]4�f` u §d �Ì�¡��«­� jGi

³ y `p_Vb u ��] w ]Bb uX|;� ��]Bz w ac]B��� w ]Bb u§d �Ì�¡��«­� j#bC�f`Va w�y `±bf��zc`Yac� | � � `±� | ac] | � � `Yz

]Bz��L� �B��¦ z } `Y_V]�§X`V�.� ¦ `V]B� y[w�}X| a |;� ` w `VaNzV£�~ y ]B_ y ~�] ��� ��`Ò~�ac] wcw ` u²| z��Lb ��� bC~�zY�" �(�$# � ª §

d �£�W��� �£«­�W� j � " �(�$# 5 ª §d �£���¡� �£« � 5 j �" � 5 # � ª §

d �£�W��� 5 «­�W� j � " � 5 # 5 ª §d �£���¡� 5 « � 5 j �" 5 �$# � ª §

d � 5 ��� �£«­�W� j � " 5 �$# 5 ª §d � 5 �¡� �£« � 5 j �" 5(5 # � ª §

d � 5 ��� 5 «­�W� j � " 5(5 # 5 ª §d � 5 �¡� 5 « � 5 jIØ

Page 13: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê �f×;׳ y ` } aNb;� | ��] � ]Bz w ]�_±_Vb u z w a | ] u[w z5(5¯%�& �(� " % # � ª�� |;u � 5(5¯%�& �(� " % # 5 ª�� dI¸:i4�f��j�L��a w�y `Va¬] �Q}:��¦Öw�yX|0w('" ª(d " �(�$# �c� ØWØWØ � " 5(5 # 5 jR_ |;u �©`Mz } `V_Y]�§�`Y�x� ¦ |´} bf] u[w ] ug � ��] � ` u zc]Bb u�|;� z }X| _V`;i�³ y ]Bz�z }�| _Y`H~�] ��� ��`HaN`V�L`VaNac`Y� w b | z §

i³ y `�¥cbf] u[w«} acbf� | �:] � ] wv¦

§dËå�ÿ � å �Yj yX| zÒ�cg }X| a |;� ` w `YaNzV�)$*,+ ª §

dËå9ÿ4ª.- � å �¡ª0/�j �~ y `VaN`�- � / �¢� � � � � � � � � i¡³ y ` } acbf� | �:] � ]�z w ]B_)_Yb u z w a | ] u[wâ¯* & �

â¯+ & � )$*,+ ª �] �h}�� ]B`Yz w�yX|0w1') z } `Y_Y]4§X`Yz |Ò} bf] u[w ] u �?� � ��] � ` u zN]Bb uX|;� z }X| _V`;i#³ y ]�z#z }X| _V`)~�] �B���`¬ac`V�L`VaNaN`V� w b | z32hi #®K� |�w ]�b u dI¸:i4�c�[j�_ |;uhu bC~���`�aN`V~�ac] wcw ` uh| z |H� ] u ` | a¡_Vb � ��] uX|�w ]�b u b;� w�y `2 }X| a |;� ` w `VaNzV� )\� ±d �ÓÑ � jYª ) � 5 � ) 5(5 � ) ² 5 � ) â 5 Õ ) � ² Õ ) 5 ² Õ ) ²(² Õ ) â ² Ø dI¸:i4���fj

  }:}���¦ ] u �´dI¸2i g[j |;u � dI¸:iW�fj#~^`±_ |;u ~�ac] w ` wNy ` � ] u ` | a w a |�u zc�Lbfa �6|0w ]Bb u �Lacb �ø|} bf] u[w4') ] u 2 w b |�} bf] u�w5'" ] u §

�" �(�$# � ª ) �(� � ) � 5 � ) 5 � � ) 5(5 � " �(�$# 5 ª ) �(� � ) � 5 � ) ² � � ) ² 5 �" � 5 # � ª ) ² � � ) ²(² � ) â � � ) â ² � " � 5 # 5 ª ) 5 � � ) 5 ² � ) â � � ) â ² �" 5 �$# � ª ) � ² � ) � â6� ) 5 ² � ) 5 â � " 5 �$# 5 ª ) � ² � ) � â6� ) ²(² � ) ² â �" 5(5 # � ª ) ² 5 � ) ² â6� ) â 5 � ) â(â � " 5(5 # 5 ª ) 5(5 � ) 5 â.� ) â 5 � ) â(â ûw�y ]Bz«_ |�u ��`H~�aN] wcw ` u ] uÖ�6|�w aN] �6�Lbfa �'| z '" ª87 ') i�Ò]4�f` u�|R} b;] u[w9'" ] u §

� z }�| _Y`�£ w�y `^z w ac]B_ w+� bC~�`Ya+��bf� u �>b u  )\� ±d �ÒÑ � j_ |�u ��`>��` w `Va � ] u `V�Ñ� ¦ zNb � �2] u � wNy `>�Lb �B� bC~�] u � � ] u ` | a } aNbf�;a |;�h� ] u � } aNbf� �� ` � i: ] u ] � ]B�Y` ) � 5 � ) 5(5 � ) ² 5 � ) â 5 Õ ) � ² Õ ) 5 ² Õ ) ²(² Õ ) â ²zN�:�K¥c`Y_ w±w b�� â¯* & �

â¯+ & � );*<+ ª � �7 ') ª '" � dI¸:i4�C�[j);*<+ = ���Lb;a>- � / �¢� � � � � � � � � i

Page 14: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

eÇ����¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ: bfac`YbC�f`VaY£H�Lbfa } acbf� � ` � zÖb;� w�y ]BzÖzN]��Y`;£ } aNbK_Y`V����aN`Vz | ac` | � | ] �¶| � � `²�Lb;a

��`VaN]��K] u ��z ¦K� ��b � ]B_¬`G� } aN`VzNzc]Bb u z | z)~^` �B� dIg |;� ú;`��ÿ×f×Ç�fjJ£ � ` | ��] u � w b w�y `{�Lb �4�� bC~�] u � � bC~�`Ya��©b;� u �Çb u´w�y ` w ac` |�wN� ` u[w `G¨T`V_ w

 )\� Rd �ÓÑ � j = �Q| ���������������� ��������������

" 5(5 # 5 � " �(�$# � Õ&�" 5(5 # � � " �(�$# 5 Õ&�" 5(5 # � Õ " 5(5 # 5 Õ " 5 �$# 5 Õ " � 5 # � Õ " 5 �$# �" 5(5 # 5 Õ " 5(5 # � Õ " 5 �$# � Õ " � 5 # 5 Õ " 5 �$# 5Õ " � 5 # 5 Õ " 5 �$# 5Õ " � 5 # � Õ " 5 �$# �" �(�$# 5 Õ " � 5 # 5 Õ " 5 �$# 5 Õ " � 5 # � Õ " �(�$# �" �(�$# � Õ " � 5 # � Õ " 5 �$# � Õ " � 5 # 5 Õ " �(�$# 5

? �������������@�������������AØ dI¸:i4�ÿe | j

�2] � ] �¶| a ��¦ £ wNy `Ò� }�} `Ya��©b;� u �Ö]Bz��f]4�f` u � ¦

 )\� Rd �ÒÑ � jCB � ] u��������������� ��������������

�SÕ " � 5 # 5 Õ " 5 �$# ��SÕ " � 5 # � Õ " 5 �$# 5Õ " � 5 # � � " � 5 # 5 � " �(�$# 5 � " 5(5 # � � " �(�$# �Õ " � 5 # 5 � " 5(5 # 5 � " �(�$# 5 � " � 5 # � � " �(�$# �" 5(5 # 5 � " �(�$# 5" 5(5 # � � " �(�$# �Õ " 5 �$# 5 � " 5(5 # 5 � " �(�$# 5 � " 5(5 # � � " 5 �$# �Õ " 5 �$# � � " 5(5 # � � " �(�$# � � " 5(5 # 5 � " 5 �$# 5

? �������������@�������������AØdI¸:i4�ÿef�÷j

ª ` �6|ÿ¦²|;� zcb���`VaN]��;`>��bf� u ��z±�LbfaMdµ¸:iW¸fj |;u ��dµ¸:i ×[j)] u ��]��K]���� |;����¦ dL� u ��`Vaw�y `¬z |;� ` � ] u ` | a�_Vb u z w a | ] u[w z�jG£X�f]4�2] u ���

§d � 5 «­´¶µ d � � jcj = �Q| � ����� ����

" 5 �$# � � " 5(5 # � Õ " �(�$# 5 Õ " 5(5 # 5" 5 �$# 5" 5 �$# �" � 5 # � � " 5 �$# � Õ " �(�$# 5 Õ " � 5 # 5? ���@���A �

§d � 5 «­´¶µ d � � jcjDB � ] u ����� ����

" � 5 # � � " 5 �$# � � " 5 �$# 5 � " 5(5 # 5�SÕ " �(�$# 5�SÕ " �(�$# �" 5 �$# � � " 5(5 # � � " � 5 # 5 � " 5 �$# 5? ���@���A û dI¸:i4�?�fj

§d � 5 « ´¶µ d � 5 jNj = �Q| �

����� ����" 5(5 # �" 5(5 # 5Õ " �(�$# � Õ " � 5 # � � " �(�$# 5 � " 5(5 # 5Õ " � 5 # � Õ " 5 �$# � � " 5 �$# 5 � " 5(5 # 5

? ���@���A �

Page 15: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê eÇ�2�§d � 5 «­´¶µ d � 5 jNjDB � ] u

����� �����]Õ " � 5 # 5�]Õ " � 5 # �" �(�$# � � " 5(5 # � � " 5 �$# 5 � " 5(5 # 5" 5 �$# � � " 5(5 # � � " �(�$# 5 � " 5(5 # 5

? ���@���A Ø dI¸:i4�cg[j³ y `VzN`Ç`G� } aN`VzNzc]Bb u zh�f]4�f` w�y ` w ]B� y[w `Vz w�} bfzNzc]B� � ` | zNzc� �h}:w ]�b u2� �Lac`Y`FEQ��bf� u ��zb u´w�y `¬®�� |�u�w ] w ]B`Yz�] u �;b � �;`Y� i

ä«å('�åHG /ÚÙpë+) ð¡ï[}pìXðY-JI&*7}HñpòHù³ y `>`J� } aN`VzNzN]�b u �Lbfa± )\� Rd � Ñ � j{dµ`Y®�� |0w ]Bb u dI¸:i e�jcj�_ |;u �©`{�©b;� u ��`V�â� ¦w ~�bMzN] �h}�� `��Lb;a � � �B| zY£K` | _ y´�6| �:`±� } b�� w�y `±§XaNz w^w ~�b w `Ya � z�] u dµ¸:i4�ÿe |s|�u �dI¸2i��ÿe;�÷jNjHd � bf�:] u zH�C×;¸f× ûK:.|�u zNú�]¤�C×f×�� û ��` | a � �C×;×;e�jG� )\� Rd �ÒÑ � j = §

d � 5 « � 5 j·Õ §d � 5 «­�W� j·Õ §

d � 5 �¡� �£« � 5 j·Õ §d �£�9�¡� 5 «­�W� j

 )\� Rd �ÓÑ � jDB §d � 5 «­� 5 j·Õ §

d � 5 « �W� j�� §d �£���¡� ��«­� 5 j�� §

d � 5 �¡� 5 «­�W� jdI¸:i4���fj

g^`V_ | ��zc`Òb;� wNy `Y]�a�zN] �Q}:� ]B_V] wv¦h|;u �Ö~�]���`Ha |�u �f`pb;� |;}�}:� ]B_ | ��] � ] wv¦ £ w�y `H��bf� u ��z�f]4�f` u � ¦ dI¸:i4���fjâ~^`VaN` uX|�� `V� w�y ` S�9;5µ/KFN9;1 ��bf� u ��z¢dIg |�� ú;` |;u � �Ô` | a ��C×f×f�fjGi�³ y ` u�|�w ��a |;� ��bf� u ��z²�f� | a |;u[w `V` wNyX|�wÖw�y `Ñ_ | ��z |;� `G¨T`V_ w b;� w�y `| _ w � |;��w ac` |�wN� ` u[w _ |�u�u b w �©`²z �Q|;�B� `Ya w�yX|�u�w�yX|0w b;� wNy `.` u _Yb;��a | �;` � ` u[wd§d � 5 « � 5 jYÕ §

d � 5 « �W� jNj)� ¦ � bfac` wNyX|;u w�y `MzN� � b;� w ~^b � ` | zc��a | � � `>®�� |;uK�w ] w ]�`YzY£

§d � 5 �¡� �£«­� 5 jL� §

d �£���¡� 5 «­�W� j û#w�y ` ¦x|�� zNb²�f� | a |;u[w `V` w�yX|0w{wNy `6_ | ��z |��`V¨©`Y_ w b;� w�y ` w aN` |�w�� ` u[w _ |;u:u b w `G�2_Y`V`Y� w�yX|0w b;� w�y `²` u _Vbf��a | �f` � ` u[w � ¦� bfaN` w�yX|;u²w�y `>zc� � b;� w ~^b´b w�y `Va � ` | zN�:a | � � `s®�� |;u[w ] w ]�`YzV£

§d �£���¡� ��«­� 5 jM�§

d � 5 �¡� 5 «­�W� jGi¡³ y `^~�]B� wNy b;� wNy ` uX|0w ��a |;� ��bf� u ��zY£ u b w zN�:a } ac]BzN] u � �4¦ £0]Bz¡�;]��f` u� ¦�w�y `¬a |�w `Òb;� u b u _Yb �h}�� ] |;u _V`;£

§d � 5 « �W� j�� §

d � �¿«­� 5 jGi³ y `«~�]B� wNy b;� w�y `«z yX| a } ��bf� u ��z#] u dµ¸:i��Ye | �÷j�_ |;u ��`«zN����z w?|;u[w ] |;����¦{uX| a �aNbC~�`Ya w�y b;��� y i tvu g |�� ú;`.dÅ�ÿ×f×Ç�fj |;u ���Ô` | a � d°�C×f×Ç�0�÷jG£Ý] w ]�zÒz y bC~ uxwNyX|�wů`V�;` u � u ��`VaÒ_Yb u �:] w ]�b u zpb����¿�ON u b u _Yb �h}�� ] |;u _V` ¯&w�y `VzN`>��bf� u �:z �Q|ÿ¦ _Vb �4��¶|�} zN` w b |«} b;] u[wÔ|;u � w�y ��z } `Va � ] w _Vb u zN]�z w ` u[w `Vz w ] �Q|�w ]�b u b;�� )\� ±d �ÒÑ � jGi³ y ]�z+b2_V_Y��acz+~ y ` u `V�f`Vapd | j wNy ` } `YaN_V` u[w?| �f`«b;��zN����¥c`Y_ w z¤_Yb �h}���¦ ] u �R~�] wNyM| z �zN]�� u�� ` u[w.�W� ]�z w�y `sz |;� ` | z wNy bfzc`M_Yb �h}���¦ ] u �Q~�] w�yr� 5 |;u ��dL�÷j �'|�u � �P ó�\`@�@�K?½�DcP�> GIN?Î PMFMBIN@ËD¿BEFMLON9PMô�½�> dI=9P7U£L�B�U£L<PËPMLOF{P�L�F�½!QÉa¨LV½�BIÊILVN?G®B�@�@�K?½�DcP�> GIN¿@BEU£GIK?P¨»�B�H(P�G�FM@¨PM=B_P,TcL<PMLOFM½�>kN?L�@�K?UcòÍL¡H(PM@,RcH<G�½7DQk>ABENHOLIºWU?KcP,aÉL`FMLOQ R]GIN�P�=?L�B�@�@�K?½�DcP�> GIN¿@GE»�ÀÏ>6Æ[FJBENT?GI½�>TS�L�T7BI@M@Ë> dIN½7L�N9P�BINT�ÀÏ> >AÆ�N?GY@Ë>ATcL,L<U£L¡H(PM@�ºWBI@�Tc>A@ËDQ B4R�L�T{>kN�PM=?L,dIFJBED?=�ÀÏL�e d¿ekºX�>kd�K?FMLiýekõ4Æ(e

Page 16: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

eÇ�[�:¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ| ac` } `YaÅ�L`Y_ w���¦ _VbfaNac` �B|�w `V�²] u²|0w±� ` | z w b u ` w aN` |�w�� ` u[w±| a � � dµzc`Y`¬³ | � � `¬¸:i4�] u �2`V_ w ]Bb u ¸:i �fjGi

  �4w�y bf�:� y�� b;aN`�_Vb �Q}:� ]B_ |�w `V� wNyX|;u�w�y ` u�|�w ��a |;� ��bf� u ��zMb;�sdI¸:i4��� | �÷jG£w�y `6z y�| a } ��bf� u �:zsb;�¬dµ¸:i4�ÿe | �÷j | ac` u `G�f`Ya w�y ` � `VzNzM` | z ¦Ñw b | zczN`VzNz>b u _Y`6~�`yX| �f` wNy `M�LaN`Y®���` u _ ¦ � |�w?| ] u wNy `M`Y]�� y[w _Y` �B� zRb;�

§d �Ì�¡��« � jGi t w _ |;u�|;� zNb´��`

z y b�~ u dIg |�� ú;`h�C×f×��fj w�yX|0w)wNy ` uX|�w ��a |;� ��bf� u ��z | aN`Òb }:w ] �Q|;� ~ y ` u ~�`{_ |;uz | �L` ��¦â| zNzc� � ` wNyX|�w¬u bÇzN���K¥c`V_ w ]Bz =?-0SX5µFN9;FJ@µ9;S�¯ ]B] u b w�y `VaH~^bfac��zY£ w�y�|�w¬u bzN�:�K¥c`Y_ w ~^bf� � ��_Vb u zN]�z w ` u[w��4¦ _ y bKbfzc` |{w ac` |�wN� ` u[w«| a � _Vb u[w a | a ¦hw b wNy `pb u `| zczN]B� u `Y� iV b w ` w�yX|0w £�]�� wNy `MaN`Vz } b u zc` � ]�zp_Yb u�w ] u ��bf�:zY£ w�y ` u b u `>_ |;uÑ| zNzcb2_V] |�w `� 5 |�u � �£� ~�] wNy w�y `M�:] uX| a ¦ `G�f` u�w z �XWZY±|;u � � B Y dµac`Yz } `Y_ w ]4�f` �4¦ j |�u �� ` w[Y � | a ¦ _Yb u[w ] u ��bf��z ��¦ bC�f`Ya w�y `Qa |;u �;`6b;� � i�dµ¸:i4�?�fj |;u ��dµ¸:i��Wg[jH~�bf� � �w�y ` uâ} acb��K]���`s��bf� u �:zpb u.wNy `>` u[w ]BaN`{��]Bz w ac]B��� w ]Bb u b;� wNy ` w ac` |�wN� ` u[w `V¨©`Y_ w§d �]\^Y®«�´�µ d � jcjGi

ä«å('�å`_ õ�asëÔîTïv*�íú/hìXëÔð¡ï�èøë�ñ^ïv*^ñ(ï[Ùpë /QìXëÝð¡ï�ëÝò: ��_ y b;� wNy ` � ] w `Ya |0w ��ac` | zNzc� � `Vz wNyX|�w  )\� ±d �ÓÑ � j�]�z wNy ` }X| a |;� ` w `Vab;�¡] u[w `VaN`Yz w £÷��`Y_ | ��zN`¬ )\� pd �ÓÑ � j } ac`Y��]�_ w z wNy `{] �Q}�| _ w b;� |;}�}��4¦ ] u � w�y `w ac` |�wN� ` u[w � u ]��Lbfa �Q�4¦ dLbfa�a |;u ��b �h��¦ j^bC�f`Ya w�y ` } b } � �B|�w ]�b u i¤�±bC~�`V�;`YaY£©]�� |} b � ]�_ ¦Ò�Q| ú;`Va�]�z u b w ] u[w `VaN`Vz w `V�>] u ] u[w acb2���:_Y] u � u `G~ w aN` |�w�� ` u[w¡} b � ]B_V]B`YzÝ��� wa |0w�y `Vas] u ��`Y_V]B��] u �²~ y ` w�y `Ya w b �6| ] u[w?| ] u bfa w `Ya � ] u�|�w ` |;u `J�2]Bz w ] u � } aNb ��fa |;� � u �:`Ya¤] w z�_V��aNac` u[w ] u _V` u[w ]4�f`)z ¦ z w ` � £ wNy ` uhw�y ` }X| a |;� ` w `Ya¤b;�©] u[w `VaN`Yz wz y bf� � � � ` | zc��aN` w�y `)] �Q}�| _ w b;� w�y ` w aN` |0w�� ` u[wH-0S´5L7XE)5µF�EN9;5°EN3 £ uX|;� ` �4¦ £ w�y `� ` |;u aN`Vz } b u zc`{b;� w�y ` w aN` |�w `V�²zc���K¥c`Y_ w z±_Yb �h}X| ac`Y� w b w�y ` � ` |�u ac`Yz } b u zc`b;� w�y `YzN`«z |;� `�zN����¥c`Y_ w z yX| � w�y ` ¦>u b w ��`Y` uMw aN` |0w `Y�²dË�±`Y_?ú �Q|;u �C×f×[�;jGiÔ³ y `|;}�} aNb } ac] |�w `p�Lbfa � � �¶| �Lb;a w�y ]�z }�| a |;� ` w `Ya«]�z

 )\� cb�d �ÒÑ � j ª §d � ÿ � ª � 5 « � 5 j·Õ §

d � ÿ � ª � 5 « � 5 jª ¯ ± Ö § d � 5 « � 5 �4© j·Õ §d � 5 « � �W�4© j(× § d ©�« � 5 j � dI¸:i4�C¸[j

~ y ]�_ y ]Bz�zc] � ] �¶| a w b�dI¸2i e�j>`J�:_V` }:w �Lbfa�aN` }��¶| _V] u � wNy `²`J� } `V_ w?|0w ]Bb u bC�f`Va ©~�] wNy´w�y `Ò_Yb u ��] w ]Bb u�|;� `J� } `Y_ w?|�w ]Bb u �f]4�f` uü� ª � 5 i³ y ` |;uX|���¦ zN]�z�b;�Ý )\� b d �ÒÑ � j�ac`V�;` |;� z w�yX|0w £:� u ��`Ya^_Yb u �:] w ]�b u z�b;� ST-@8SX5µFG/[AG@�-0S dµ]�i `;i4£

§d � 5 « �W� j�ª¥�:£ | z)] u´� b;z w _ � ] u ]�_ |;��w ac] |;� z?jG£K )\� b d �ÒÑ � j

_ |�u �©`²]B��` u�w ]�§X`V� } aN`V_Y]�zN` �4¦ dIg � b2b � �ÿ×f¸;e û   u �fac]Bz w´|;u � tv� �©` u z �C×f×2��jGi

Page 17: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂkj:Â�ÌpÍhÎÔÉRÏpðÉsÐ ¹¡»hΡÊ2»l�¾¤Æ+Æ+¾)¹)½¡Ê eÇ�;�³ y ` uX|0w ��a |;� ��bf� u ��zH�fbC�f`Va u ] u �´ )\� b d �ÒÑ � j�] u.wNy `M�f` u `Ya |;� _ | zN`M_ |;u��`¬b;� w?| ] u `Y�Ö� ¦ zc] � ] �¶| a � ` |;u zV£:~ y ]�_ yǦ ]B` � �

 )\� cb�d �ÒÑ � j =§d � 5 «­� 5 j·Õ §

d � 5 «­�W� j§d � 5 j$d §

d � 5 j Õ§d �£�W�¡� 5 «­�W� j§d � 5 j �

 )\� b�d �ÒÑ � jDB §d � 5 «­� 5 j·Õ §

d � 5 « �W� j§d � 5 j,d §

d � 5 j � §d � 5 �¡� 5 « �W� j§d � 5 j Ø dI¸:i4�C×[j

³ y `´z yX| a } ��bf� u ��z | aN` } aN`VzN` u�w `Y��] u g |;� ú�` dÅ�C×;×Ç�2£ } i��f�C�[jJiÑ\ � ` | a �4¦ £+] uzN] w � |0w ]Bb u z^~ y `VaN` w ac` |�wN� ` u�w��Q|ÿ¦ ��`Hbf� w�| ] u `Y�Öb u��4¦ � ¦�wNy bfzN`H` u _Yb;��a | �;`Y�dµ� ¦Ö| zNzN]�� u�� ` u[w jJ£�~^` yX| �;`

§d � 5 « �W� jYªx� |;u �

 )\� bCd �ÓÑ � j2ª§d � 5 «­� 5 j·Õ §

d � 5 « �W� j§d � 5 «­� 5 j Ø dI¸:iW�£�[j

h u:� ]Bú�`� )\� pd �ÒÑ � j �  )\� b d �ÒÑ � j¬]Bz u b wh|�u ] u[w ac] u zc]B_ } acb } `Ya wv¦ b;�w�y ` w aN` |�w�� ` u[w £�zN] u _Y`p] w � | aN]�`Yz^~�] w�y�wNy `H` u _Vbf��a | �f] u �M] u z w aN� � ` u�w iY�±` u _Y`�£] w z¤zN]B� u ]�§X_ |;u _V` � ]B`Vz�] u z w �:��]B`Vz�~ y `VaN`±] w ]Bz¤��`VzN]�aN`Y� w b{`V� |;� � |�w ` w�y `)`Gýh_ | _ ¦b;� |;u `G�2]Bz w ] u � } aNbf�fa |;� b u ] w z«_V��aNac` u[wR}�| a w ]B_Y] }X|;u[w zVi

ä«å('�å�e õÇö±ð¤è�é.-Åëcf�/�Ùpëõga>ëÝîTï�*�íih Ù�*{-°ëÔù�ïkj)ìXð¤è�ôcñpë³ÔbQ��` � b u z w a |�w `¬� ¦ `J� |��Q}�� ` y bC~ w�y `¬��bf� u ��z)�Lb;a« )\� pd �ÒÑ � j�_ |;u ��`��zc`Y� w b } aNbC�K]B�:` � ` |�u ] u ���L� � ] u �Lbfa �Q|�w ]Bb u.| �©b;� w _ | �:z |;� `G¨T`V_ w zV£�_Vb u zc]B��`Vaw�y `ç�Ý] } ]B� � `VzN` | aN_ y \ � ] u ]�_Yz�\^bfaNb u�| a ¦ �¤ac] �Q| a ¦ �#aN`G�f` u�w ]Bb u ³ÔaN] |�� � |�w�|d��#acbf�fa |�� �C×f¸�e�jGi�  } bfa w ]�b u dµ_VbC�f`Yac] u ���f�[�.zc���K¥c`V_ w z�j�b�� wNy ]Bz>� |�w?| ~ | z|;uX|���¦ �Y`V�Q] u #�Lacb uQ|;u ���÷` � � �Q|;u d°�C×f×:�Cj |;u ��]Bz wNy `«�Lb2_V��z¤b;� wNy ]Bz#`J� |;�Q}:� `;i�2����¥c`Y_ w zR~�`YaN`sa |;u �:b � ]��Y`V�²] u�w b w ~�b w ac` |�wN� ` u[w �facbf� } z±b;�+acbf��� y:��¦ `Y®�� |��zN]��Y` û ] u b u `.�faNbf� } £ |��B� zN���K¥c`V_ w zÖ~�`Yac` } aN`VzN_VaN]B��`Y��_ y b � `Yz wv¦ a |;� ] u `�d � 5 jG£~ y ] � `Rzc���K¥c`V_ w z)] u�wNy `pb wNy `Ya«�;aNbf� } ~�`Yac` } aN`VzN_Yac]B��`Y� |>}��B| _Y`V�©bÖd �W� jGi¤$R�f`VazN`G�f`Va |;�R¦ ` | acz´b;� w ac` |�wN� ` u[w £�` | _ y zc���K¥c`V_ w 8 zÖ_ y b � `Yz w `Yacb �R� `V�f` � ~ | z � ` |0�zN�:aN`Y� �6|�u�¦xw ] � `YzV£ |�u � wNy ` | �;`Ya | �f`Çb;� wNy `Yzc` � ` | zc��aN` � ` u[w z�~ | zh��zc`Y�| z wNy ` } bfz wNw ac` |�wN� ` u[w _ y b � `Yz w `Yacb �^� `V�f` � dL_Yb u[w ] u ��bf��zs� | ac] | � � `JlCm�jGi.³ y `_Yb �Q}�� ] |;u _Y`)b;�Ý` | _ y zc���K¥c`V_ w ~ | z���` w `Ya � ] u `Y�6� ¦Qw a | _?ú�] u � wNy `R®�� |�u�w ] wv¦ b;�} ac`YzN_VaN]��©`V�²��b;z | �f`Ò_Vb u zN� � `V��d | _Vb u[w ] u ��b;��z�®K� |;u[w ] wv¦ jJi

tvu bfac��`Ya w b |�}�}���¦6w�y `Ò��bf� u ��z)b;��dI¸:i4���fj w bQ� |�w?| �LaNb ��w�y ]�z«z w ��� ¦ £ w�y `_Yb u�w ] u ��bf�:z{� |0w?| ]BzH§Xacz wÒw a |;u zc�Lb;a � `Y� £Ý��zc] u � w�y ac`Yz y b � �:zY£ w bÖ��] u�| a ¦ � | ac] �| � � `Yzpac` } ac`Yzc` u[w ] u � w ac` |�wN� ` u�wH| zNzc]B� u�� ` u[w d � jJ£ aN`V_Y`Y]4�f`V� w aN` |�w�� ` u[w d � jG£

Page 18: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

eÇ�;eX¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ|;u � w ac` |�wN� ` u[w ac`Yz } b u zN`xd � jJiÁ³ y ` wNy aN`Vz y b � ���Lb;aM��bfz | �f`²_Vb u zc� �Q}2w ]Bb u~ | z�zc` � `Y_ w `Y� | z�acbf��� y��4¦>w�y ` � ]B� } bf] u�w �©` w ~^`V` u�� ] u ] � � ��|;u � �6| �2] � � �_Yb u zN� �h}:w ]Bb u û[wNy ` wNy aN`Vz y b � �Q�Lb;a�_ y b � `Vz w `VaNb �÷� `V�f` � aN`Y�:��_ w ]Bb u ~ | z�zc` w^|�w �;¸� u ] w zViH ±� w `Va wNy ]Bz 1°w�y ac`Yz y b � ��] u � 3Q} aNbK_Y`V����aN`�£ w�y `s� |�w?| z |;�h}�� `VzR�f]4�f`>ac]Bzc`w b wNy `H�Lb �B� b�~�] u �>`V]B� y[w�} acbf� | ��] � ] w ]B`VzY�on

§d �¿�9�¡� ��« �W� j ª ��Ø ×:�C× � §

d �£���¡� ��«­� 5 j ª �%ØW�:�c� �§d �¿�9�¡� 5 « �W� j ª ��Øk����� � §

d �£���¡� 5 «­� 5 j ª �%Ø��C�;× �§d � 5 �¡� ��« �W� j ª ��Øk�f¸:� � §

d � 5 �¡� ��«­� 5 j ª �%Ø �[��� �§d � 5 �¡� 5 « �W� j ª ��Øk����� � §

d � 5 �¡� 5 «­� 5 j ª �%Ø e�����س y `Yzc`s� |0w?| aN` } aN`VzN` u[wR| _Vb �h}�� ] |;u _Y`Ra |�w `Hb;�§

d � 5 «­� 5 jYªø��Ø��ÿ�f×3� ��Ø e��;��ª¥�%Ø g:� �|M� ` |;u ��]�¨©`Yac` u _V`�dL��zN] u �

§d � 5 jYªx��Ø �£�[j^b��§

d � 5 « � 5 j·Õ " d � 5 « � � jYª ��Ø e��;���Ø e��;�3� ��Ø��ÿ�f× Õ ��Øk�[�;�3� �%Ø �f¸2�� � ��Ø �:�?�0� ��Ø �f�;� ªx��Økg�g[� �

|;u � |;u ` u _Ybf��a | �f` � ` u�w `G¨T`V_ w dµ] u[w ` u[w�w b w ac` |�w j^b;�§d � 5 « � 5 j·Õ §

d � 5 «­�W� j�ª¥��Øk�[�;�p� ��Ø e��;��Õ&��Øk�f¸:�Vªx�%Ø eÇg��%Ø R_V_Ybfac��] u � w b dµ¸:i4���fjG£2 )\� pd �ÓÑ � j^_ |;u ��`Ò�©b;� u ��`V�²� ¦

 )\� Rd � Ñ � j = ��Ø eÇgÇ�Õ&��Øk�[�;��Õ&��Øk������ªø��ØW�;×[� � )\� Rd � Ñ � jqB ��Ø eÇgÇ�0� �%ØW�:�c�0� ��Øk������ª¥��ØW�;¸��%Ø

³ y `Yzc` | aN`QaN` �6| acú | � ��¦ ] u �Lbfa �6|0w ]��;`��©bf� u ��zY� |���w�y bf��� y �f¸:i ¸ON�b;� w�y `zN�:�K¥c`Y_ w zR�:`V�K] |�w `Y�Ö�LaNb �êw�y `V]Ba w ac` |�wN� ` u[w±} aNb w b2_Vb � £ w�y `Ò`G� } `Yac] � ` u[w `Va±_ |;u_ |0w `Y�;bfaN]�_ |;����¦ z w?|�w ` w�y�|�w £�~ y ` uÖ|�}�}�� ]�`Y�h� u ]4�Lbfa �h��¦>w b wNy ` } b } � �B|�w ]�b u £ w�y `w ac` |�wN� ` u[w ]�zÒ�f� | a |;u[w `Y`V� w bÇ] u _Yac` | zc`Q� ¦Ñ|�w¬� ` | z w �;×:ik�rN w�y ` } acbf� | �:] � ] wv¦b;��aN`V����_Y] u � w�y ` � `G�f` � b;�¡_ y b � `Vz w `VaNb � � ¦ �;¸ } bf] u[w z«bfa � b;aN`;is<t®L�½]BIÊIL·P�=?LiQABEFMdILOÎë@MBE½�D?Q L`B�@�@�K?½�DcP�> GIN�BEN¿TSPJBEÊIL·PM=?LY@�BI½�D?QkL`»ÏFML�÷9K?LON¿H<> L�@,BI@!FMLOD?ÎFML�@�LON9P�> N?d{¾]À�Á�ÂËÃ�Ä ÅWÆ(e�;�G7BIH�H<GIKN�P¨»ÏGIF!@�BI½�D?QkL`^_BEFM> BIU?>kQ >kPÍRIºWBEQ Q£U£GIKNT?@!@Ë=GIK?QAT]U£L·@�K?D?D?Q L<ν�LON9P�L¡T�a,>kP�=�H<GIN?¼¿TcLON¿H<L¤> N9P�L�F�^_BEQA@·BENT®@Ë> dIN?>k¼¿H�BENHOLYQkL�^IL�Q @�ºÇBI@`>kN�P�FJBITc>kP�> GINBIQ�BINBEQ Rc@ËL¡@GE»YHOGIN9P�FMGIQ QkL¡TæL<ðcD¿L�F�> ½�LON9PM@�eçìWL�H<P�> GINüýe u?ekõ.BI@M@�L�@M@ËL¡@]@�BI½7DQkL�^_BIF�>ABEU?> Q >êPÍR�K@Ë> N?dJv·> U?U@@MBE½�D?Q >kN?d¿e

Page 19: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂxwK ¹«Í�ÎÔÉ�½�¾¤¿±ÆK»¬¹)½RÎÔ»l�ÊÇ»pÉRÏ lݾ)Ð�»l�¿±¾�Ê:¼)Í«ÉpÊKðÌ^Ã<l�ÃY½py eÇ���³ y `¬] �h}X| _ w b;� w aN` |�w�� ` u[w�1 b uÖw�y ` w aN` |�w `V� 3 ]Bz«`Y®�� |��B��¦ aN`G�f` |�� ] u ��iYh±z �

] u �.`Y®�� |0w ]Bb u dI¸:iW�£�[jJ£¡ )\� cb�d �ÒÑ � j¬_ |;u ��`6`V� |;� � |�w `Y� } ac`Y_V]BzN` ��¦ dµzc] u _V`§d � 5 « �W� j�ª¥�[jG�

 )\� bÿd �ÓÑ � j¤ª ��Ø e�gÇ���Ø g2�c� ª¥��ØW�£g[�%Øtvu b w�y `VaM~^bfac��zY£ w�y b;zN`ÇzN���K¥c`V_ w zQ~ y bÑz w�|C¦ `Y�Á] u�w�y ` } acbf�fa |;� | ac` � ��_ y��` wNw `Ya�b�¨ w�y�|;u�w�y ` ¦ ~�bf� � � yX| �f`H��`Y` u ]4� u b w�w aN` |�w `V� � w�y ` w aN` |�w�� ` u[w _ |;u��`^_Yac`Y��] w `V�>~�] w�y aN`V����_Y] u �p_ y b � `Yz w `VaNb �2� `V�;` � z¡� ¦>|0w¡� ` | z w �;¸)� u ] w z¡] u �£g:iW�kNb;� w�y `VzN`¬zN����¥c`Y_ w zYi

ØÖÙ{z p ßhá Ü�Þ nHm#ofqsr Þ á qtsWq q Ü à |�n o qts~}(n.q.�xß Ü q ã��� ã s ãVÞ�� +� |;� � |0w ]Bb u b���_Yb;� u[w `YaÅ� | _ w � |;�K} aNb;� | ��] � ] w ]B`Vz _Ybf� � �s��`�` u�� ]B� y[w ` u ] u �)] u zcb � `� `V� |;� _ | zN`Vz{] u ~ y ]B_ y�|Ç}��B| ] u[w ]�¨�_ �¶| ] � z w�yX|�w¬| ��`G�L` u � |�u�w 8 z | _ w ]Bb u zÒ~�`Yac`aN`Vz } b u zc]B� � `��Lb;a wNy ` }��¶| ] u[w ]�¨�8 z � ]BzÅ�Lbfa w � u `�i tv�h} aNb } `Va¬ac� � ] u �fzH_ |;u ` | zN] ��¦��`{]�zNzN�:`Y�²~�] w�y b;� wp|�u.| �:`Y®�� |�w ` w ac` |�wN� ` u[w b;�¡_Ybf� u�w `YaÅ� | _ w � |;� zVi)\^b u zc]B��`Vaw�y `Ö�Lb �B� bC~�] u � y[¦K} b wNy ` w ]B_ |;��|;u ��§X_ w ] w ]Bbf�:zM_ | zc`Çz w ��� ¦ £�z } `V_Y] |��B��¦ _Ya | � w `V�] u g |�� ú;` |;u �b��` | a � d°�C×f×;e | j w b | _Y_V` u[w � |�w ` w�y `¬��]�z }X| ac] wv¦ �©` w ~^`V` u _ | ��z |��`V¨©`Y_ w z |;u �Ç_ | �:z |;�Ý|�wcw ac]B��� w ]Bb u i

³ y ` �Q| aNú�` w `YaÔb;� ��` }:w  ±]B�Öd |;u[w?| _V]B� � `V��]B_ |�w ]�b u jTa |�u ��b �h��¦H�Q| ] � `Y�Òbf� w} acb2���:_ w z |;�h}�� `Yz w bÑ�c�ON(b�� wNy ` y bf��zN` y b � ��z{] uâw�y `�_Y] wv¦ b;��� w ac`YzNzV£#\ |;� ] ��Lbfa u ] | i tvu�| �Lb �B� bC~ � � } z w ��� ¦ £:aN`VzN` | aN_ y `VaNz��:` w `Va � ] u `V�Q�Lbfa¤` | _ y ] u ��]��K]B�:� |;�~ y ` w�y `Va wNy ` ¦ aN`V_Y`Y]4�f`V� wNy `ç��` }:w  ±]B��z |;�h}�� `;£«~ y ` wNy `Ya wNy ` ¦ _Yb u zc� � `Y���` }:w  ±]B��£ |;u ��~ y ` wNy `Ya wNy ` ¦ ��`G�f` � b } `Y� } ` }:w ]�_.� � _Y`VaNz6] u�w�y `.�Lb �B� bC~�] u �� b u[w�y i

³ y `«_ | ��z |;� z w ac��_ w ��aN`«�Lbfa w�y ]�z+zN_V` uX| aN]BbH]�z+]B�:` u[w ]�_ |;�Kw b wNy ` }X| a w ] |�� _Vb ���}�� ] |;u _V` � bK��` � �f]��;` u � ¦ �¡]B�;��aN`s¸:i��;£�~ y `VaN` � 5 | zczN`Va w z w�yX|�w ��` }:w  ±]B� ~ | zaN`V_Y`V]��f`V�&�LaNb � wNy ` �Q| aNú�` w `VaY£ � 5 | zczN`Va w z wNyX|�w ��` }:w  ±]B�&~ | z�_Yb u zN� � `Y�T£|;u � � 5 | zNzN`Va w z w�yX|0ws} ` }:w ]B_h� � _V`Ya |�w ]�b u bK_Y_V��aNac`Y� idz y `Q� |�w?| z y bC~�`Y� w�y `�Lb ��� bC~�] u �s��]Bz w aN]���� w ]Bb u �§

d �£�9��� ��«­�W� jYªx�%ØW�[� � §d �£���¡� ��«­� 5 j¤ªø��Øk�[� �§

d �£�9��� 5 «­�W� jYªx�%ØW�[� � §d �£���¡� 5 «­� 5 j¤ªø��Ø4��� �§

d � 5 ��� ��«­�W� jYªx�%Ø �;e � §d � 5 �¡� ��«­� 5 j¤ªø��Økg[� �§

d � 5 ��� 5 «­�W� jYªx�%ØW�[� � §d � 5 �¡� 5 «­� 5 j¤ªø��Ø4�ÿe[Ø

Page 20: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

eÇ��g�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔɳ y `VzN`Ç� |�w?| ] u ��]B_ |�w ` | y ]B� y _VbfaNac` �¶|0w ]Bb u �©` w ~^`V` u�w�y bfzc`´~ y b _Yb u zc� � `Y���` }:w  ±]B� |;u � w�y b;zN`H~ y bh��`V�;` � b } `V� } ` }:w ]B_Ò� � _Y`YaczY�§

d � 5 « � 5 j¤ªø��Ø �£� � §d � 5 « � � j¤ªø��ØW�£g�Ø

tvu | ���:] w ]�b u £ w�y `s] u[w ` u�wÅ� w b � w aN` |0wp|;u�|;��¦ zN]�zRz y bC~^`V� wNyX|�wpwNy bfzc`>] u �:]��K]B��� |;� z~ y bQaN`Y_V`Y]4�f`Y� w�y `®��` }:w  ±]B�.z |;�h}�� `Vz y�| � | e¶�kN��fac` |�w `Ya�_ yX|;u _Y`>b;�¡��`V�;` � �b } ] u � } ` }:w ]�_H� � _V`YaczY�§

d � 5 «­� 5 j¤ªø��Ø ¸:� � §d � 5 « �W� j�ª¥��Ø ��g�Ø

³ y ` }��B| ] u[w ]�¨ d : aYis� � ] w�y jG£ yX| �K] u � y ` | ac� b;� wNy `�z w ��� ¦ £ � ] w ]B� |�w `Y�| � | ] u z w ��b wNyhwNy ` �Q| acú;` w ] u �Ò§Xa �ø|;u � wNy `V��` }2w  R]�� } acb2�:��_Y`VaYi#³ y ` }��¶| ] u2�w ]4¨]8 z |�wcw bfa u ` ¦>| aN�f�:`Y� | � | ] u z w¡w�y ` } aNbK����_V`YaY£[_ �¶| ] � ] u � w�yX|�w�w�y `«_Vb u zc� �Q}K�w ]�b u b;���` }:w  ±]B� w ac]B�f�f`VaN`V� y ]Bz>_ � ]B` u[w 8 zM� � _V`Ya |;u �·ac`YzN� ��w ] u � � `Y��]�_ |;� `J� �} ` u zc`YzYiS��]�ú;`V~�]�zN`�£ wNy ` }��¶| ] u[w ]�¨�8 z |0wNw b;a u ` ¦Ç| aN�f�:`Y� | � | ] u z w�w�y ` �Q| aNú�` w `YaY£_ �B| ] � ] u � w�yX|�w)y ]�z±_ � ]�` u[w ~^bf� � � u b wRy�| �;`s��`G�f` � b } `Y� |�u � � _Y`Ya�]4� w�y ` �Q| a �ú;` w `Va yX| � u b w ��]Bz w aN]���� w `Y� w�y ` } acb2�:��_ w z |��Q}�� `YzVi

³ y `)��`V�L` u zN` |�wNw bfa u ` ¦ £�ac` } ac`YzN` u[w ] u �s��b w�yMw�y ` �Q|;u �:� | _ w ��ac`Ya |;u � �Q| a �ú;` w `Vahb����Ô` }:w  R]B�T£�ac`Y��� wcw `V� wNy ]Bz | aN�f� � ` u[w £�z w?|�w ] u � wNyX|�wMwNy ` y ]B� y _Ybfa �aN` �¶|�w ]Bb u ��` w ~�`Y` u ��` }:w  ±]B�â_Vb u zN� �Q}:w ]Bb u.|;u �â� � _Y`VaNz±~ | z |�wcw ac]B��� w�| � � ` w b| _Yb �h� b u � | _ w bfaV£ uX|;� ` ��¦ £ } ac` � � � _Y`Vap��]�zN_Yb � �Lbfa w i tvu ��]4�2]���� |;� zR~�] wNy � | z �w acbf] u[w `Yz w ] uX|�� ��]Bzc_Yb � �Lbfa w ~�bf� � �{�©` � ��_ y>� bfac` � ]Bú�` ��¦ ��b wNy{w b±��zN`7��` }:w  ±]B�|;u � w bÇ��`G�f` � b } z w b �6| _ y � � _Y`YaczYih³�bÇ��b � z w `Ya y ]BzÒ_ � ]B` u[w z98 _ �¶| ] � zV£ w�y `���` ��L` u zN` |�wNw bfa u ` ¦ ] u[w aNbK����_V`Y��`G� } `Ya w.|�uX|;��¦ zc]Bz�b;� wNy ` � |�w?| z y bC~�] u � wNyX|�w £b uÇ| �f`Va | �f`�£X_Yb u zN� �h}:w ]�b u b;�`��` }:w  ±]B� | _ w � |;����¦ ��`Y_VaN` | zN`Vz |;u ] u ��]��K]���� |;� 8 z_ yX|;u _Y`Yz±b;����`V�;` � b } ] u ��� � _Y`VaNz�� ¦Ö|�w)� ` | z w �?�rN´i

tvu ��`V`Y� £ w�y ` |;}:}�� ]�_ |�w ]Bb u b�� dµ¸:i4�ÿe | �÷j�aN`Yzc� ��w zÝ] upw�y `+�Lb �B� b�~�] u �^��bf� u ��zÝb uw�y ` | �f`Ya | �f`p_ | ��z |;� `G¨T`V_ w b;�¬�Ô` }:w  R]���_Yb u zN� �h}:w ]�b u b u6} ` }:w ]�_R� � _Y`Ya |�w ]�b u �

Õ��ØW�;�1B  )\� Rd �ÒÑ � jCB+ÕV��Ø4�?� ûw�y ]Bz } acb��;`Yz w�yX|�w �Ô` }:w  R]��²]�z«�©` u `V§X_V] |;� w b w�y ` } b } � �B|�w ]�b uÖ| z | ~ y b � `�i

³ y ` }��¶| ] u[w ]4¨]8 z |�wNw bfa u ` ¦ £ w�y bf��� y £�z w aN`VzNzc`Y� wNy `¤��]Bz w ] u _ w ]Bb u ��` w ~�`Y` usw�y `| �f`Va | �f` w aN` |�w�� ` u[w `V¨©`Y_ w z^�Lbfa w�y `±` u[w ]�aN` } b } � �B|�w ]�b uQ|;u �6�Lbfa w�y `pzN��� } b }K�� �B|�w ]�b u _Yb u zN]�z w ] u �Mb;� w�y bfzN`Ò] u ��]��K]B�:� |;� z¤~ y b�£ � ]Bú�` y ]Bz^_ � ]�` u[w £2aN`V_Y`Y]4�f`V� w�y `��` }:w  ±]B�âz |;�Q}:� `;£©_Yb u zN� � `Y� ] w £ |�u � w�y ` u ��`V�;` � b } `V�â� � _V`YaNzViÒ  uX|;�4¦ zN]Bz±b;�w�y ` } b } � �¶|0w ]Bb u � |�w�| ] u �:]B_ |0w `Y� w�yX|�w £ yX| �ü��` }2w  R]�� u b w ��`Y` u ��]Bz w ac]B��� w `Y� £

Page 21: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂxwK ¹«Í�ÎÔÉ�½�¾¤¿±ÆK»¬¹)½RÎÔ»l�ÊÇ»pÉRÏ lݾ)Ð�»l�¿±¾�Ê:¼)Í«ÉpÊKðÌ^Ã<l�ÃY½py eÇ�f�: aVi�� � ] w�y ~�bf� � � yX| �f` y�| � |�w�� bfz w^| �kN�_ y�|;u _Y`±b;����`G�f` � b } ] u �s� � _Y`VaNz ¯aN`V� | ac� � `VzNz�b;� |;u[¦ _Yb u �Lbf� u ��] u �M� | _ w bfaNz«zc��_ y | z } aN` � � � _Y`Ya }X| ] u i���]�ú;`V~�]�zN`�£]�� : aYi¬� � ] wNy.yX| � u b w _Vb u zN� � `V�¢�Ô` }:w  R]�� £ y `s~�bf� � � yX| �f` y�| � |�wH� bfz w| �kNê_ yX|�u _Y`¬b;����`V�;` � b } ] u ��� � _Y`VaNzYi

³ y `H� |;�Q| �f] u �sz w?|�w ]Bz w ]B_Vz | � | ] u z w^w�y ` �6| acú;` w `Ya | aN`pb;� w?| ] u `Y�´� ¦ `V� |;�4�� |�w ] u � wNy `M��bf� u ��zHb u wNy `M_Yb;� u[w `YaÅ� | _ w � |;�#} acbf� | �:] � ] wv¦Öw�yX|�wRw�y ` }��B| ] u[w ]�¨~�bf� � � yX| �f`)��`G�f` � b } `Y� |Ò} ` }:w ]B_�� � _V`Ya#]4� y ` yX| � u b w ac`Y_Y`V]��;`Y� wNy `���` }:w  ±]B�z |��Q}�� `;£ �;]��f` uâw�yX|�wÒy `h] u � | _ w ac`Y_V`Y]��;`Y� w�y `Mz |;�h}�� `¦�Ô` }:w  R]���£�_Yb u zc� � `Y�w�y `b��` }:w  ±]B� £ |�u ����`G�f` � b } `Y� } ` }:w ]B_Ç� � _V`YaczYi�³ y ]Bz } aNb;� | ��] � ] wv¦��6|ÿ¦ ��`~�aN] wNw ` u ] u´w `Va � z�b;� wNy ` }X| a |;� ` w `VaNz ) 5 â � ) â 5 £ |;u � ) â(â | z§

d � ���{ª � 5 « � 5 �¡� 5 �_� 5 j ª§dËå �(ªR��jVd ) 5 â6� ) â 5 � ) â(âGj§

d � 5 �¡� 5 �_� 5 j �zN] u _Y`sb u���¦Çw�y `>_Yb � ��] uX|0w ]Bb u z � å9ÿtªê� � å �®ª�� � �t� å9ÿtª�� � å �®ªê� � £ |�u �� å9ÿ�ª�� � å ��ª � � z |�w ]�zc� ¦Öw�y `�¥cbf] u[w `V�f` u[w¦�c� ª � 5 �.� ª � 5 �.� ����ª � 5_� i³ y `VaN`V�Lb;aN`;£ §

d � ���¤ª � 5 « � 5 �¡� 5 �_� 5 j ª ) 5 â.� ) â 5 � ) â(â§d � 5 �¡� 5 «­� 5 j Ø

³ y ]�z+`G� } ac`YzNzc]Bb u ]�z � ] u ` | a�] usw�y ` ) }X| a |;� ` w `VaNz |�u � �Q|C¦ ��`��©b;� u ��`V�M��zN] u �� ] u ` | a } aNb;�fa |;�h� ] u � w b��f]��;`§d � ���¤ª � 5 « � 5 �¡� 5 �4� 5 j = �" 5(5 # 5 �Q| �

����� �����" 5(5 # 5 Õ " �(�$# �" 5(5 # � Õ " �(�$# 5 Õ " 5 �$# 5" 5 �$# � Õ " � 5 # 5 Õ " 5 �$# 5

? ���@���A �§d � � � ª � 5 «­� 5 �¡� 5 �¡� 5 jCB �" 5(5 # 5 � ] u

��� �� " 5(5 # 5" 5 �$# � � " 5(5 # ��SÕ " �(�$# � Õ " 5 �$# 5? �@�A Ø

�2] � ] �B| a �4¦ £ wNy `p� |��6| �;] u �¬`V�K]B�:` u _V` | � | ] u z w ��` }2w  R]��!8 z } aNbK����_V`Ya�]�z�bf� �w?| ] u `V�Ç� ¦ `G� |�� � |�w ] u � w�y `Ò��bf� u ��z)b u´w�y `¬_Vbf� u[w `Vac� | _ w � |��Ô} aNbf� | ��] � ] wv¦§

d � ÿ��¤ª � 5 « � 5 �¡� 5 �_� 5 j ª ) 5 â.� ) â(â" 5(5 # 5 Øt �s~�` � ] u ] � ]B�V` |;u � �Q| �2] � ]B�V` w�y ` u � � `Ya |�w bfa�dLzN���K¥c`V_ wÇw b dI¸:i4�C�[jJ£±~�`

Page 22: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

eÇ�f¸�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉbf� w�| ] u

§d � ÿ��¤ª � 5 « � 5 �¡� 5 �_� 5 j = �" 5(5 # 5 �Q| �

��� �� �" 5(5 # 5 Õ " �(�$# � Õ " 5(5 # �" 5 �$# � Õ " � 5 # 5 Õ " 5 �$# 5? �@�A �

§d � ÿ��{ª � 5 « � 5 �¡� 5 �4� 5 jDB �" 5(5 # 5 � ] u

��� �� " 5(5 # 5" 5 �$# � � " 5(5 # ���Õ " �(�$# � Õ " 5 �$# 5? �@�A Ø

�2����z w ] w � w ] u � w�y `�bf��zN`Vac�;`Y�M��]�z w ac]B��� w ]Bb u§d �Ì�¡��«­� j�] u[w b wNy `Yzc`��Lbfa � � �B| zY£

w�y `Ò�Lb �B� bC~�] u �>��bf� u ��z«~�`Yac`{bf� w?| ] u `Y� ���Ø ×f��B §

d � ���¤ª � 5 «­� 5 ��� 5 �4� 5 j0B �£Ø ��� ��%ØW×f��B §d � ÿ � ª � 5 «­� 5 �¡� 5 �¡� 5 jCB ��Øk����Ø

³ y ��zY£ |�wp� ` | z w ×f�kN b;� wNy ` } `Vb }�� `s] u wNy ` }��B| ] u[w ]4¨]8 z±_ |�w `V�fbfa ¦ ~^b;� � � u b wyX| �f`���`G�f` � b } `Y� � � _Y`Yacz yX| � w�y ` ¦ u b w ��`Y` u ` u _Ybf��a | �f`V� w b w?| ú�`���` }:w  ±]B�d �W� j¬bfaY£+zc] � ] �B| a �4¦ £ yX| � w�y ` ¦�u b wsw�| ú;` u �Ô` }:w  R]�� d � � jGiâ³ y ]�z � ` u ��zs�;`Ya ¦z w aNb u �Çzc� }�} bfa w �Lbfa w�y ` }��B| ] u[w ]4¨]8 zR_ �B| ] � wNyX|�wÒy `M~ | z | �:�f`VaNzc` ��¦â| ¨T`V_ w `Y�� ¦�w�y ` �Q| acú;` w `Ya |;u � } acb2���:_Y`Ya�8 z | _ w ]Bb u z |;u � } aNbK����_ w i

tvu \ yX|;}:w `YaÒ×Q~�`>~�] ��� _Yb u[w ] u ��` w�y ` |;uX|;�4¦ zc]Bz±b;�¤_ | �:z |;�¡|�wNw aN]B�:� w ]�b u ] uz } `Y_V]�§X_Ò`G�f` u[w zY£ |;u �Ö~^`Ò~�] ��� `Yz w?| � � ]Bz y _Yb u �:] w ]�b u z«� u ��`Ya«~ y ]�_ yÇwNy ` } aNbf� �| ��] � ] wv¦ b;�#_VbfaNac`Y_ wp|�wcw ac]B��� w ]Bb u _ |�u ��`{]���` u[w ]4§X`Y�Ç�LaNb � ��b wNy `G� } `Yac] � ` u[w?|;�|;u � u b u `G� } `VaN] � ` u�w�|;� � |0w?| i

ØÖÙ<� � � n.q Þ ofßhm ÛÝÜ q Þ m¤á�� n Ü�Þ q RzR��`V§ u `Y� ] u �K`Y_ w ]Bb u ¸:ik�K£Tbf��a � bK��` � b;�¤] �Q} `YaÅ�L`Y_ w `G� } `YaN] � ` u[w aN`Vz w zHb uw ~�b | zNzc� �Q}2w ]Bb u zY� � ]�z6a |;u ��b � ]B�V`Y� £ |;u � �(yX| z u b�zc]B��`.`V¨©`Y_ w b u�� i³ y `VzN` w ~�b | zczN� �h}:w ]Bb u z�] �Q}��4¦6wNyX|�w�� ]�z)] u ��` } ` u ��` u[w b;�{�{£ | _Vb u ��] w ]Bb uw�y�|�w `Y_Vb u b � ]Bz w zH_ |��B��1 `J�2bf�f` u `Y] wv¦�3²|;u �â~ y ]�_ y ®�� |;� ]�§X`Vz ��| z |;u ] u z w aN� �� ` u[w?|;� � | aN] | � � `RdLzN`V`)�2`Y_ w ]Bb u z��2i e�i � |;u �M�2i e:iA�fj�aN` �¶|�w ]��;` w b w�y `^aN` �B|�w ]�b u �©` �w ~�`Y` u�� |;u � � ii�Xbfa |p� b u � w ] � `;£�`G� } `YaN] � ` u[w?|;� �f`Yac]�§X_ |�w ]�b u b;�÷~ y ` w�y `Va |� | aN] | � � ` � ]Bz�`G�2bf�f` u bf��z^bfa^] u z w aN� � ` u[w?|���y�| z^��`Y` u6w�y b;��� y[w^w bs��`R] �h} bfz �zN]�� � `Hd tv� ��` u z |�u �M  u �;aN]Bz w �C×f×�e�jG£fzc] u _V` w�y `«��`V§ u ] w ]�b u ] u �;b � �;`Yz#� u b;��zN`Vac� �| � � `H� | _ w bfaNzsdµbfa)��]�z w �:aN� |;u _Y`YzV£ | z wNy ` ¦²| ac`{��zc� |;����¦ _ |;�B� `Y�÷j�zN�:_ yâ| z w�y bfzc`

Page 23: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂx�KÂÁ»�½�¾�Êݽ·Æ�Í«¿ ðÉRÊݽ�¿MÎ�Ä.¾¤É�½¡Ê eÇ�;×aN` } aN`VzN` u�w `Y��� ¦ �sio� ³ y ` u b w ]Bb u b;�H`J�:b;�f` u `V] wv¦ £ � ]Bú�` wNyX|�w b;�p_ | ��z |�w ]�b u] w zN` � �°£ yX| zR��`Y` u �2]�`V~�`Y� | z |6} aNbK����_ w b;�¤zc���K¥c`V_ w ]4�f` � b2�:` � ] u �p¥c����� � ` u[w £`G�2` �Q}:w �Lacb �êwNy `ÒzN_Yac� w ] u[¦ b;� u b u `G� } `Yac] � ` u[w?|;� � |�w?| i

³ y `s��bf� u ��z } aN`Yzc` u[w `V�â] u dI¸:i4�ÿe | �÷j w ` ���Ý| �:]�¨©`YaN` u[w z w bfa ¦ i«þp`Vz } ] w `Ò] w z` � ��zN]4�f` u�|�w ��ac`;£�`G�2bf�;` u `V] wv¦ _ |;u ��` �f]4�f` u�|�u ` �h} ]Bac]B_ |;�«w `Yz w i ³ y ` w `Yz w]Bz u b w �f� | a |;u[w `Y`V� w b²�:` w `V_ w{|;��� �K]Bb �B|�w ]�b u zRb���`G�2bf�;` u `V] wv¦ £Ô��� w ] w _ |;u £�] u_Y`Va w�| ] u _Y]�aN_V� � z w?|;u _Y`YzV£÷zN_Yac`Y` u bf� w �;`Ya ¦ � | �Ö~^bf� � � � �©`H] u z w aN� � ` u[w zVi

g ¦ ] u zN]Bz w ] u � w�yX|0w ` | _ y � }�} `Va{��bf� u ��] u dµ¸:i4�ÿef�÷jp��` y ]B� y `Va w�yX|�u�w�y `_Yb;aNaN`Vz } b u ��] u � � bC~^`VaR��bf� u �²] u dµ¸:i4�ÿe | j^~�`sbf� w�| ] uÇw�y `Ò�Lb ��� bC~�] u � w `Yz w�| � � `_Yb u z w a | ] u�w z�b u´w�y `Òbf��zc`YaÅ�f`Y�.��]Bz w aN]���� w ]Bb u �§

d �£�9��� ��«­�W� j�� §d � 5 ��� ��«­� 5 jqB � �§

d �£�9��� 5 «­�W� j�� §d � 5 ��� 5 «­� 5 jqB � �§

d � 5 ��� ��«­�W� j�� §d �£�9��� ��«­� 5 jqB � �§

d � 5 ��� 5 «­�W� j�� §d �£�9��� 5 «­� 5 jqB ��Ø dI¸:iW�2��j

t � |;u[¦ b;� wNy `Yzc`p] u `V®�� |;� ] w ]�`Yz¤]Bz¤�K]Bb �B|�w `V� £ w�y `±] u �f`Vz w ]�� |�w bfa�_ |;u ��`V����_V` w�yX|0w|�w6� ` | z w b u `.b�� w�y ` | zNzc� �Q}2w ]Bb u z6� u ��`Ya ��¦ ] u ��bf��a � bK��` � ]�zh�K]Bb �¶|�w `Y� | z~�` �B� i t � wNy ` | zNzc]B� u:� ` u�w ]�zH_ | ac`V�L� ����¦ a |;u �:b � ]��Y`V� £ w�y ` ux|�u�¦ �K]�b �¶|�w ]Bb u b;�w�y `YzN`¬] u `Y®�� |;� ] w ]�`Yz � ��z w ��` |0wNw ac]B��� w `Y� w b�zcb � `Ò��]�aN`V_ w ] u�� ��` u _V` wNyX|�w�w�y `| zczN]B� u�� ` u[w#} aNbK_Y`VzNz yX| z#b u zN����¥c`Y_ w z98[ac`Yz } b u zc`Yz±dµ`�i �:i�£ |Rw a | � �Q|�w ]B_�`G� } `VaN] �` u _Y`CjJi#  �4w `Ya uX|�w ]��f` ��¦ £K]4� ��]�aN`V_ w `V¨©`Y_ w z^b;� � b u�� _ |;u ��`±` � ] � ] uX|�w `V� ¯ z |ÿ¦ £w�y aNbf��� y�|�u `G¨T`V_ w ]4�f`���zc`hb�� |´}��B| _Y`V�©b ¯&w�y ` uÑ|;u[¦ bf��zc`Yac�;`Y�x�K]�b �¶|�w ]Bb u b;�w�y `�] u `V®�� |;� ] w ]�`YzÝ_ |;u z | �L` ��¦ �©` |�wNw aN]B�:� w `V� w bRz } ��ac]Bbf�:z+_YbfacaN` �¶|�w ]Bb u �©` w ~^`V` u� |;u �r�{� u�|;� ` ��¦ £ w b | zczN]B� u�� ` u[w �:] | z |;u � y ` u _V` � bfzNz�b;��`G�2bf�f` u `Y] wv¦ i�����i���M�F����M����������� ��� ��¡������H¢£��¤³ y `Q] u `V®K� |;� ] w ]B`VzÒ] u dI¸:iW�2��jJ£�~ y ` u �f` u `Ya |�� ]B�V`Y� w b � � ��w ]�� |;� �:`Y�â� | aN] | � � `VzY£| zczN� � ` w�y `H�Lbfa �

�Q| �ÿ ¯ � Ö �6| �� §d �Ì�¡��«­� j<×�B � � dI¸:iW�f�fj

~ y ]�_ y ]Bz¬_ |��B� `V� wNy ` @8S:AG5IFJ/��6EGSX5 9;1¤@8STE¡èY/:9;1P@85IO i�  } aNbKb;��]�z{�;]��f` u ] u �Ô` | a �dÅ�ÿ×f×Ç��� £ _CjJi° ¡� w ` u ��] u � w�y ` ] u z w aN� � ` u[w�|;� ] u `Y®�� |�� ] wv¦·w b w�y `²_ | zc` ~ y `Yac`¥�;!=?L`P�L�@ËPM@!TcL�^IL�QkG�D¿L¡TSUWR�L�H<G�N?GI½�>A@ÍPJ@`À¦t®K�õ�öWí{§9Æ[½�LOFMLOQ R�HOGI½�DBIF�L`L�@ËP�> ½�BEP�L¡@¨U¿BI@�L�TGIN�PÍa¨GVGIF·½7G�F�L¤>kN@ËP�FMK?½�LON9PJ@·BINTǺ�> N®HOB�@ËL¤GE»¨Tc>A@MH<FMLOD£LONHORIºÇTcG�NGEP·P�LOQ Q�K@·GIUcòÍL¡H(P�> ^IL�QkRa,=?>AHJ=�L�@ËP�> ½�BEP�Li>A@,>kN¿H<GIFMFML�H(P¡e

Page 24: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

e��c��¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ� bfa � ]BzÒ_Vb u[w ] u ��b;��z } aN`Yzc` u[w z u b²z } `V_Y] |;� ��]�ýh_Y� ��wv¦ i t ��à+d �¬« �C�_� jR]Bz w�y `_Yb u ��] w ]Bb u�|;� �:` u zc] wv¦ �L� u _ w ]�b u b;� � �f]4�f` ur� |;u � � £ w�y ` u wNy `M] u `Y®�� |;� ] wv¦��`Y_Yb � `Vz ¨ � �6| �� Ö à+d �¬« �C�_� j § d ��« � j<× ´Ç� B � © � Ø dI¸:iW�;�[j�±bC~�`V�;`YaY£ wNy ` w a |;u zN] w ]�b u�w b | _Yb u�w ] u ��bf�:z � zc]B� uX|�� z | ��a | z w ]B_â_ yX|;u �f`] u ��` yX| �K]BbfaV£ |�u ��] w zN`Y` � z w�y�|�wÒw�y `hz w aN��_ w ��ac`6b;���¡]B�f�:aN`h¸:i4�Q] u ����_V`Yz u b_Yb u z w a | ] u�w ~ y�|�w zcb2`G�f`Ya±b u´w�y `Òbf�:zN`YaÅ�f`V� ��` u zN] wv¦ d���` | a � �C×;×Ç��_CjJi�XaNb � dµ¸:ik�K��j�~�`^zc`Y` wNyX|�wÔw�y `�] u z w ac� � ` u[w?|;� ] u `Y®�� |;� ] wv¦ ]BzÝ�K]Bb �B|�w `V�¬~ y ` uw�y `�_Yb u[w aNb ��� ] u �²] u z w aN� � ` u[wf�ê�Q|;uX| �f`Yz w b } acb2���:_Y`´zN]�� u ]4§X_ |;u[w _ yX|;u �f`Vz] u�wNy `ÖaN`Yz } b u zN`Ö� | aN] | � � ` � ~ y ] � ` w�y ` w ac` |�wN� ` u[w¦� ac` �Q| ] u zM_Yb u z w?|;u[w i  �4w�y b;��� y zN��_ y _ yX|;u �f`Yz¬_Ybf� � � ] uâ} ac] u _V] }�� `>��`h`J� }��¶| ] u `Y� � ¦ z w aNb u �Ö_Ybfa �aN` �¶|�w ]Bb u z{��` w ~�`Y` u � �t� £ |�u � � dµzc] u _V` � ��bK`Yz u b w zN_VaN`V` u b;¨ � �LaNb �� jJ£ w�y `´] u z w aN� � ` u[w�|;� ] u `Y®�� |�� ] wv¦ zc` w z | � ] � ] w b u�wNy ` �Q| � u ] w ���:`6b;� w�y `_ yX|;u �f`YzVi

³ y `¬zN] � ] �B| aN] wv¦ b;� w�y `Ò] u z w aN� � ` u[w?|�� ] u `V®�� |;� ] wv¦Qw bhg^` ��� 8 z�] u `V®�� |;� ] wv¦ ] u®�� |;u[w � � }:y�¦ zc]B_Yz d��2� }�} `Vz.�C×;¸f¸ û \^��z y ] u � |;u � : _ : � ��� ] u �ÿ×f¸f×[j>]Bz u b w| _V_Y]B�:` u[w?|;�Iû ��b wNy ] u `Y®�� |�� ] w ]B`Vzp��` � ] u ` |�w ` | _ �¶| zcz¬b���bf��zc`YaÅ�f`Y��_YbfacaN` �¶|�w ]Bb u zw�y�|�w _ |;u�u b w ��`R`G� }��B| ] u `Y�6� ¦hy[¦K} b wNy `Yzc]B�V] u � �B|�w ` u[w _Vb �Q� b u _ | �:zN`YzVi¤³ y `] u z w ac� � ` u[w?|;� ] u `V®K� |;� ] wv¦ _ |;u £[] uh| zc` u zc`;£K�©`«�K]B`G~^`V� | z | �f` u `Ya |;� ]�� |�w ]Bb u b;�g^` �B� 8 zR] u `Y®�� |�� ] wv¦ �LbfaR_ | zc`Yzp~ y `VaN`���]Bac`Y_ w _ | ��z |;� _Yb u�u `Y_ w ]Bb u ]�z } `Ya � ] wcw `V�w b�b } `Ya |�w `Ò��` w ~�`Y` u²w�y `¬_VbfaNac` �B|�w `V�²bf�:zN`YaÅ� | � � `VzY£ � |;u � � i

³ y `¬] u z w aN� � ` u�w�|;� ] u `Y®�� |;� ] wv¦ _ |;u ��` w ]B� y[w ` u `V� |;}:} aN`V_Y] | � ��¦ ]��Ý~^` | ac`~�] ��� ] u � w b �Q| ú�` | ����] w ]Bb uX|;�+| zNzN� �Q}:w ]Bb u z | �©b;� w zN����¥c`Y_ w z98Ý��` yX| �K]Bbfa ¯ �Lbfa`G� |;�h}�� `;£ w�y�|�w�u bh] u ��]4�K]B��� |�� _ |�u ��`Ò��]Bzc_Ybf��a | �f`V�²� ¦´w�y `¬` u _Yb;��a | �;` � ` u[w] u z w ac� � ` u[w bfa{d �Q|�w�y ` �Q|�w ]�_ |;����¦ j w�yX|�w £��Lbfa |;���É© £�~^` yX| �;`§

d � 5 «­� 5 �¡© j = §d � 5 «­�W�c�4© jIØ

�2��_ y�|;u | zczN� �h}:w ]Bb u |;� bf� u[w z w b yX| �K] u � u bÖ_Yb u[w a | aN] |;u zp] u w�y ` } b } � �¶|0�w ]�b u £ w�y�|�w ]BzY£ u bÇzN�:�K¥c`Y_ w zs~ y bÖ~�] ��� _Yb u zN]�z w ` u[w���¦ _ y b2bfzc` w aN` |�w�� ` u[w _Yb uK�w a | a ¦âw b wNy `Y]�a | zNzN]�� u�� ` u[w i�h u ��`Va wNy ]Bz | zczN� �h}:w ]�b u £ w�y `h] u `Y®�� |;� ] w ]�`YzH] udI¸2ik�2�Cj^_ |;u �©` w ]B� y[w ` u `V��dµg |;� ú�` |;u �ú��` | a � �C×f×f�fj w b ¦ ]B` � �§

d �¨��� 5 «­� 5 j = §d �Ì�¡� 5 «­�W� j§

d �¨��� �¿«­�W� j = §d �Ì�¡� �«­� 5 j dI¸:iW��e�j

�Lbfa |;�B�Y� � �?�¿�W�4� 5¡� i�ªR]Bb �B|�w ]�b u b;� w�y `VzN`´] u `Y®�� |;� ] w ]B`Yz u bC~ � ` |;u z{`Y] w�y `Va

Page 25: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ�«� ¹¡»hÎ�Ê2»�l�ðÉRÆ+¾¤¿)¾�És¹+¾ Æ+¿HÍ�ÄüơðɱÃY½�¾ ÊK»HÄ.¼Ylݾ�Ê e��;�zN` � `V_ w ]�b u ��] | zÖbfaÖ��]�aN`V_ w `V¨©`Y_ w b;� � b u¥� bfa w�y ` } aN`VzN` u _Y`�b��>��`G§ |;u[wzN�:�K¥c`Y_ w zYi

ØÖÙ�¬ p q>árq�qts ÛÝÜ ofnHm#n Ü r�n®­Öm#ßJ� ­ ãÿÜâãVÞ n°¯´q9� �rsYn�qä«å`_�åcæ ± ô�z.zpù³²)ð¤èêé.-ÅôÅñ!�³ y ]�z)zc`Y_ w ]Bb u �:`YzN_VaN]��©`Vz |M� ` w�y b2�Öb;��`Yz w ] �Q|�w ] u �>_ | ��z |;� `V¨©`Y_ w z |;u �Ç_Ybf� uK�w `Vac� | _ w � |��X} aNbf� | ��] � ] w ]B`Yz��LaNb � | § u ] w `�z |��Q}�� `;£ | z } ac`YzN` u[w `Y�´] u \ y ]B_?ú�`YaN] u �|;u �®��` | a � d°�C×f×[�;jGiT´s³ y ` � ` w�y bK�Ò]Bz |;}�}:� ]B_ | � � `¡~�] wNy ] uHw�y `�g |C¦ `Yzc] |;u �La |;� ` �~�bfaNú÷£ | _Y_Yb;aN��] u � w b{~ y ]�_ y dµ]¶j |�u�¦ � u ú u bC~ u z w?|�w ]Bz w ]B_ |;�X}X| a |;� ` w `Va�_ |;u ��`| zczN]B� u `Y� } ac]Bbfa } aNbf� | ��] � ] wv¦�|;u �dµ]B]¶j wNy ` `Yz w ] �6|�w ]Bb u b;� w�yX|0w´}X| a |;� ` w `Va|;� bf� u[w z w b�_Vb �h} � w ] u �x] w z } b;z w `VaN]�bfa6��]�z w aN]B�:� w ]�b u £^_Vb u ��] w ]�b u `Y�Áb u&w�y `z |��Q}�� `Y�´� |0w?| i tvu bf��a^_ | zc` wNy ` }X| a |;� ` w `Va�] u ®���`Vz w ]Bb u ]Bz w�y ` } acbf� | �:] � ] wv¦§dËå9ÿ � å �Yj�dµbfa §

dËå;j��Lbfa¤z y bfa w jJ£��Lacb � ~ y ]�_ y ~�`R_ |;u ��`Y���:_Y`R )\� ±d �ÒÑ � jGit �p~�` w�y ] u ú�b��

§dÍåfj u b w�| z } acbf� | �:] � ] wv¦ �:� w a |�w�y `Ya | z wNy `Ö�La | _ w ]Bb uµ·¶ b;��] u �:]��K]B��� |;� z¬] uxwNy ` } b } � �B|�w ]�b u ~ y b } bfzczN`Yzcz�aN`Yz } b u zN`´_ yX| a | _ w `Yac]Bz �w ]�_Yz¬�f]��;` u � ¦ ãäª åK£ wNy ` uxwNy `h]B��` | b;� | zNzN]�� u ] u � } aNbf� | ��] � ] wv¦.w b.zc��_ y·|

®�� |;u[w ] wv¦ ~�bf� � � § wHw�y `sz w?|�u � | ac� }:y ] � bfzNb }:y�¦ b;��g |ÿ¦ `VzN] |;uÑ|;uX|;�4¦ zc]Bz û µ·¶ ]�z|²} b w ` u[w ] |;�B�4¦.� ` | zc��a | � � `.d |;� �©`V] w � u ú u b�~ u j }�y[¦ zN]�_ |;� ®�� |;u[w ] wv¦x|�u �x_ |;uw�y `YaN`G�LbfaN` | � � ] w¬|Ç} aN]�bfa } aNbf� | ��] � ] wv¦ £Ýb u ` wNyX|�w ` u _Vb2��`Vz>bf��a¬� u _V`Ya w?| ] u�wv¦] u´wNyX|�w ®�� |;u[w ] wv¦ i

 RzczN� � ` w�y `YaN` | aN`[¸ózN����¥c`Y_ w zp] uÇw�y `{`G� } `YaN] � ` u[w i ª `{��zN` �¹(�V��¹M����¹w b.��` u b w ` wNy `�bf�:zN`YaÅ�f`V�·� |;� ��`hb;� �V�®���¦� £¡aN`Yz } `V_ w ]��f` ��¦ £+�Lbfa{zc���K¥c`V_ w1º i�2] � ] �¶| a ��¦ £Ô~^`Ç��zc`úå ¹ w bÑ��` u b w ` wNy `�dµ� u b;��zN`Vac�f`V�÷jh_Vb �h}�� ] |;u _Y`âdËå9ÿCj |�u �aN`Vz } b u zc`MdËå��ÿj+_Vb � ��] uX|�w ]�b u �Lbfa¤zN����¥c`Y_ wDº i ª `)��zN`�» ¹ w b{�:` u b w ` wNy ` w aN] }�� `��¼¹(����¹J�4�½¹ � i�Ò]4�f` u�w�y `´bf�:zN`YaÅ�f`V�Á� |0w?| » �LaNb �üw�y `6`G� } `VaN] � ` u�wh|�u � | } aN]�bfas�:]Bz �w ac]B��� w ]Bb u b��;`Ya w�y `�� u ú u b�~ u �La | _ w ]�b u z µ·¶ £Ýbf��a } acbf� � ` � ]Bz w bÖ��`Yac]��f` w�y `} bfz w `YaN]�bfaÝ��]Bz w ac]B��� w ]Bb u �Lb;aÔ )\� )d � Ñ � jpi�³ y ` } bfz w `YaN]�bfa���]�z w ac]B��� w ]Bb u z�b;���b w�y µ¼¾ |;u �� )\� )d � Ñ � j�_ |;u �©`���`VaN]4�f`Y����zN] u � w�y `h�fa |;}�y ]�_ |;�+� bK��` �z y b�~ u ] u �¡]B�f�:aN` ¸:i e�£p~ y ]�_ y `G� }�� ]B_V] w��4¦ ac` } ac`YzN` u[w z wNy `�] u ��` } ` u ��` u _Y]B`Vzw�y�|�w^y b � �h] u�w�y `+¥cbf] u�w dµg |ÿ¦ `Yzc] |;u j¤��]�z w aN]B�:� w ]�b u ��`V§ u `Y��bC�f`Va wNy `)� | ac] | � � `Yz� » � µ¾ �  )\� ±d � Ñ � j � i´³ y ` � bK��` � _ |;u ��`Q� u ��`Yacz w bKbK� | z[¸ aN` |;� ]�� |0�¿ \�@�>k½�> Q BIFS½�L<P�=GWT¶º!P�=?G�K?dI=�QABIHJÊW>kNdtP�=?L.d�FMBID?=?>AHOBEQ,D£LOFJ@�D¿L¡H(P�> ^IL�º!> @�D?F�L¡@ËL�N9P�L�Tæ> N¹�½¤U£LON¿@CBENT�À,K?U>kN¦ÀÍõ¡öIöWíEÆ(e

Page 26: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

e����:¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉw ]�b u z>b�� wNy `�ac`Yz } b u zc` � � | aN] | � � ` � bK��` � d��¡]��f��aN`6¸:i �[jG£#b u `6�Lbfa>` | _ y�w aN] }�� `] u ».£p_Vb u�u `V_ w `Y� w bf�f` w�y `VaÇ�:zN] u � wNy ` u bK��` aN` } aN`Yzc` u[w ] u � w�y ` � u ú u bC~ u�La | _ w ]�b u z µ¾ ª�d µ�¶ � � µ·¶�Á � Ø9ØWØ � µ�¶ �¦Â jGih³ y ` � bK��` � `G� }�� ]�_Y] w���¦ aN` } aN`VzN` u�w z w�y `| zczN� �h}:w ]�b uswNyX|�w £;�f]��;` u>w�y `^�La | _ w ]�b u z µ¾ £ wNy ` } acbf� | �:] � ] wv¦ b;� | zN�:�K¥c`Y_ w �©` �� b u �f] u � w b |�u�¦ b;� w�y `h�cgh_Yb �h}�� ] |;u _V` � ac`Yz } b u zN`ÒzN��� } b } � �¶|�w ]Bb u z)�:b2`Vz u b w��` } ` u �²b uÖwNy `H_Yb �Q}�� ] |;u _Y` |�u �ÖaN`Yz } b u zN`¬�©` yX| �K]Bbfa�b�� wNy `Hb w�y `Ya«zN�:�K¥c`Y_ w z] u�wNy `p`J� } `Yac] � ` u[w i¤�XaNb � dµ¸:i��W�[jG£2 )\� ±d � Ñ � j^]�z | ��` w `Ya � ] u ]Bz w ]B_)�L� u _ �w ]�b u b�� µ¼¾ |�u �â_Yb u zN`Y®���` u[w���¦  )\� )d � Ñ � j^]�zR] u ��` } ` u �:` u[w b;� |;��� b w�y `Va� | aN] | � � `Yz�] uÖwNy `Ò��b �Q| ] u b u _Y` w�y `YzN`Ò�La | _ w ]�b u z | ac`Hú u bC~ u i

z à 1 Ä

x Å 1

r Æ 1 Ä

y Ç 1 Ä

ν È R

z à 2 É

x Å 2

r Æ 2 É

y Ç 2 É

z à m Êx Å m Ê r Æ m Ê

y Ç m Ê

ACE(X Ë

Y)

�¡]��f��aN`�¸:i e:� : bK��` � ��zN`V� w bâaN` } aN`Yzc` u[whw�y `´] u ��` } ` u ��` u _V]B`Vzh] u §d � » �ÍÌ� µ¼¾ �0Ì �  )\� Rd � Ñ � j � jGi

tvus} ac] u _V] }�� `;£ w�y ` u £�`Yz w ] �Q|�w ] u �« )\� ±d �ÓÑ � j�ac`Y���:_Y`Yz w b w�y `�z w�|;u � | ac�] u �L`YaN` u _Y` w�| zNúÇb;��_Yb �Q} � w ] u � w�y ` } bfz w `VaN]Bb;a } acbf� | ��] � ] wv¦ �Lbfa | � | ac] | � � `s] u| �L� ����¦ z } `Y_V]�§X`V��g |C¦ `Yzc] |;u u ` w ~^bfacú÷i´dµ³ y `M�fa |;}:y ]B_ |;�Ôw `Y_ y�u ]B®���`YzÒ�Lbfa w�y ]�z] u �L`YaN` u[w ] |�� _Yb �h} � w�|�w ]�b u�| aN`Ç��aN]�` ��¦ zN� �h�6| aN]B�V`Y��] u �2`Y_ w ]�b u �;ik�2i e�iWj tvu�Q|;u[¦ _ | zN`VzY£ wNy `^] u ��` } ` u ��` u _V]B`Vz#` � ��b2�:]B`Y�{] u¬w�y `��fa |;}:y _ |;u �©`�`J� }�� bf] w `V�w bsaN` u ��`Ya w�y `p] u �L`Yac` u _V` w�| zNúh`Výh_Y]�` u[w iYh u �Lbfa w � uX|0w ` �4¦ £2�©`V_ | ��zc` w�y `å ¹�| ac`u `G�f`Ya�bf��zc`Yac�;`Y� £:��`Yac]��K] u � wNy ` } bfz w `Yac]Bbfa¤��]�z w aN]B�:� w ]�b u �Lbfa¤ )\� ±d � Ñ � j^]�zu b w�w a | _ w?| � � `´] u bf��a � bK��` � £�`G�f` u ~�] wNy�wNy `Ö�f]��;` u ] u �:` } ` u ��` u _Y]�`YzViÁ³Ôbbf� w�| ] ux|;u `Vz w ] �6|0w `6b�� w�y ` } bfz w `VaN]Bb;as��]�z w ac]B��� w ]Bb u b;�� )\� ±d �ÒÑ � jG£ |;u|;}�} aNbÿ�2] �Q|�w ]�b u�w `Y_ y�u ]B®���`âú u bC~ u¢| z �H]B���:z6z |��Q}�� ] u �Ñ_ |�u �©` �:zN`Y�Ti  �fa |;}�y ]�_ |;� �;`YaczN]Bb u b;� w�y ]Bz w `V_ y�u ]B®���`�£�_ |��B� `V� 1 z w b2_ y�| z w ]B_ÖzN] � � �¶|0w ]Bb u £ 3 ]�z��`VzN_Yac]B��`Y�Ö] u �Ô` | a � dÅ�C×;¸f¸;� £ } i÷�2�c�fj û2wNy `H��` w?| ] � zÒd | z |;}�}:� ]B`V� w b wNy `H�fa |�}�yb;���¡]B�f�:aN`�¸:i e�j | aN`���]�zN_V��zNzc`Y��] u \ y ]�_?ú;`VaN] u � |;u �¢��` | a � d°�C×f×[�fjJiæ�)`Yac`h~�`} ac`YzN` u[w�wv¦K} ]�_ |;� ac`YzN� ��w zV£2] uMw�y `«�Lbfa � b;� y ]Bz w bf�fa |;� zY£ wNyX|�w ��` � b u z w a |�w ` w�y `

Page 27: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ�«� ¹¡»hÎ�Ê2»�l�ðÉRÆ+¾¤¿)¾�És¹+¾ Æ+¿HÍ�ÄüơðɱÃY½�¾ ÊK»HÄ.¼Ylݾ�Ê e��ÿ��f` u `Ya |;�Ý|;}�}�� ]B_ | ��] � ] wv¦ b;� w�y ]Bz w `V_ y�u ]B®���` w b } acbf� � ` � z«b;�¡_ | ��z |;� ] u �L`VaN` u _Y`�i

ä«å`_�å(' /ÚÙpë õga>ëÝîTï:ù *�í�²)ð¤è�é.-Åëβ±ô<Ï�ë ð¤ñpò yÑì÷ôO*�ìÑÐ ô°ùKï:ì÷ôJÝz.}±ïXô<*�ñ³ y ` � ` wNy bK� w?| ú;`Yz | z.] u:} � w dÅ��j w�y `�bf��zN`Vac�;`Y� � |�w�| » £¬`J� } aN`VzNzN`V� | zw�y ` u � � ��`Ya�b;�p_ | zc`Yz´bf��zc`YaÅ�f`Y���Lbfah` | _ y b;� wNy ` ¸ } bfzczN]�� � `²aN` |�� ]B� |�w ]�b u zb;� ������C�4� � £ |;u �¢d��;j | þp]Bac]B_ y�� ` w¬} ac]Bbfa{b��;`Ya wNy `�� u ú u bC~ u �La | _ w ]�b u z µ¼¾ £`G� } ac`YzczN`Y�Ö] uhw `Ya � z�b;�+�cg }X| a |;� ` w `VaNzYi¡³ y `Rz ¦ z w ` � bf� w�} � w z wNy ` } bfz w `Yac]Bbfa��]�z w ac]B��� w ]Bb u b;�¡ )\� )d � Ñ � jR£X`G� } ac`YzczN`Y�.] uÇ|My ]Bz w bf�fa |;� i

³�bÓz y bC~ wNy `�`V¨©`Y_ w b;� w�y ` } ac]Bbfaâ��]�z w aN]B�:� w ]�b u b u�wNy `·bf� wN} � w £M~�`} ac`YzN` u[w.|��B�«w�y `âac`Yzc� ��w z���zN] u � w ~�b���]�¨©`Yac` u[w´} ac]BbfaczYi ³ y ` §Xacz w ]�z |¢�÷|0wdµ� u ]��Lbfa � j¤��]Bz w ac]B��� w ]Bb u b��;`Ya w�y `M�Wg � �;`Y_ w bfa µ¼¾ £ |;u �Ö]Bz«_Vb �h� b u:��¦ ��zN`V� w b`G� } ac`Yzcz6]B� u bfa |�u _Y` | ��bf� wMwNy `Ç��b �Q| ] u ix³ y `²zc`Y_Vb u � } aN]Bb;a>]Bz�zNú�`V~�`Y� w baN` } aN`VzN` u�wQ| z w aNb u �â��` } ` u ��` u _ ¦ �©` w ~^`V` u·w�y `´_Vb �h}�� ] |;u _Y` |;u �·ac`Yz } b u zc`_ yX| a | _ w `Yac]Bz w ]B_Vz�b;� w�y ` zN����¥c`Y_ w zYiÜ�¡]B�;��aN`.¸:iA�Ñz y bC~�z w�y `.��]Bz w aN]���� w ]Bb u b;� )\� )d � Ñ � j«] u ����_V`Y��� ¦�w�y `YzN` w ~^b } aN]Bb;a���]�z w aN]B�:� w ]�b u z±dµ] uhwNy ` | �:zN` u _Y`b;� |;u[¦ � |�w�| jGi ª `Özc`Y` w�yX|0wMw�y `ÖzNú�`V~�`Y� } aN]Bb;a>b;��¡]B�f�:aN`´¸:i ��dL�÷j | zNzc]B� u z|;��� bfz w�|;���©w�y `Ò~�`Y]�� y[w�w b u `Y� |�w ]4�f`H� |;� ��`Vz�b;�¡ )\� ±d �ÒÑ � jGi

(a) Ò

(b) Ò-1 1 -1 0 1 0

�¡]��f��aN`�¸:i �2� d | j ³ y ` } aN]�bfa ��]Bz w ac]B��� w ]Bb u b;�& )\� ±d � Ñ � j^] u ����_V`Y�� ¦ �÷|�w�} aN]�bfaNz�bC�;`Ya w�y ` }�| a |;� ` w `Yacz µ¼Ó�¾ i dµ�÷jx³ y `¢�:]Bz w aN]���� w ]�b u �Lb;a )\� )d � Ñ � j«] u ����_V`Y�Ç� ¦ zNú�`V~�`Y� } ac]Bbfacz«b��;`Ya w�y ` }�| a |;� ` w `YaczYi

³�b²] ��� ��z w a |0w ` y bC~ ] u _Yac` | zc] u �Öz |;�Q}:� `MzN]��Y`�~ | z y `Vz | ~ |ÿ¦ w�y `h`G¨T`V_ w b;�w�y ` } aN]Bb;aH��]Bz w aN]���� w ]Bb u £©~^` |;}�}��4¦.wNy ` � ` wNy bK� w bÖzN] � � �¶|�w `Y� � |�w�| ��a | ~ u�LaNb �'| ��]Bz w ac]B��� w ]Bb u

§d �C�¡�¬« � j��Lbfa�~ y ]�_ y  )\� &]Bz«ú u bC~ uÇw b��©`Ò]B�:` u[w ]4§X`Y� i

�2��_ y�| ��]�z w ac]B��� w ]Bb u ]�z¬z y bC~ u ³ | � � `Q¸:i4�fiü�Xbfa wNy ]BzÒ��]�z w aN]B�:� w ]�b u £ w�y `haN` �zN� ��w ] u ��� }:} `Va |;u � � b�~�`YaR�©bf� u ��zpb;�Rdµ¸:i��Ye | �÷j�_Yb ���¶|;} zc` w b | zN] u � � ` } b;] u[w � )\� )d � Ñ � j2ª¥��Ø ���2i�¡]B�;��aN`¬¸:ikghz y bC~�z w�y `{bf� w�} � w b;� w�y `��Ò]�����z�z |��Q}�� `Ya«~ y ` uâ|;}�}�� ]B`V� w b� |�w�| zN` w zÒb;�¤� | aN]Bb;��zpzN]��Y`Vz¬�:a | ~ u �Lacb � w�y `���]�z w ac]B��� w ]Bb u z y bC~ u ] u ³ | � � `

Page 28: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

e��ÿeX¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ

� � � §d �C�4�Ì�_� j� � � �:iW�f���� � � �:ik�� � � �:iW�f���� � � �:ik�

� � � �:iW�f���� � � �:ik�� � � �:ik�� � � �:iW�f���

³ | � � `Ò¸2i��f�DÔ @BAG5IFJ@ ­ /K5I@�-0SâF�E?AG/K1P5I@8S��Q@8SÑ9;S.@µ3KEGSX5I@ #�9 ­ 1BE�Õ9ÖM×�Ø �ÒÑ �ÍÙ¸:i4�f£[��zc] u �¬�©b w�yMw�y ` �÷|0w^|�u � w�y `)zNú�`V~�`Y� } ac]BbfaVi¡ ±z�`J� } `V_ w `V� £ | z w�y ` u � �����`YaHb;��_ | zc`Yz¬] u _VaN` | zN`VzY£ w�y ` } bfz w `Yac]BbfaH��]�z w aN]B�:� w ]�b u zp��`Y_Vb � `M] u _Yac` | zc] u � ��¦_Yb u _Y` u�w a |�w `Y� u ` | a wNy `p� |;� �:`.��Ø ���2i tvu �f` u `Ya |�� £���`Y_ | ��zN` wNy `HzNú�`V~�`Y� } ac]Bbfa�LbfaÒ )\� )d �ÓÑ � j^]BzÒ_Yb u _Y` u[w a |�w `Y�x�L��a w�y `VaH�LaNb � �%ØA��� w�y�|;uÑwNy `h� u ]��Lb;a �} ac]BbfaV£ � b;aN`«_ | zN`Yz | ac` u `Y`V��`Y�Q�©`G�LbfaN` wNy ` } bfz w `YaN]�bfa#��]�z w ac]B��� w ]Bb u _Yb u �;`YaN�;`Yzw b wNy `H� |�� ��`���Ø ���2iä«å`_�å�� h�ðY}pù2ðY-¬ë�asëÔîTï:ù�íJì�*�è îC-ÅôcñHô°î�ðY-{òHð¡ï:ðÛÚ�ôvï[Ù ôcè�ésëÝì%Ý

íJëÔîTï�î,*�èêé.-ÅôÅð¤ñpî�ëtvu�w�y ]Bzszc`Y_ w ]Bb u ~^` |;uX|;�4¦ �V` w ~�bâ_ � ] u ]B_ |;� � |�w�| zN` w z>b;� w?| ] u `Y�x� u ��`Va>_Yb uK���] w ]Bb u z.b;��] �Q} `YaÅ�L`Y_ w _Yb �Q}�� ] |;u _Y`;i \^b u zc]B��`Va §XaNz w w�y `Ú�Ý] } ]B� � `VzN` | aN_ y\ � ] u ]B_VzÒ\^bfacb uX| a ¦ �#aN] �6| a ¦ �#aN`G�f` u�w ]Bb u � |�w�| ��`Yzc_YaN]��©`V��] u �2`Y_ w ]�b u ¸:ik�Ki g:i³ y `Mac`YzN� ��w ] u ��� |0w?| zc` w d | � w `Ya w�y ac`Yz y b � �:] u ��j�]Bzpz y bC~ u ] u ³ | � � `M¸:iW�2i�h±z �] u � wNy ` �¶| ac�f` � z |;�h}�� ` | zNzc� �h}:w ]�b u £Mdµ¸:i4�ÿe | �÷j6�;]��f`Vz w�y `Ñ�©b;� u ��zú��Ø �f×ÜB )\� )d � Ñ � jÝBø��ØW�;¸:i�¡]B�;��aN`Ö¸:iW� z y bC~�z } b;z w `VaN]�bfa>��` u zN] w ]B`VzM�LbfaM )\� )d � Ñ � jp£�� | zc`Y��b uw�y `YzN`�� |�w�| i �)|�wNy `YaÔaN` �6| acú | � ��¦ £�`V�f` u ~�] wNy b u:��¦ �f�f��_ | zN`Vz+] uÒw�y `�� |�w�| zc` w £��b w�yÑ} b;z w `VaN]�bfaH��]Bz w ac]B��� w ]Bb u z | aN` y ]�� y���¦ _Yb u _Y` u[w a |�w `Y�x~�] w�y ] u wNy ` �¶| ac�f` �z |��Q}�� `H�©b;� u ��z�b;�·�:i �f× |;u �³�:ik��¸:i

Page 29: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ�«� ¹¡»hÎ�Ê2»�l�ðÉRÆ+¾¤¿)¾�És¹+¾ Æ+¿HÍ�ÄüơðɱÃY½�¾ ÊK»HÄ.¼Ylݾ�Ê e��c�

0.55

(b) Þ

(c) Þ

(d) Þ0.55

(a) Þ

(e) Þ

(f) Þ

(g) Þ

(h) Þ0.55 0.55 0.55 0.55 0.55 0.55

�¡]��f��aN`�¸2i g:�i$p� w�} � w^y ]�z w bf�fa |�� z+�Lbfa¤]���` u[w ]4§X`Y� w ac` |�wN� ` u[w `G¨T`V_ w ��zN] u � w ~�b} ac]BbfaczYi^d | jJ£:dL�÷jG£2dL_CjG£ |;u �6dµ�Xj�z y bC~ wNy ` } bfz w `VaN]Bb;aNz��Lbfa� )\� )d �ÓÑ � j^��zN] u �w�y ` �X|�w�} aN]�bfa |;u �²� |0w?| zN` w z w�yX|�w _Vb u zc]Bz w `Y�Çb;���c�:£ �c�£�:£ �f£k����� |;u ���c�2£ ���£�zN�:�K¥c`Y_ w zY£Òac`Yz } `Y_ w ]4�f` �4¦�û dµ`ÿjG£�d8�?jJ£MdL��jG£ |;u ��d y j�z y b�~ w�y ` } bfz w `YaN]�bfaNz��Lb;a )\� )d � Ñ � j«�:zN] u � w�y `.zNú�`V~�`Y� } ac]BbfaM~�] wNyÁwNy `.z |;� `ÇaN`Vz } `Y_ w ]��;` � |�w�|zN` w zVi

(a) ß

(b)

0.39 0.78 -1 0 1 -1 0 1 0.39 0.78

�¡]��f��aN`±¸:iW�2�Y$p� wN} � w�y ]Bz w bf�fa |;� z#�Lbfa w�y `��] } ]��6� |�w�| i�d | j¤h)zN] u � �÷|�w^} aN]�bfaNz|;u �xdµ�Xj^��zN] u �Qzcú;`G~^`V� } aN]�bfaNzVi

 Rz | zc`Y_Vb u �6`J� |;�Q}:� `;£f~�`)_Yb u zc]B��`Va |�u `J� } `VaN] � ` u[w ��`Yzc_Yac]B��`Y�h� ¦ �Kb ���� `Ya#` w�|�� i«d°�C×f¸�gfj w�yX|0w ~ | z¤��`VzN]�� u `Y� w bÒ��` w `Va � ] u ` wNy `�] �h}X| _ w b;���2] w?|;� ] u &zc� }�}�� ` � ` u[w?|�w ]Bb u b u _ y ] � � y bKbK� � bfa w�|;� ] wv¦ i tvu>w�y `^z w �:� ¦ £[e¶�¿�±�K] �B�B| �f`Vz�] uu bfa w�y `Va u �K� �6|0w a | ~^`VaN`Ma |;u ��b �h��¦´| zNzc]B� u `V� w b }X| a w ]B_V] }X|�w `Ò] uâ| �2] w?|;� ] u  zN� }�}:� ` � ` u[w�|�w ]�b u zc_ y ` � `.b;ahzc`YaÅ�f` | z | _Yb u[w acb � �faNb;� } �Lbfa�b u ` ¦ ` | aVi\ y ] � ��ac` u ] u�wNy ` w ac` |�wN� ` u[w �faNbf� } ac`Y_V`Y]��;`Y� w ~^b �B| aN�f`R��bfzN`Vz�b;�Ý�2] w?|;� ] u  d � 5 jJ£f~ y ] � ` w�y b;zN`�] uMw�y `)_Vb u[w acb � �facbf� } aN`Y_V`Y]4�f`Y� u b w ac` |�wN� ` u�w d � � jGi+ )� w `Vaw�y ` ¦ ` | a yX| ��`J� } ]BaN`V� £ w�y ` u � � ��`YaHb��^��` |�w�y z �¿� ~�`Yac`Q_Vbf� u[w `V�Ñ�Lb;aH�©b wNy�facbf� } zVi�³ y `¬ac`Yzc� ��w z)b�� wNy ]Bz�z w ��� ¦Ö| aN` |;� zcb�z y bC~ u ] u ³ | � � `¬¸:iW�2ih u �:`Ya w�y ` �B| aN�;` � z |;�h}�� ` | zNzc� �h}:w ]�b u £ wNy `·] u `V®�� |;� ] w ]�`YzÇb;�âdµ¸:i4�ÿe | �÷j¦ ]B` � � w�y `Ö��bf� u �:zúÕV��Ø4�C×àB  )\� )d � Ñ � jáBÒ��Øk�:�fi&��]B�f��ac`´¸:i ¸âz y bC~�z} bfz w `YaN]�bfa¬��` u zc] w ]B`Yz¬�Lbfas )\� )d � Ñ � jR£+�f]4�f` uxw�y `6� |�w?| £��Lbfa w ~^b } aN]Bb;aNzYi

Page 30: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

e��cg�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ

�Ý] } ]B�Ç� w ��� ¦ ªR] w�|;� ] u  �� w �:� ¦� � � $H�:zN`YaÅ� |�w ]�b u z $p��zc`YaÅ� |0w ]Bb u z� � � �?�;¸ �0e� � � �ÿe �;�f£A�K�ÿe� � � � �� � � � �� � � �f� ��e� � � ��� �K£W�f¸��� � � �;� �C�� � � �;¸ ×2£ g�g;�

³ | � � `Ò¸:iW�2�ãâ�­ ACEGFJD;EN3�3[9;5�9±Uÿ-0FH5L7XEåä�@ ��@µ3hAG5µ/�3;O69;S�3Q587XEçæÝ@85�9£�M@8SgÕ�AG5I/:3;O

t w ]Bz^] u[w `VaN`Yz w ] u � w b u b w ` w�y�|�w £:�Lb;a w�y ]�z�z w ��� ¦ £ w�y `H_ y b;]B_Y`Hb�� wNy ` } ac]Bbfa��:]Bz �

(a) (b)

-1 1 -1 1 0 -0.19 0 -0.19

�¡]��f��aN` ¸:iW¸2� $H� w�} � wÇy ]�z w bf�fa |�� z6�Lb;a w�y `�ªp] w?|;� ] u   �2� }�}�� ` � ` u[w?|0w ]Bb u� |�w�| �)d | j^��zN] u � �X|�w�} aN]�bfaNz û dµ�Xj«�:zN] u �hzcú;`G~^`V� } aN]�bfaNzViw ac]B��� w ]Bb u�yX| z | zc]B� u ]4§X_ |�u�w `V¨©`Y_ w b u�w�y ` } bfz w `Yac]BbfaVi+³ y ]Bz«zc���f�f`Vz w z wNyX|�w ]4�w�y `M_ � ] u ]B_V] |;u ]Bz u b w �;`Ya ¦ _Yb u §X��` u[w ] u w�y ` } aN]Bb;a w�y ` uÑ| zN` u zN] w ]��K] wv¦ |;u�|;�4�¦ zN]�zMz y bf� � �·�©` } `YaÅ�Lbfa � `Y�Ti tvu zN��_ y _ | zN`YzV£ wNy ` | z ¦2�h}:w b w ]�_��©b;� u ��z | ac`� bfaN`�] u �Lbfa �Q|�w ]4�f` w�y�|;u�wNy `�g |ÿ¦ `YzN] |;u `Vz w ] �Q|�w `VzY£ |�u � wNy ` �Q| ¥cbfaÒaNb � `�b;�w�y `��Ò]����!8 z±z |;�h}�� `Va±~�bf� � �.�©` w b��f]4�f` |�u ] u �:]B_ |0w ]Bb u b;� w�y `>z yX| a }�u `YzczHb;�w�y `¬��bf� u � | aN]�`Yz | acbf� u � w�y bfzN`Ò��bf� u ��zVi

Page 31: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂ�«� ¹¡»hÎ�Ê2»�l�ðÉRÆ+¾¤¿)¾�És¹+¾ Æ+¿HÍ�ÄüơðɱÃY½�¾ ÊK»HÄ.¼Ylݾ�Ê e��C�ä«å`_�åHG I·ð�j�ëÝù�ô°ð¤ñ õ²ùKï�ôcèøð¡ï:ë *�íè²±ôcñ!�{-°ë!Ý�õúÛ^ëÔñ^ï hxðY}pù:ð`Ý

ïXô<*�ñtvu�| ����] w ]�b uxw b | zczN`VzNzN] u �Ñ_ | ��z |;� `G¨T`V_ w zV£ wNy `Ög |ÿ¦ `VzN] |�u�� ` wNy bK� ¥c�:z w ��` �zN_VaN]��©`V��]�z |;� zcbÇ_ |;}X| � � `²dL~�] w�y b u��4¦ � ] u bfa � bK��]�§X_ |�w ]�b u j�b;� |�u zc~�`Yac] u � |� | aN]�` wv¦ b;�H_Vbf� u[w `Vac� | _ w � |�� ®���`Yac]B`Vz6_Yb u _Y`Va u ] u ��] u ��]��K]B�:� |;� zM~�] w�y z } `V_Y]4§X__ yX| a | _ w `Yac]Bz w ]B_VzYiêéH�:`YaN]�`Yz´b;� wNy ]Bz wv¦K} `â~^`VaN` |;uX|���¦ �Y`V� |�u ���©bf� u ��`Y��] u�2`V_ w ]�b u ¸:i �x� u �:`Ya w�y ` �¶| ac�f`.z |;�h}�� ` | zNzc� �Q}2w ]Bb u i tvu&w�y ]�z6zc`Y_ w ]Bb u £)~�`��` � b u z w a |�w ` | g |ÿ¦ `YzN] |;u·|;uX|���¦ zN]�z{b;� w�y `h�Lb �B� b�~�] u �Ö®���`Ya ¦ i ª�y�|�w ]�z w�y `} acbf� | ��] � ] wv¦.w�y�|�w�ë bK`�~^bf� � � y�| �;` yX| � |;u ] �h} aNbC�f`V�x_ y b � `Yz w `VaNb � aN` | ��] u �yX| � y ` w�| ú;` u _ y b � `Yz wv¦ a |;� ] u `�£��;]��f` u�wNyX|�w �ÑdÅ��j ë bK`�~ | zh] u�w�y `Ö_Yb u�w aNb ��facbf� } b;� wNy `¦��] } ]��.z w ��� ¦�û dI�fj ë bK` w bKbfú w�y ` }��¶| _V`Y��b | z } aN`Yzc_Yac]B��`Y� £ |�u �dI�fj ë b2`¿8 z�_ y b � `Yz w `Yacb ��� `G�f` � �:]B� u b w ] �Q} acbC�f`;i

³ y ]BzQ®���`Ya ¦ _ |;u ��` |;u zc~�`YaN`V� � ¦ ac� u�u ] u � wNy `r�H]B���:zhz |;�h}�� `VaQb u�|� b2��` � ]���` u[w ]�_ |;��w b w�y�|�w z y bC~ u ] u �¡]��f��aN`>¸:i e:£�`G�2_Y` }:wÒw�yX|�wHwNy `>�L� u _ w ]�b u )\� )d � Ñ � j¬dL`Y®�� |�w ]Bb u dµ¸:i��W�[jNj�]Bz«ac` }��B| _Y`V�.� ¦Ç|;u b w�y `Va��L� u _ w ]Bb u b;� µ¼¾ £b u ` w�yX|0w aN` } aN`VzN` u�w z´bf��ah®���`Ya ¦ i t � ë bK`Ç~ | z�] uÁwNy `._Vb u[w aNb � �facbf� } |�u �w bKbfú w�y ` }��B| _Y`V�©b�£ w�yX|0wp� ` |;u z y `>]�z±`V] w�y `Ya | _Yb �h}�� ]�`Ya�bfa |6u `V�;`Ya � w?| ú;`YaVi�X��a wNy `Ya � b;aN`;£X��`Y_ | ��zc` ë bK`£8 z±_ y b � `Vz w `VaNb ��� `V�;` � ��]B� u b w ] �Q} acbC�f`;£ ë bK`£8 z�aN` �z } b u zc`´�©` yX| �K]Bbfa{]Bz>`V] w�y `Ya u `V�f`Va � aN`Y_VbC�f`Yahbfa y ` ��} `V� iÑ\^b u zc`Y®���` u[wN��¦ £ ë bK`� ��z w �©` |«� ` � �©`VaÝb;�:b u `�b�� wNy `��Lb �B� bC~�] u �^�Lbf��aÝ_Vb �Q}:� ] |�u _Y` � aN`Vz } b u zc` } b }2�� �B|�w ]�b u zY� � dËå9ÿ�ª � � å �¦ª'��j � dËå9ÿfª � � å��tª��fj � dËå9ÿ�ª'� � å��¦ª'��j � dÍå9ÿfª� � å���ª �fj � i ë bK`#~^b;� � � yX| �f`�] �h} aNbC�f`V� y�| � y ` w�| ú;` u _ y b � `Yz wv¦ a |;� ] u `¤]�� y ]�zaN`Vz } b u zc`s��` yX| �K]Bbfa�]�z)`V] wNy `Ya y ` ��} `V��dÍå �ª���j«bfa |;� ~ |ÿ¦ z � ac`Y_YbC�;`YaMdÍå �ª �[jGit w �Lb �B� bC~�z w�yX|0w«w�y `Ò®���`Ya ¦ b;��] u[w `VaN`Yz w ]Bz�_ |;}2w ��ac`Y�Ç� ¦6w�y `H�L� u _ w ]�b u

à+d µ¼¾ j¤ª µ � 5 � µ 5(5µ � 5 � µ � ² � µ 5(5 � µ 5 ² Ø�¡]B�;��aN`Vz�¸:i ×Xd | j |;u �dL�÷jQz y bC~ wNy ` } ac]Bbfah��]Bz w ac]B��� w ]Bb u b;�®à+d µ¼¾ j w�yX|0w�Lb ��� bC~�z±�Lacb � wNy ` �÷|0wH} aN]�bfa |�u � w�y `Mzcú;`G~^`V� } ac]BbfaV£TaN`Vz } `Y_ w ]��;` �4¦ i®�¡]B�f�:aN`Yz

¸:i ×Xdµ_ÿj |;u �dL�÷j6z y bC~ w�y ` } bfz w `Yac]Bbfa��:]Bz w aN]���� w ]�b u§dMà+d µ¼¾ « »´jcjQbf� w�| ] u `V�

�LaNb �øwNy `���] } ]B�Q� |�w?| ~ y ` u ��zc] u � w�y ` �÷|�w�} aN]Bb;a |;u � w�y `RzNú;`G~^`V� } aN]Bb;aY£�aN` �z } `Y_ w ]4�f` �4¦ iY�Xbfa¤aN`G�L`Yac` u _V`;£ wNy `R��bf� u �:z�_Vb �Q} � w `V��� u ��`Va wNy ` �B| aN�;` � z |;�h}�� `| zczN� �h}:w ]�b uÇ| aN`���Ø �2�ãBøà+d µ¼¾ « »ÖjDB���Ø ¸�g:i

³ y ��zY£���`Yz } ] w `Q�;×ON u b u _Yb �h}�� ] |;u _V`>] uâwNy ` w aN` |�w�� ` u[w �facbf� }Ñ|;u �Ñ��` �z } ] w ` y�| �K] u �±¥c��z w �;�[�szN����¥c`Y_ w zY£ w�y `Hz w �:� ¦ z w aNb u � ��¦ zN� }:} bfa w z wNy `Ò_Yb u _ � � �zN]�b uxwNyX|�wů �f]��;` u�y ]Bzsz } `V_Y]4§X_ y ]�z w b;a ¦f¯ìë bK`Q~�bf� � � yX| �f`���`Y` u �©` wNw `Vasb;¨

Page 32: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

e��C¸�¹+º±»p¼R½�¾¤¿ÁÀKÂhÃÅÄ.¼¤¾�¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿�ÃÅÄ.¾¤É�½¡ÊTË�ÌHÍ�Î�ɱÏpðÉ>ÐѾ�Æ¡Æ+¾)¹)½¡ÊÒ»RÉRÏÓ¹«ÍhÎÔÉ

(c) (d) í

(a) (b) í

0.51 0.86

0 1

0 1

0 1

0 1 0.51 0.86

�¡]��f��aN`±¸:i ×:���#aN]�bfaRd | £K�÷j |�u � } bfz w `YaN]�bfaRdµ_;£2�÷j��:]Bz w aN]���� w ]�b u z¤�Lbfa | zN��� } b }K�� �B|�w ]�b u à¡d µ¼¾ j�z } `V_Y]4§X`Y�â� ¦Çw�y `{_Ybf� u[w `Yac� | _ w � |;� ®���`Va ¦ � 1°ª bf� � � ë bK` y�| �;`] �h} acb��;`Y� yX| � y ` w?| ú�` uMw�y `«�:aN����£[�;]��f` u>w�yX|0w#y `«��]�� u b w ] �Q} acbC�f`^~�] w�y bf� w] w i 3 � | a w d | j^_YbfacaN`Yz } b u ��z w b w�y ` �÷|0w±} aN]Bb;aY£�dL�÷j w b w�y `¬zcú;`G~^`V� } aN]�bfaYiw?| úK] u � w�y `Ò��ac����i : b;aN`YbC�;`YaY£ w�y `¬_Vb u _ � ��zc]Bb uÖy b � ��z��Lbfa«��b w�yÖ} ac]BbfaczYiØÖÙ{î p ß Ü rVsYárq ã ß Ü³ y ]�z´_ yX|;}:w `Ya yX| z´�:`V�f` � b } `Y�¢_ | ��z |;� �°|;u�|;��¦�w ]�_ w `Y_ y�u ]�®���`Yz´�Lbfa �Q|;uX| �f] u �b u `hb;� wNy ` �Q| ¥cbfa } acbf� � ` � z¬] u _ � ] u ]B_ |�� `G� } `YaN] � ` u[w zY� wNy ` | zNzc`YzNz � ` u[w b;�w ac` |�wN� ` u[w `Výh_ | _ ¦ ] usw�y `�� | _Y`«b;�÷] �Q} `YaÅ�L`Y_ w _Yb �Q}�� ] |;u _Y`;i` #z w ] �Q|�w `Vz+� | zc`Y�zNb � ` �4¦ b u ] u[w ` u[wc� w b � w aN` |�w�|;uX|;�4¦ zc]Bz ¯¢| z+~�` ���©| z wNy bfzN`)� | zc`Y�Qb u ] u z w aN� � ` u2�w?|�� � | ac] | � � `H�Lbfa � � �¶| z ¯ _ |�u ��` � ]Bz � ` | �:] u �h] uÇw�yX|�w±w�y ` ¦ �Q|ÿ¦²� ]B`Ò` u[w ]BaN` ��¦bf� w zN]���` w�y ` w�y `Ybfac` w ]�_ |;� ��bf� u ��zYi�³ y `¬�Lb;a � � �B| z«`Yz w?| � � ]Bz y `Y�Ç] u´wNy ]Bz�_ yX|;}K�w `Va } acbC�2]���`R] u z w aN� � ` u[wÅ� ] u ��` } ` u �:` u[w �f� | a |;u[w `Y`Yz��Lbfa } b � ]B_ ¦M|;uX|;�4¦ zc]Bz |;u � £] ux| ����] w ]�b u £¡z y bf� � ��` u�| � � ` |;uX|;�4¦ z w z w b �:` w `Va � ] u ` w�y `�`G� w ` u[wMw b²~ y ]B_ y`V¨©bfa w z w b´` u �Lbfac_Y`>_Vb �h}�� ] |;u _Y` �6|ÿ¦ ] u _YaN` | zN`MbC�;`Ya |��B��w aN` |�w�� ` u[w `G¨T`V_ w ]��f` �u `VzNzYi

³ y `)] �h} bfa w?|;u _Y`�b;�©] u ��]Bac`Y_ w `J� } `VaN] � ` u[w�|�w ]�b u ]Bz u b w _Vb u § u `Y� w b{z w ��� �]B`Vz>] u �;b � �K] u � y � �6|;u zc���K¥c`V_ w zVi& ¡� } `Yac] � ` u[w?|�� _Yb u ��] w ]Bb u z>`Y®���]4� |�� ` u[w>w bw�y bfzN`�b;��] �Q} `YaÅ�L`Y_ w _Vb �Q}:� ] |�u _Y`¤bK_Y_Y�:aÔ~ y ` u `V�;`Ya wNy `�� | ac] | � � `+~ y bfzc`^_ | ��z |��`V¨©`Y_ w ~^`Özc`Y`Yú w b | zczN`YzczQ_ |;u�u b w �©` �Q|;u ] } � �B|�w `V�x��]�aN`Y_ w��4¦�¦ ` w _Ybf� � ����`}X| a w ] |;�B�4¦ ] u�� �:` u _V`Y�¢� ¦ ] u �:]BaN`V_ w´� ` |�u zYi�³ ¦K} ]�_ |;�H|�}�}�� ]�_ |�w ]Bb u z6] u �;b � �;`w�y `â��] | � u bfzN]�z�b;�¬b u �;bf] u � } aNbK_Y`VzNzN`VzÇ�Lb;a6~ y ]B_ y wNy `âzNbf�:aN_Y` b;� �Q|;� �L� u _ �w ]�b u ] u �H�©` yX| �K]Bbfa � �:z w �©`«]���` u[w ]�§X`V�Q��zc] u �Ò] u ��]Bac`Y_ w¤� ` |;u z#��`Y_ | ��zN`)��]Bac`Y_ w

Page 33: B | uQ|¬} | }X| }X| w uMw ybayes.cs.ucla.edu/BOOK-99/ch8.pdf · 2000-03-14 · f¸f¸ ¹+º±»p¼R½ ¾¤¿ÁÀKÂhÃÅÄ.¼¤¾ ¿)Æ+¾)¹)½Ç¾+ÈH¼¤¾¤¿ ÃÅÄ.¾¤É ½¡ÊTË

ÀKÂðï[ ¹«Í«És¹il±Î�Ê2ÃGÍ«É e��ÿ×�Q|;u ] } � �¶|�w ]Bb u b;��zc��z } `Y_ w `Y�szNbf��ac_Y`Vz¡]BzÝ`V] w�y `Ya }�y[¦ zc]B_ |;�B�4¦ ] �h} bfzNzc]B� � `#bfa } aNb �y ]���] w ]��f` ��¦ `J� } ` u zN]4�f`;i²  u `J� |;�Q}:� `hb;� wNy ` �B|�wNw `YaÒ~^bf� � �x��`6] u[w `YacaN� }2w ] u �w�y ` u bfa �Q|;� b } `Ya |0w ]Bb u b;� |p} acb2���:_ w ]�b uM� ] u `^zcb | z w b | _ y ]B`G�f`���]�aN`Y_ w _Yb u�w aNb �bC�f`Va |�}�y[¦ zN]B_ |;�Ô}X| a |�� ` w `Ya wNyX|�w ]Bz�zc��z } `Y_ w `Y�âb�� �Q|;� �L� u _ w ]Bb u ] u ��iY� | a w ] |;�_Yb u�w aNb � bC�f`Va wNyX|�w�}X| a |;� ` w `VaY£K] u�w�y `��Lbfa � b;��] u ��]Bac`Y_ w ] u�� ��` u _Y`�£K~�bf� � �6��`� ��_ yÇ� bfaN`H_Vb u �f` u ]B` u[wR|;u �Ö~^b;� � � |;�B� bC~ wNy ` } aNbK����_ w ]�b u�w bh_Yb u[w ] u ��`;i: ` wNy bK��b � b;�f]B_ |;�B�4¦ £ w�y ` � `YzNz | �f`Mb;� w�y ]BzH_ yX|;}2w `Ya yX| zH�©`V` u w bÖ��` � b u2�z w a |�w ` w�yX|0w £Ý`V�;` u ] u _ | zc`YzH~ y `VaN`�_ | �:z |;� ®�� |;u[w ] w ]�`Yz | ac` u b w ]���` u[w ]4§ | � � `;£aN` | zNb uX| � � ` | zNzN� �Q}:w ]Bb u z | ��bf� wÒwNy `Qz |;� ]�` u[w aN` �¶|�w ]Bb u z y ] } zH] u w�y `h��b �Q| ] u_ |�u �©` y�| a u `VzNzc`Y� w b ¦ ]B` � �&�:zN`V�L� � ®�� |;u[w ] w?|�w ]��;` ] u �Lb;a �Q|�w ]�b u�| �©b;� w�w�y `_ | ��z |;� �LbfaN_V`Yz w�yX|0w b } `Va |�w `>] uâwNy `M��b �Q| ] u i�$ u _V`�zN��_ y�| zNzc� �h}:w ]�b u z | ac`| a w ]B_V� �¶|0w `Y�M] u �fa |;}:y ]B_ |;� �Lbfa � £ wNy ` ¦ _ |;u ` | zN] ��¦ �©`�zN�:� � ] wNw `Y� w b |;� �;`Y��a | ]B_� ` w�y b2��z w�yX|0w{¦ ]B` � � w�y `Q��`Yzc]BaN`V����bf� u ��zsbfaY£ |;��w `Ya u�|�w ]4�f` �4¦ £Ô] u �K] w `��H]B���:zz |��Q}�� ] u � w `Y_ y�u ]�®���` w b)� | _Y] � ] w?|�w `¤g |ÿ¦ `Yzc] |;u `Yz w ] �Q|�w ]�b u b;� wNy `^_ | ��z |�� ®�� |;uK�w ] w ]�`Yz�b;�¡] u[w `VaN`Vz w i

� rDñ Ü ßãòÓsYnpà o � n Ü´Þ³ y ` �6| ] u aN`VzN� �4w z ] u wNy ]Bzâ_ yX|�}:w `Va | aN`·� | zc`Y��b u _Vb �B�B| ��bfa |�w ]��;`x~^b;aNú�z~�] wNy   � `J�Çg |;� ú;` |;u �Çþ | �K]B�²\ y ]�_?ú;`Yac] u �:i�³ y `{` u _Ybf�:a | �f` � ` u[w z)b;� ëf|�� `Vz� bf��] u z |;u � : ] � `Vz �±b ���¶|;u ��`Ya y ` ��} `V�¢�:z´` u ���:aN` |�� b u � |;u � | �;b u ]B�V] u �} ��� � ]B_ |�w ]�b u } aNbK_Y`VzNzVi´³ y `Yzc`�] u �f`Vz w ]B� |�w ]Bb u z y�| �;`6��` u `G§ w `V�x�LaNb � ��]Bzc_Y��z �zN]�b u z�~�] wNy � y ] � þ | ~�]�� £ ëf|;� `Vz¬�±`Y_?ú �Q|;u £��Ò��]���b tv� �©` u zY£f� w `G�f` ëf| _Ybf�:zN` u £� w `V¨©` u � | ��aN] w �Y` u.|�u �²\ yX| a � `Yz :.|;u zNú�]�i