Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 8: April 20th, 2006 EE512 - Graphical Models - J. Bilmes Page 1

University of WashingtonDepartment of Electrical Engineering

EE512 Spring, 2006 Graphical Models

Jeff A. Bilmes <[email protected]>Jeff A. Bilmes <[email protected]>

Lecture 8 Slides

April 20th, 2006


• READING: – Chapter 11,12,15 in Jordan’s book

• Reminder: TA discussions and office hours:– Office hours: Thursdays 3:30-4:30, Sieg Ground Floor

Tutorial Center– Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor

Tutorial Center Lecture Room

• Reminder: take-home Midterm: May 5th-8th, you must work alone on this.

Announcements


• L1: Tues, 3/28: Overview, GMs, Intro BNs.• L2: Thur, 3/30: semantics of BNs + UGMs• L3: Tues, 4/4: elimination, probs, chordal I• L4: Thur, 4/6: chrdal, sep, decomp, elim• L5: Tue, 4/11: chdl/elim, mcs, triang, ci props.• L6: Thur, 4/13: MST,CI axioms, Markov prps.• L7: Tues, 4/18: Mobius, HC-thm, (F)=(G)• L8: Thur, 4/20: phylogenetic trees, HMMs• L9: Tue, 4/25• L10: Thur, 4/27

• L11: Tues, 5/2• L12: Thur, 5/4• L13: Tues, 5/9• L14: Thur, 5/11• L15: Tue, 5/16• L16: Thur, 5/18• L17: Tues, 5/23• L18: Thur, 5/25• L19: Tue, 5/30• L20: Thur, 6/1: final presentations

Class Road Map


• L1: Tues, 3/28: • L2: Thur, 3/30:• L3: Tues, 4/4: • L4: Thur, 4/6:• L5: Tue, 4/11:• L6: Thur, 4/13:• L7: Tues, 4/18:• L8: Thur, 4/20: Team Lists, short abstracts I• L9: Tue, 4/25:• L10: Thur, 4/27: short abstracts II

• L11: Tues, 5/2• L12: Thur, 5/4: abstract II + progress• L13: Tues, 5/9• L14: Thur, 5/11: 1 page progress report• L15: Tue, 5/16• L16: Thur, 5/18: 1 page progress report• L17: Tues, 5/23• L18: Thur, 5/25: 1 page progress report• L19: Tue, 5/30• L20: Thur, 6/1: final presentations

• L21: Tue, 6/6 4-page papers due (like a conference paper).

Final Project Milestone Due Dates

• Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only).

• Final reports must be turned in electronically in PDF (no other formats accepted).

• Progress reports must report who did what so far!!


• Factorization property on MRF, (F)• When (F) = (G) = (L) = (P)

• inclusion-exclusion

• Möbius Inversion lemma

• Hammersley/Clifford theorem, when (G) => (F)

• Factorization and decomposability• Factorization and junction tree• Directed factorization (DF), and (G)• Markov blanket• Bayesian networks, moralization, and ancestral sets

Summary of Last Time


• d-Separation, (DL), (DO), and equivalence of all Markov properties on BNs.

• Phylogenetic Trees and Chordal Models• Mixture Models• Hidden Markov Models (HMMs)• Forward () recursion and elimination• Backwards () recursion and elimination

Outline of Today’s Lecture


Books and Sources for Today

• M. Jordan: Chapters 11,12,15.• Lauritzen, chapter 3.• J. Pearl, Probabilistic Reasoning in Intelligent Systems:

Networks of Plausible Inference, 1988.• T. McKee “Topics in Intersection Graph Theory”


Preservation of (DF) in ancestral sets


Example (DF) – (G)


Example (DF) – (G)


d-Separation revisited


d-Separation revisited


All Together Now


What else can chordal graphs do?


Phylogenetic Tree: example

species

characters resultingphylogenetic

tree


Perfect Phylogeny


Examples


Phylogenetic Trees


Examples: GT, GI


Phylogenetic Trees


Phylogenetic Trees


Intersection Graphs


Intersection Graphs, Chordality, Phylogeny


Summary

• But triangulated graphs (really ``trees'') have many other properties as well.

• We are interested in them since they are exactly the class of models on which we can perform exact inference, which is the topic we will next spend some time on.

• Next topic: Morphing from mixture models to HMMs


Mixture Models


Mixture Models


Inference on Hidden Markov Models

• Hidden Markov Models (HMMs) are a ubiquitously used model in speech recognition, natural language processing, bioinformatics, financial markets, and many time-series problems.

• HMMs are rich enough to be interesting, but simple enough so that they are a perfect example to start with when performing exact inference.

• HMMs can be described either with a BN or an MRF– so this means they must be decomposable

• Since HMMs are already triangulated (after moralization if necessary), there is no triangulation step.

• Moreover, since the clique sizes are small, HMMs are easy to deal with (compexity only O(TN2)


HMMs and Bayesian Networks


HMMs and Markov Random Fields


HMMs


HMMs


HMMs, elimination orders, and forward recursion


HMMs, elimination orders, and forward recursion


HMMs, elimination orders, and backward recursion


HMMs, elimination orders, and backward recursion

Documents

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,