Dynamic Programming Tutorial

Elaine ChewQMUL: ELE021/ELED021/ELEM021

26 March 2012

Sources• Hillier, F. S. & G. J. Lieberman. Introduction to Operations

Research. Chapter 11: Dynamic Programming, 7th, 8th, 9th editions, McGraw-Hill.

• Dixon, S. & Widmer, G. (2005). MATCH: A Music Alignment Tool Chest. In Proceedings of the 6th Intl Conf on Music Information Retrieval, London, UK, 492-497.

• Yang, A., E. Chew & A. Volk (2005). A Dynamic Programming Approach to Adaptive Tatum Assignment for Rhythm Transcription. In Proceedings of the 1st IEEE Intl Wkshop on Multimedia Information Processing and, Irvine, CA, 577-584.

Sources• Hillier, F. S. & G. J. Lieberman. Introduction to Operations

Research. Chapter 11: Dynamic Programming, 7th, 8th, 9th editions, McGraw-Hill.

• Dixon, S. & Widmer, G. (2005). MATCH: A Music Alignment Tool Chest. In Proceedings of the 6th Intl Conf on Music Information Retrieval, London, UK, 492-497.

• Yang, A., E. Chew & A. Volk (2005). A Dynamic Programming Approach to Adaptive Tatum Assignment for Rhythm Transcription. In Proceedings of the 1st IEEE Intl Wkshop on Multimedia Information Processing and, Irvine, CA, 577-584.

Shortest Path Problem• V = set of vertices (or nodes)• E = set of edges (or arcs), including s (source) and t (sink)• C = [cij] = set of arc costs• X = indicator variable for whether arc ij is used

• Linear Programming Problem Formulation:

Min Σij cijxij

s.t. Σi xij = 1Σj xij = 10 ≤ xij ≤ 1

Hogwarts (Shortest Path) Example

• Determine which route from entrance to exit has the shortest distance

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

Gringott’sWizarding

Bank

Entrance to Wizarding

World

QualityQuidditchSuppliesOllivander’s

Exit toCharing Cross

Road

The Leaky Cauldron

Magical Menagerie

Algorithm

• At iteration i– Objective: find the i-th nearest node to the origin– Input: i-1 nearest node to the origin, shortest path

and distance– Candidates for i-th nearest node: nearest unsolved

node to one solved– Calculation for i-th nearest node: for each new

candidate, add new edge to previously found shortest path to the i-1-th nearest node. Among the new candidates, the node with shortest distance becomes the i-th nearest node.

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

0

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 Q

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 R

2 QR

CC

67

C 6 Q

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

6

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 R

2 QR

CC

67

C 6 Q

3 QC

OO

99

O 9 QC

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

6

9

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 R

2 QR

CC

67

C 6 Q

3 QC

OO

99

O 9 QC

4 QCO

GMM

121010

M 10 CO

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

6

9

10

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 R

2 QR

CC

67

C 6 Q

3 QC

OO

99

O 9 QC

4 QCO

GMM

121010

M 10 CO

5 QOM

GGE

121114

G 11 O

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

6

9

10

11

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

1 R Q 5 Q 5 R

2 QR

CC

67

C 6 Q

3 QC

OO

99

O 9 QC

4 QCO

GMM

121010

M 10 CO

5 QOM

GGE

121114

G 11 O

ImplementationIteration, i Solved nodes

connected to unsolved

Closest connected unsolved node

Total distance involved

i-th nearest node

Minimum distance

Last connection

5 QOM

GGE

121114

G 11 O

6 OMG

EEE

141413

E 13 G

Hogwarts (Shortest Path) Example

7

4

22

5

14

4

5

1 7

G

OE Q

R

M C

3

5

0

6

9

10

11

13

Sources• Hillier, F. S. & G. J. Lieberman. Introduction to Operations

Research. Chapter 11: Dynamic Programming, 7th, 8th, 9th editions, McGraw-Hill.

• Dixon, S. & Widmer, G. (2005). MATCH: A Music Alignment Tool Chest. In Proceedings of the 6th Intl Conf on Music Information Retrieval, London, UK, 492-497.

• Yang, A., E. Chew & A. Volk (2005). A Dynamic Programming Approach to Adaptive Tatum Assignment for Rhythm Transcription. In Proceedings of the 1st IEEE Intl Wkshop on Multimedia Information Processing and, Irvine, CA, 577-584.

Dynamic Time Warping

Euclidean Distance

(i,j)

Sources• Hillier, F. S. & G. J. Lieberman. Introduction to Operations

Research. Chapter 11: Dynamic Programming, 7th, 8th, 9th editions, McGraw-Hill.

• Dixon, S. & Widmer, G. (2005). MATCH: A Music Alignment Tool Chest. In Proceedings of the 6th Intl Conf on Music Information Retrieval, London, UK, 492-497.

• Yang, A., E. Chew & A. Volk (2005). A Dynamic Programming Approach to Adaptive Tatum Assignment for Rhythm Transcription. In Proceedings of the 1st IEEE Intl Wkshop on Multimedia Information Processing and, Irvine, CA, 577-584.

Tatum Segmentation

• Tatum = smallest perceptual time unit in music

Example Results

40

60

80

100

120

140

0 100 200 300 400

Beats

Tatum

Coarse Manual

Segmentation

bepthq3

son08_3

Segmentation

• Example of tatum selection• Input:

– Note onset times, O(1,n) = {O1, O2,…, On}

• Output:– Segmentation points, S = {S1, S2,…,Sm}– Optimal tatum in each segment

Remainder Squared Error

o(1) … o(j)

p

1 … j j+1

p

ERR(.)

The error incurred by a tatum assignment, p, for onsets O(i+ 1,k) is given by the remainder squared error (RSE) function:

where oj (= Oj+1 – Oj) is the inter onset interval (IOI) between onsets j and j+1.

€

e(p,i + 1,k) =ojp

−ojp

+ 12

⎢ ⎣ ⎢

⎥ ⎦ ⎥

⎛

⎝ ⎜

⎞

⎠ ⎟

j =1

k − i −1

∑2

,

€

ERR(O(i + 1,k)) =p

min e(p,i + 1,k).

Optimal Segmentation

• Define OPT(k) to be the best segmentation (cost) for a given set of onsets O(1,k):

where ERR(.) returns the error incurred by the best tatum assignment for the set of onsets O(i+1,k).€

OPT(k) =1≤ i ≤kmin {OPT(i)+ ERR(O(i + 1,k))},

ki i+1 …1 …