A backtrack data structure and algorithm by Donald Knuth (wim
bohm cs.colostate.edu) Except as otherwise noted, the content of
this presentation is licensed under the Creative Commons
Attribution 2.5 license. Slide 2 Dancing Links Doubly linked list
nodes have references L to left node L and R to right node x Slide
3 Dancing Links Doubly linked list Remove x R[L[x]]=R[x]
L[R[x]]=L[x] notice that x still, via its L and R, points at left
and right nodes, and thus we can easily... x x Slide 4 Dancing
Links Doubly linked list Remove x R[L[x]]=R[x] L[R[x]]=L[x] Put x
back R[L[x]]=x L[R[x]]=x x x x Slide 5 Using Dancing Links O(1) put
x back operation Works in following BackTrack scenario. we have
created a space to be searched as a global doubly linked data
structure. we search this space by DFS, selecting: taking out
certain options and putting them back in reverse order What does
not work. adding completely new options. putting options back in
other than reverse order Slide 6 Why reverse order? x y xy remove x
Slide 7 Why reverse order? x y xy remove x xy remove y Slide 8 What
happens if x is put back first? x y xy remove x xy remove y Slide 9
What happens if x is put back first? xy remove y xy put back x x
points at y and y at x, but y is not in the chain!! Slide 10 Exact
sub-set / cover problem Given a matrix of 0-s and 1-s, find a
subset of rows with exactly one 1 in each column of A Backtrack
approach Pick a column c, pick a row r with 1 in c, remove columns
c and j with 1 in r and rows with a 1 coinciding with 1-s in r a b
c d e f g reducing the problem 1 0 0 1 0 1 1 0 eg c in row 1 2 1 0
0 1 0 0 1 selects subset {c,e,f} 3 0 1 1 0 0 1 0 so remove columns
c,e,f 4 1 0 0 1 0 0 0 and row 1 and row 3 5 0 1 0 0 0 0 1 because
it overlaps with row 1 Slide 11 Exact sub-set / cover problem
Backtrack approach Pick column c, row 1 subset {c,e,f} which rows
and cols disappear? a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3
0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 Slide 12 Exact
sub-set / cover problem Backtrack approach Pick column c, row 1
{c,e,f} rows 1,3 and cols c,e,f disappear Pick column a, row 2
{a,d,g} which rows, cols disappear? a b d g 2 1 0 1 1 4 1 0 1 0 5 0
1 0 1 Slide 13 Exact sub-set / cover problem Backtrack approach
Pick column c, row 1 {c,e,f} Pick column a, row 2 {a,d,g} all
rows/cols disappear what is the problem? Slide 14 Exact sub-set /
cover problem Backtrack approach Pick column c, row 1 {c,e,f} Pick
column a, row 2 {a,d,g} all rows/cols disappear b is not covered,
so backtrack Slide 15 Exact sub-set / cover problem Backtrack Pick
column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick
column a, row 4 {a,d} which rows, cols disappear now? a b d g 2 1 0
1 1 4 1 0 1 0 5 0 1 0 1 Slide 16 Exact sub-set / cover problem
Backtrack Pick column c, row 1 {c,e,f} rows 1,3 and cols c,e,f
disappear Pick column a, row 4 {a,d} rows 2,4 cols a,d disappear b
g 5 1 1 Slide 17 Exact sub-set / cover problem Backtrack Pick
column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick
column a, row 4 {a,d} rows 2,4 cols a,d Pick column b, row 5 {b,g}
row 5 disappears Exact cover accomplished! matrix is empty Slide 18
Exact sub-set / cover problem Subsets / rows 1,4,5 provide exact
cover! a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1
0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 Slide 19 Exact set cover The
columns: elements of a universe The rows: subsets of elements in
the universe Find a set of subsets that has each element exactly
once Union of the set is the Universe Intersection of any two
subsets is empty Finding an exact cover is tough NP-Complete e.g.
when each subset has three elements Great candidate for backtrack
search Representation: row column doubly linked sparse matrix only
containing 1-s Use dancing links to remove and put back candidates
Slide 20 Pentominoes A pentomino is a size 5 n-omino composed of n
congruent squares connected orthogonally by side (not point wise)
There are 12 different pentominoes when rotation, and mirroring are
allowed There are 18 pentominoes when only rotation is allowed
Slide 21 The 12 pentominoes F I L N P T U V W X Y Z Slide 22 Early
implementation Dana Scott wrote a backtrack program in 1958 for the
Maniac (4000 instructions / sec) tiling a chessboard with a 2x2
hole in the middle with the 12 pentominoes, using each pentomino
exactly once The code ran in ~3.5 hours: (50 million instructions)
Slide 23 One of the 65 solutions Slide 24 Board shapes Tiling
different board shapes Chessboard with 2x2 hole in middle or with 4
holes in arbitrary places Rectangles 6 x 10 5 x 12 4 x 15 3 x 20 3D
boxes Slide 25 Matrix with 72 columns 12 pentominoes and 60
positions in the boards grid Each row has 6 1-s 1 for the
pentomino, 5 for its positions Each row is a description of a
pentomino in a certain position There are 1568 such rows Pentomino
problems are exact cover problems Slide 26 Translating puzzle to
set cover Pentominoes matrix: 72 columns 1568 rows Lets do a
simpler puzzle 4 triominoes I: L: : -: No rotation or flip
Rectangle 3 x 4 # possible solutions? Slide 27 Translating puzzle
to set cover Pentominoes matrix: 72 columns 1568 rows Lets do a
simpler puzzle 4 triominoes I: L: : -: No rotation or flip
Rectangle 3 x 4 Possible solution: Slide 28 Setcover matrix for
simple puzzle 16 columns: 4 columns for the pieces 12 columns for
the positions Rows: 4 I placements I L - (1,1) (1,2) (1,3) (1,4)
(2,1) (2,2) (2,3) (2,4) (3,1) (3,2) (3,3) (3,4) 1 0 0 0 1 0 0 0 1 0
0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 1 0
0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 Slide 29 Setcover matrix
cont Rows 6 L placements I L - (1,1) (1,2) (1,3) (1,4) (2,1) (2,2)
(2,3) (2,4) (3,1) (3,2) (3,3) (3,4) 0 1 0 0 1 0 0 0 1 1 0 0 0 0 0 0
0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 1
0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 1 0 0
0 0 0 0 0 0 1 0 0 0 1 1 Slide 30 Setcover matrix cont 6 placements
6 placements Slide 31 Backtrack algorithm X sketch // A is the
exact cover matrix if (empty(A)) return solution else { choose a
column c choose a row r with A r,c = 1 include r in solution for
each column j with A r,j = 1 { delete column j from A for each row
i with A i,j = 1 delete row i from A } repeat recursively on
reduced A } Slide 32 algorithm X The subsets form a search tree
Root: original problem Level k node: k subsets have been chosen and
all columns (elements) in the subsets and all overlapping subsets
(rows) have been removed Any systematic rule for choosing columns
will find all solutions Slide 33 Choosing c One way Pick
pentominoes in alphabetical order F I L N P T U V W X Y Z Not good:
F has 192 possible places, I then has ~32, very big search space
~2x2 12 Better ways Choose lexicographically first uncovered
position, starting with (1,1), search space ~10 7 Scott realized
that X has essentially three possible places centered at (2,3),
(2,4) and (3,3). The rest leads to symmetrical solutions. Search
space ~350,000 Knuth has a more general solution: pick column with
minimal number of 1-s Slide 34 Lets dance Represent each 1 in A by
a node with 5 links L(x), R(x) (left right) U(x), D(x) (up down)
C(x) column header Each row is a doubly (L,R) linked circular list
Each column is a doubly linked (U,D) circular list Each column has
a column header node c with additional name N(c) and size S(c). The
column headers form a circular row with a header h Each node points
at its column header with link C Slide 35 Our example a b c d e f g
1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0
1 0 0 0 0 1 h a 2 b 2 c 2 d 2 e 1 f 2 g 2 Slide 36 Search algorithm
DLX search(int k){ // search is called with k=0 from outside
if(R[h]=h) print O else { // O is the set of currently picked rows
choose column c cover column c // pick element c for each row r =
D[c], D[D[c]] while r!=c { O[k]=r // pick a row with element c for
each j = R[r], R[R[r]] while j!=r cover column C[j] // all elements
in the row are now covered search(k+1) for each j = L[r], L[L[r]]
while j!=r uncover column C[j] // uncover in reverse order }
uncover column c and return; } Slide 37 cover column c // remove c
from headers L[R[c]]= L[c]; R[L[c]]=R[c] // remove all rows in cs
column for each row r = D[c], D[D[c]] while r!=c for each j = R[r],
R[R[r]] while j!=r { U[D[j]]=U[j]; D[U[j]]=D[j]; S[C[j]]--; } Slide
38 uncover c // put rows back IN REVERSE ORDER // last out first
back in for each row r = U[c], U[U[c]] while r!=c for each j =
L[r], L[L[r]] while j!=r { S[C[i]]++; U[D[j]]=j; D[U[j]]=j; } //
put header back L[R[c]]=c; R[L[c]]=c; Slide 39 cover a: remove
header h a 2 b 2 c 2 d 2 e 1 f 2 g 2 a b c d e f g 1 0 0 1 0 1 1 0
2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1
Slide 40 cover a: remove row 2 h a 2 b 2 c 2 d 1 e 1 f 2 g 1 b c d
e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0 4 0 0 1 0 0 0 5 1 0 0 0 0 1 Slide
41 cover a; remove row 2 and 4 h a 2 b 2 c 2 d 0 e 1 f 2 g 1 b c d
e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0 5 1 0 0 0 0 1 Notice the
asymmetry in D2 (element D in row 2) versus D4 (element D in row
4). Putting back D2 first would change D4s Links, but D4 is not in
the set! This is why we need to uncover in exactly the reverse
order, so we know that the element that is put back refers to
elements in the set Slide 42 cover a; last step: remove row 5 b c d
e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0 Slide 43 Picking subset 2 leads
to failure a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0
0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 select a, row 2 cover a,d,g
remove rows 2,4,5 b c e f 1 0 1 1 1 3 1 1 0 1 select b, row 3 cover
b,c,f remove rows 1,3 nothing left for e Slide 44 Picking subset 2
leads to failure a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0
1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 select a, row 2 cover
a,d,g b c e f 1 0 1 1 1 3 1 1 0 1 select c, row 1 remoce rows 1,3
now what? Slide 45 Picking subset 2 leads to failure a b c d e f g
1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0
1 0 0 0 0 1 select a, row 2 cover a,d,g b c e f 1 0 1 1 1 3 1 1 0 1
select c, row 1 cover c,e,f nothing left for b Slide 46 Picking
subset 4 succeeds a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0
1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 select a pick subset 4
cover a,d remove rows 2,4 b c e f g 1 0 1 1 1 0 3 1 1 0 1 0 5 1 0 0
0 1 select b picking subset 3 remove rows 1,3,5 fails again pick
subset 5 cover b,g remove rows 3,5 c e f 1 1 1 1 select c pick
subset 1 empties A Exact cover: subsets 4 {a,d}, 5 {b,g} and 1
{c,e,f}
