Path Consistency for 1- General CSPs 2- STPs

Peter SchletteWesley BothamCSCE990 Advanced CP, Fall 2009

Outline

04/22/23

2

General CSPs Review of Path Consistency & PC

Algorithms Path Consistency Algorithms

PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs

Review of Triangulated Graphs Path Consistency on STPs

Floyd-Warshall, Bellman-Ford, STP, P3C, Prop-STP

Path Consistency: Properties

04/22/23

3

A CSP is path consistent iff it is strongly 3-consistent [van Beek & Dechter, JACM95] Domains are filtered by arc consistency Consistent solutions over 2 variables can be

extended to every 3rd variable PC algorithms typically iterate over triplets of

variables End variables in triplets need not be distinct In STP, variables domains are not relevant, thus PC

algorithms on STPs enforce only 3-consistency A given algorithm

Must determine that a CSP is path consistent or not May or may not filter the constraints as much as

possible (e.g., DPC)

List of Algorithms Discussed

PC-1 [Mackworth 77, , Dechter Fig 3.10]

PC-2 [Mackworth 77, Dechter Fig 3.11]

DPC [Dechter & Pearl 89, Dechter Fig. 4.9]

PPC [Bliek & Sam-Haroud 99]

PC-8 [Chmeiss & Jégou 98]

PC-2001 [Bessière et al. 05]

04/22/23

4

Path Consistency: Algorithms

04/22/23

5

They all stop when a relation/domain is empty; omitted for clarity

Queue Does is it have one? Edges (e.g., PPC) Triplets of variables (e.g., PC-2) Tuples of ‘vv-pair, variable’ (e.g., PC-8 & PC-2001)

Properties Determines strong 3-consistency? What is the time and space complexity?

Requires additional data structures to remember supports? Graph: Complete? Chordal?

What is the practical performance (i.e., phase transition)?

PC-1 [Mackworth 77]

04/22/23

6

1 Repeat until quiescence2 For i,j,k variables3 Rij Rij Rik Rkj

Has 4 nested loops, iterates over vertices, needs no queue

Updates every edge & every domain (i=j) Uses composition and intersection Determines strong 3-consistency (when i=j) Time complexity is O(n5d5)

One sweep costs O(n3d3) Number of sweeps O(n2d2)

Space: no queue, no additional data structure, complete graph

PC-2 [Mackworth 77]

04/22/23

7

Has 1 loop over a queue of triplets of variables When an edge (domain) is updated, only triplets with an ‘external’ 3rd

node are added to queue Allows i=j, thus determines strong 3-consistency.

Dechter Fig 3.11 specifies i<j in which case domains are not updated Theoretically & practically faster than PC-1 (queue) Time complexity is O(n3d5) Space: Queue size is O(n3), no additional data structures,

complete graph

1 Q (i,j,k) | i j, i k, j k 2 While Q is not empty3 For (i,j,k) from Q4 Rij Rij Rik Rjk

5 If Rij changed, Q Q U (m,i,j), (m,j,i) | m i, m j

mm jik

n=16, a=16, d=30%

PC-1 vs. PC-2[Botham & Schlette]

04/22/23

8

PC-2

PC-1

PPC [Bliek & Sam-Haroud 99]

04/22/23

9

Q ← E

Until Q is empty do

edge ← DEQUEUE(Q)

for every triplet i,j,k related to edge

Rij ← Rij ∩ (Rik Rkj)

if Rij was changed then EnQueue((i,j), Q)

First triangulates the graph Keeps Q, a queue of edges For an edge in Q

Pops edge from Q, retrieves all triplets where edge appears In each triplet, updates each edge Each updated edge is added to Q Does not specify whether or not domains are filtered (some

should be for soundness)

PPC [Bliek & Sam-Haroud 99]

04/22/23

10

Triangulated graph is usually sparser than complete graph

For triplet (i,j,k) allows i=j, algorithm is sound (not clear in paper)

Enforces Strong path-consistency Weaker filtering than PC-2

Time: O(ed2), degree of graph Space: queue O(e) for storing triplets, no additional data

structure, chordal graph Weakness: if 2 or more edges of a given triplet are in Q

All three edges are updated once for each edge May do redundant work (fixed in STP)

i

vn

j

v2

v1

1 For k=n downto 1do2 For i =1 to k REVISE(i, Rik)3 For i,jk i,k connected & j,k connected4 Rij Rij Rik Rjk

k-2k

k-4

k-6

DPC [Dechter Fig 4.9, Dechter & Pearl 89]

Given an ordering for the variables From bottom to top, enforces directional arc-consistency (DAC) From bottom to top, for every variable, updates the edge

between every two of its parents Properties

Moralizes the graph, determines strong directional path consistency relative to ordering

Time O(min(t.d3,n3d3)) Space: No queue, no additional data structures, chordal graph

04/22/23

11

DPC: Constraint revision [Dechter Fig 3.9]

1 For each (a,b) Rij

2 If no c Dk is s.t. (a,c)Rik & (a,b)Rjk

3 Remove (a,b) from Rij

04/22/23

12

k

j

i

Does not operate on matrices in reality (,)

Iterates over Tuples in constraints (i.e., (a,b) Rij)

and Values in domain (i.e., c Dk)

Not yet tested against PC-1, PC-2, PPC

PC-8 [Chmeiss & Jégou 98]

PC-8

INITIALIZE

While Q do

POP((i,a),k), Q)

PROPAGATE((i,a),k)UPDATE

if WITHOUTSUPPORT((i,a),(j,b),k)

REMOVE (a,b) from Rij, (b,a) from Rij

Q ← Q U ((i,a,)j),((j,b),i)

04/22/23

13

INITIALIZEQ ← for i,j,k=1 to n (i<j,k≠i,k≠j) for (a,b)Rij

UPDATE((i,a),(j,b),k)

PROPAGATE for j=1 to n (j≠i,j≠k) for bDj and (a,b)Rij

UPDATE((i,a),(j,b),k)

PC-8 Analysis[Chmeiss & Jégou 98]

Queue: a list of (vvp, var) = ((var,val),var) Determines strong PC-property (if you allow i=j) Achieves ‘full’ filtering Time complexity

INITIALIZATION: O(n3d3) PROPAGATE is called O(n2d2) times, each call costs

O(nd2) PC-8: INITIALIZATION + (n2d2) PROPAGATE = O(n3d4)

Space Queue O(n2d), data structure Status-PC: O(n2d) Graph is complete

04/22/23

14

PC-8 vs. PC-2[Botham & Schlette]

04/22/23

15

PC-2

PC-8

n=16, a=16, d=30%

04/22/23

16

PC-2001 [Bessière+ 05]

PC-20011 INITIALIZE(Q)2 While Q 3 POP((i,a),k) from Q4 REVISEPATH((i,a),j)

INITIALIZE(Q)1 For i,j,k variables2 For each (a,b) Rij 3 If (a,b) has no support c in Dk

4 Remove (a,b) from Rij

5 Q Q U ((i,a),j), ((j,b),i)6 Else7 Last((i,a),(j,b),k) the first support of (a,b) Dk

04/22/23

17

PC-2001 [Bessière+ 05]

REVISEPATH((i,a),k,Q)1 For each j k2 For each b Dj | (a,b) Rij

3 support Last((i,a),(j,b),k)4 While support is nil or was deleted5 support next value in Dk

6 If no supports exist7 Remove (a,b) from Rij

8 Q Q U ((i,a),j), ((j,b),i) 9 Else10 Last((i,a),(j,b),k) support

Records supporting values to improve time complexity (at the cost of space overhead)

PC-2001 Analysis [Bessière+ 05] Queue: same as PC-8, list of (vvp,var) Achieves the same properties as PC-1, PC-2, PC-8 Time: O(n3d3) Space

Queue: O(n2d) Data structure: Last structure dominates O(n3d2) Graph complete

Compared to PC-8, PC-2001 Is easier to understand and implement Has lower time complexity Is faster in general in experiments Has worse space complexity

04/22/23

18

PC-2001 vs. PC-2, PC-8[Botham & Schlette]

04/22/23

19

PC-8

PC-2001

PC-2

n=16, a=16, d=30%

04/22/23

20

QueueDataS

trTime

Graph

PC-p?

Filtering Empirically

PC-1 None None O(n5d5)Comple

teYes Full

PC-2TripletsO(n3)

None O(n3d5)Comple

te Yes FullBetter than PC-1 in all cases

PPCEdgesO(e)

None O(ed2)Chorda

lYes Partial

Advantageous on sparse graphs

DPC None NoneO(min(td3,n3d3)

Chordal

No‘weak’ partial

Not evaluated yet, but likely best

PC-8(vvp,var

)* O(n2d)

StatusO(n2d)

O(n3d4)Comple

teYes Full

Significantly better than PC-2 around phase transition, ambiguous otherwise

PC-2001

(vvp,var)*

O(n2d)

LastO(n3d3)

O(n3d3)Comple

teYes Full

Better than PC-8 in nearly all cases

Summary of PC Algorithms for General CSPs

Comparisons: CPU time, #CC for preprocessing

Outline

04/22/23

21

General CSPs Review of Path Consistency & PC

Algorithms Path Consistency Algorithms

PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs

Review of Triangulated Graphs Path Consistency on STPs

Floyd-Warshall, Bellman-Ford, STP, P3C, Prop-STP

Triangulated Graphs: Motivation [Bliek & Sam-Haroud 99] showed that PPC

Operates on triangulated graphs Determines the property of strong path consistency When constraints are convex, PPC also yields minimal CSP

[Xu & Choueiry 03] studied STP In STP constraints are convex Proposed STP, which adapts PPC to STPs w/o updating

domains (‘weak’ path consistency) May do fewer updates than PPC: queue of edges versus queue

of triangles

[Planken et al. 08] studied STP Showed that STP is O(t2), t is the number of triangles Proposed P3C, for STP, that is O(t)

04/22/23

22

Vertex elimination

Vertex elimination operation: When removing a vertex, connect all neighbors if they are not already connected

Fill edges: are the edges added when eliminating a vertex

Simplicial vertex Vertex whose neighbors are all connected (form a

clique) Eliminating a simplicial vertex does not add any edges

Perfect elimination ordering: There is always a simplicial vertex to be eliminated. All nodes can be eliminated w/o adding any fill edges

04/22/23

23

A graph is triangulated iff it has a perfect elimination order

The width of the triangulated graph is equal to the size of its largest clique -1. Why?

Finding the width is tractable, thus max. clique on triangulated graph is tractable (usually, NP-hard)

Using the reverse of the perfect elimination ordering of a triangulated graph yields A moralized graph The induced width of this ordering is equal to the

width of the triangulated graph, why? Moralizing an arbitrary ordering of a graph yields a

triangulated graph. Why?

Triangulated Graphs

04/22/23

24

Outline

04/22/23

25

General CSPs Review of Path Consistency & PC Algorithms Path Consistency Algorithms

PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs

Review of Triangulated Graphs Path Consistency on STPs

Floyd-Warshall, Bellman-Ford, STP, Prop-STP, P3C

(Prop-STP is not discussed for lack of time)

Floyd-Warshall for STP [CLR]

Basic STP solver, three nested loops Initialization: builds the distance graph

Rij = [a,b] gives eijb and eji-a

When edge does not exist, add infinite distance (complete graph) Time (n3), Space: No queue but O(n2) new edges

04/22/23

26

FLOYDWARSHALLFor k 1 to n For i 1 to n For j 1 to n w(eij) MIN(w(eij),w(eik)+w(ekj))

d[s] 0for each vertex i other than source (i s) d[i]

Repeat n-1 times for each edge eij if d[i] + w(eij) < d[j] then d[j] d[i] + w(eij)

for each edge eij if d[i] + w(eij) < d[j] then return inconsistent

04/22/23

27

Bellman-Ford for STP [CLR]

Time: O(en), Space: No queue but O(n) new edges

Detects path consistency Edges are not guaranteed minimal

PPC Operates on triangulated graphs ‘Fully’ filters convex constraints

∆STP adapts & refines PPC to STP Keeps a queue of triangles (vs. a queue of edges) Pops a triangle from queue & updates all 3 edges Implicitly separate graph in biconnected

components Enqueues triangles adjacent to only the updated

edge Best performance when queue is FIFO

04/22/23

28

∆STP [Xu & Choueiry 03]

04/22/23

29

Performance on STPs: F-W, PPC, STP [Xu & Choueiry 03]

04/22/23

30

Constraint ChecksGenSTP with 50 nodes

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Density

Co

ns

tra

ints

Ch

ec

ks

BF+APDPC+AP STP

BF+AP

DPC+AP

STP

Performance on STPs: BF, DPC, STP [Shi+ 05]

04/22/23

31

Designers of ∆STP became aware of relevance of simplicial ordering in ∆STP in 2005 (ref. Nic Wilson)

Designers of Prop-STP exploited the idea The authors of P3C formalize the flaw of ∆STP Identified a pathological case where ∆STP does

unnecessary work (not useful filtering) Characterized it as set of problems where ∆STP runs in

Ω(t2), where t is the number of triangles P3C addresses flaw by using a simplicial ordering Proves that propagation can be achieved in (t)

P3C: The Idea [Planken+ 08]

P3C: Pathological Case[Planken+ 08]

ci→i+1 | 0 ≤ i ≤ t+1 with zero weight

ci→j | (1 ≤ i ≤ j−2 < t) ∧ i+j−t ∈ 1,2 with weight j−i−1

cj→i | (1 ≤ i ≤ j−2 < t) ∧ i+j−t ∈ 1,2 with weight t−(j−i−1)

04/22/23

32

Path. Case: Empirically[Planken+ 08]

04/22/23

33

cubi

cquadratic

linear

P3C: The Algorithm[Planken+ 08]

Given: A triangulated graph & a perfect elimination order

04/22/23

34

k

j

i

k

j

i

The algorithm has two steps Bottom up: For every node

Considers every pair of parents Updates the edge between parents

(ref. DPC) Top down: For every node

Considers every pair of parents Updates edges adjacent to node

1

2

P3C: Bottom up [Planken+ 08]

DPC: For each node update the edges between all parents given the edges with the node

04/22/23

35

P3C: Top Down [Planken+ 08]

Update the edges between every node and its parent

04/22/23

36

P3C: Sound & complete[Planken+ 08]

Claim: on iteration k of P3C’s second half, all edges in the subgraph consisting of Vi | i ≤ k are minimal

Base case: k = 2 We know that c1,2 (in

orange) must be minimal if it exists, due to DPC; the subgraph will always be minimal for k = 2

04/22/23

37

k=2

1

2

1

k=3

Induction hypothesis: Gk-1=Vi≤k-1 forms a minimal subgraph

Induction step: Gk-1 minimalGk minimal

In the kth iteration, wik=min(wik, wij+ wjk) Any part of a theoretical shorter path that extends

out of the Gk-1can be replaced by its two endpoints within Gk-1, due to the filtering from DPC

The only way wik could be non-minimal is if there were a shorter path wij+wjk, but this path was checked in the kth iteration

We have a base step and an inductive step, so the proof is complete! 04/22/23

38

P3C: Sound & complete[Planken+ 08]

k

i

j

Gk

k-1

Gk-1

1

Time Complexity [Planken+ 08]

P3C runs in Θ(t), t is the number of triangles

O(t) ⊆ O(nw*2), w* is the min. induced width

O(nw*2) ⊆ O(nδ2), where δ is max. degree

O(nδ2) ⊆ O(n3)

DPC visits each triangle exactly once

The second half of P3C visits each triangle exactly once

Thus we have a constant number of visits to each triangle and a linear time complexity in the number of triangles

04/22/23

39

Jobshop (enforced consistency) [Planken+ 08]

04/22/23

40

In spite of theoretical bounds, the two algorithms are quite close

Summary of STP Algorithms

GraphPC-p?

Minimality?

TimeQueu

eExperiments

FWComple

teYes Yes Θ(n3) None Worst

BF Partial Yes No O(en) None Better than FW

PPCChorda

lYes Yes O(ed2) O(e)

DPCChorda

lYes No (t) None

Low density: less good than STP, high density same as STP

STP

Chordal

Yes YesO(min(t2, ed2)) O(e)

Faster than PPC, better than DPC on low densities

P3CChorda

lYes Yes (t) None Best reported,

04/22/23

41

Future Work [Planken+ 08]

Extend P3C to general CSPs

Investigate efficiency of triangulation algorithms vs. P3C

Incremental P3C solver for STPs

04/22/23

42

References

CSP PC-1, PC-2: see [Mackworth, AIJ 1977] PPC: see [Bliek & Sam-Haroud, IJCAI 1999] DPC: see [Dechter 4.2.2, Dechter & Pearl, AIJ 1987] PC-8: see [Chmeiss & Jégou, IJAITools 1998] PC-2001: see [Bessière et. Al, AIJ 2005]

STP Floyd-Warshall, Bellman-Ford: see CLR textbook DPC: see [Dechter et al., AIJ 1991] ∆STP: see [Xu and Choueiry, TIME 2003] Prop-STP: see [Bui, Tyson, and Yorke-Smith, AAAI 07,

Workshop] P3C: see [Planken et al., ICAPS 2008]

04/22/23

43