CSCE 896-004, Fall 2009 cse.unl/~choueiry/F09-896-004/ Questions : [email protected]

Basic Concepts & Algorithms in CP, Fall 2009

Aug 31, 2009 Overview 1 1

CSCE 896-004, Fall 2009

www.cse.unl.edu/~choueiry/F09-896-004/Questions: [email protected]

Broadcast: [email protected]

Berthe Y. Choueiry (Shu-we-ri)AVH 360

[email protected] Tel: +1(402)472-5444

Constraint Satisfaction 101



Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

(Implementing backtrack search)

Advanced solving techniques

Issues & research directions



Motivating example• Context: You are a senior in college

• Problem: You need to register in 4 courses for the Spring semester

• Possibilities: Many courses offered in Math, CSE, EE, CBA, etc.

• Constraints: restrict the choices you can make– Unary: Courses have prerequisites you have/don't have

Courses/instructors you like/dislike

– Binary: Courses are scheduled at the same time– n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered

• You have choices, but are restricted by constraints– Make the right decisions!!



Motivating example (cont’d)• Given

– A set of variables: 4 courses at UNL– For each variable, a set of choices (values)– A set of constraints that restrict the combinations

of values the variables can take at the same time

• Questions– Does a solution exist? (classical decision problem)– How two or more solutions differ? How to change

specific choices without perturbing the solution?– If there is no solution, what are the sources of

conflicts? Which constraints should be retracted?– etc.



Practical applications

• Radio resource management (RRM)• Databases (computing joins, view updates)• Temporal and spatial reasoning • Planning, scheduling, resource allocation • Design and configuration • Graphics, visualization, interfaces • Hardware verification and software engineering• HC Interaction and decision support • Molecular biology • Robotics, machine vision and computational linguistics • Transportation • Qualitative and diagnostic reasoning

Adapted from E.C. Freuder



Constraint Processing

• is about ...– Solving a decision problem…– … While allowing the user to state arbitrary

constraints in an expressive way and– Providing concise and high-level feedback about

alternatives and conflicts

• Related areas: – AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc.



• Flexibility & expressiveness of representations

• Interactivity

users can constraints

Power of Constraints Processing

relax

reinforce



Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal

characterizationBasic solving techniques



Defining a problem

• General template of any computational problem– Given:

• Example: a set of objects, their relations, etc.

– Query/Question: • Example: Find x such that the condition y is

satisfied

• How about the Constraint Satisfaction Problem?



Definition of a CSP• Given P = (V, D, C )

– V is a set of variables,

– D is a set of variable domains (domain values)

– C is a set of constraints,

• Query: can we find a value for each variable such that all constraints are satisfied?

nVVV ,,, 21 V

nVVV DDD ,,, 21 D

iVbVaVVVV DDDzyxiba

),,,(,...,,C with

lCCC ,,, 21 C



Other queries…

• Find a solution

• Find all solutions

• Find the number of solutions

• Find a set of constraints that can be removed so that a solution exists

• Etc.



Representation I• Given P = (V, D, C ), where

• Find a consistent assignment for variables

nVVV ,,, 21 V nVVV DDD ,,, 21 D

Constraint Graph

• Variable node (vertex)

• Domain node label

• Constraint arc (edge) between nodes

lCCC ,,, 21 C



Representation II• Given P = (V, D, C ), where


Example I:

• Define C ?

{1, 2, 3, 4}

{ 3, 5, 7 }{ 3, 4, 9 }

{ 3, 6, 7 }

v2 > v4

V4

V2

v1+v3 < 9

V3

V1

v2 < v3

v1 < v2

lCCC ,,, 21 C



Outline





Example II: Temporal reasoning

• Give one solution: …….• Satisfaction, yes/no: decision problem

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ] B < C

A < B

B

A

2 < C - A < 5C



Example III: Map coloring

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

• Variables?

• Domains?

• Constraints?



Example III: Map coloring (cont’d)

Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color

{ red, green, blue }


{ red, green, blue }{ red, green, blue }{ red, green, blue }

WY

NE

KS

OKNM

TX

LA

CO

UT

AZ

AR



Example IV: Resource Allocation

What is the CSP formulation?

{ a, b, c }

{ a, b } { a, c, d } { b, c, d }



Example IV: RA (cont’d)

{ a, b, c }

{ a, b } { a, c, d } { b, c, d }

{ R1, R3 }T1

{ R1, R3 }

{ R1, R3 } { R1, R3, R4 }

{ R1, R2, R3 }

{ R2, R4 }

{ R2, R4 }

T2

T3 T4

T5

T6

T7

Interval Order

T2

T1Constraint Graph

T4

{ R1, R3 }T3

{ R2, R4 }

{ R2, R4 }

T6

T7T5

{ R1, R2, R3 }

{ R1, R3 }

{ R1, R3 }

{ R1, R2, R3 }



Example V: PuzzleGiven: • Four musicians: Fred, Ike, Mike, and Sal, play bass, drums,

guitar and keyboard, not necessarily in that order.

• They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’

• Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist.

• etc ...

Query: Who plays which instrument and who composed which song?



Example V: Puzzle (cont’d)

Formulation 1: • Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song Gentle Rhythm and Nice Melody.• Domains: Fred, Ike, Mike, Sal • Constraints: …

Formulation 2:• Variables: Fred's-instrument, Ike's-instrument, …,

Fred's-song, Ikes's-song, Mike’s-song, …, etc.• Domains:

{ bass, drums, guitar, keyboard } { Blue Sky, Happy Song, Gentle Rhythm, Nice Melody}

• Constraints: …



Example VI: Product Configuration

Train, elevator, car, etc.

Given: • Components and their attributes (variables)• Domain covered by each characteristic (values)• Relations among the components (constraints) • A set of required functionalities (more constraints)

Find: a product configuration i.e., an acceptable combination of components that realizes the required functionalities



Example VII: Cryptarithmetic puzzles

• DX1 = DX2 = DX3 = {0,1}• DF=DT=DU=DW=DR=DO=[0,9]

• O+O = R+10X1• X1+W+W = U+10X2• X2+ T+T = O + 10X3• X3=F• Alldiff({F,D,U,V,R,O})



Constraint types: examples• Example I: algebraic constraints

• Example II:

(algebraic) constraints

of bounded difference

• Example III & IV: coloring, mutual exclusion, difference constraints

• Example V & VI: elements of C must be made explicit

{1, 2, 3, 4}

{ 3, 5, 7 }{ 3, 4, 9 }

{ 3, 6, 7 }

v2 > v4

V4

V2

v1+v3 < 9

V3

V1

v2 < v3

v1 < v2

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ] B < C

A < B

B

A

2 < C - A < 5C



{ red, green, blue }{ red, green, blue }{ red, green, blue }

WY

NE

KS

OKNM

TX

LA

CO

UT

AZ

AR

T1

Constraint Graph

T4

{ R1, R3 }T3

{ R2, R4 }

{ R2, R4 }

T6

T7T5

{ R1, R2, R3 }

{ R1, R3 }

{ R1, R3 }

{ R1, R2, R3 }



More examples

• Example VII: Databases– Join operation in relational DB is a CSP– View materialization is a CSP

• Example VIII: Interactive systems– Data-flow constraints– Spreadsheets– Graphical layout systems and animation– Graphical user interfaces

• Example IX: Molecular biology (bioinformatics)– Threading, etc



Outline





Representation (again)Macrostructure G(P):- constraint graph for

binary constraints - constraint network for non-binary constraints

Micro-structure (P):

Co-microstructure co-(P):

a, b

a, c b, c

V1

V2 V3

(V1, a ) (V1, b)

(V2, a ) (V2, c) (V3, b ) (V3, c)

(V1, a ) (V1, b)

(V2, a ) (V2, c) (V3, b ) (V3, c)

no goods



Constraint arity I• Given P = (V, D, C ) , where


• How to represent the constraint V1 + V2 + V4 < 10 ?

{1, 2, 3, 4}

{ 3, 5, 7 }{ 3, 4, 9 }

{ 3, 6, 7 }

v2 > v4

V4

V2

v1+v3 < 9

V3

V1

v2 < v3

v1 < v2

lCCC ,,, 21 C



Constraint arity II• Given P = (V, D, C ) , where

nVVV ,,, 21 V

nVVV DDD ,,, 21 D

Constraints: universal, unary, binary, ternary, … , global.

A Constraint Network: { 3, 6, 7 }{ 1, 2, 3, 4 }

{ 3, 5, 7 }{ 3, 4, 9 }

V3

V1 V2

v2 > v4

V4

v1 < v2

v2 < v3 v1+v3 < 9

v1+v2+V4 < 10

lCCC ,,, 21 C



Domain types• P = (V, D, C ) where

• Domains:– Restricted to {0,1}: Boolean CSPs– Finite (discrete), enumeration works– Continuous, sophisticated algebraic techniques are

needed• Consistency techniques on domain bounds


mVlVkVVVV DDDzyxmlk

),,,(,...,,C with

lCCC ,,, 21 C



Constraint terminology• Scope: The set of variables to which the constraint applies• Arity: unary, binary, ternary, …, global• Definition:

– Extension: all tuples are enumerated– Intension: when it is not practical (or even possible) to list all

tuples• Define types/templates of common constraints to be used

repeatedly: linear constraints, All-Diff (mutex), Atmost, TSP-constraint, cycle-constraint, etc.)

• Implementation: – Predicate function– Set of tuples (list or table) – Binary matrix (bit-matrix)– Constrained Decision Diagrams ([Cheng & Yap, AAAI 05])– etc.



Complexity of CSP

Characterization:– Decision problem

– In general, NP-complete

by reduction from 3SAT



Proving NP-completeness

1. Show that 1 is in NP

2. Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1

– Select a known NP-complete problem 2 (e.g., SAT)

– Construct a transformation f from 2 to 1

– Prove that f is a polynomial transformation

(Check Chapter 3 of Garey & Johnson)



What is SAT?Given a sentence:

– Sentence: conjunction of clauses

– Clause: disjunction of literals

– Literal: a term or its negation

– Term: Boolean variable

Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied.

4326541 ccccccc

32 cc

61, cc

01 11 cc



CSP is NP-Complete• Verifying that an assignment for all

variables is a solution – Provided constraints can be checked in

polynomial time

• Reduction from 3SAT to CSP– Many such reductions exist in the literature

(perhaps 7 of them)



Problem reductionExample: CSP into SAT (proves nothing, just an exercise)

Notation: variable-value pair = vvp

• vvp term– V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d),

– V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c).

• The vvp’s of a variable disjunction of terms– V1 = {a, b, c, d} yields

• (Optional) At most one VVP per variable

4321 xxxx

43214321

43214321

xxxxxxxx

xxxxxxxx



CSP into SAT (cont.)Constraint:

1. Way 1: Each inconsistent tuple one disjunctive clause– For example: how many?

2. Way 2:a) Consistent tuple conjunction of termsb) Each constraint disjunction of these conjunctions

transform into conjunctive normal form (CNF)

Question: find a truth assignment of the Boolean variables such that the sentence is satisfied

)},(),,(),,(),,(),,{(21

adbccbbaaaC VV

71 xx

51 xx

5463

726151

xxxx

xxxxxx



Outline

Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

• Modeling and consistency checking• Constructive, systematic search• Iterative improvement, local search



How to solve a CSP?

Search

1. Constructive, systematic

2. Iterative repair, local search



Before starting search!

Consider:• Importance of modeling/formulation:

– To control the size of the search space

• Preprocessing – A.k.a. constraint filtering/propagation, consistency

checking– reduces size of search space



4 Rows

V1

V2

V3

V4

Importance of modeling

• N-queen: formulation 1

– Variables? – Domains?– Size of CSP?

• N-queens: formulation 2

– Variables?– Domains?– Size of CSP?

4 Column positions

1 2 3 4

84 244444

{0,1}162

16 Cells

V11 V12 V13 V14

V21 V22 V23 V24

V31 V32 V33 V34

V41 V42 V43 V44



Constraint checking Arc-consistency

[ 5.... 18]

[ 4.... 15]

[ 1.... 10 ]B < C

A < B

B

A

2 < C - A < 5

C

2- A: [ 2 .. 10 ]

C: [ 6 .. 14 ]

3- B: [ 5 .. 13 ]

C: [ 6 .. 15 ]

1- B: [ 5 .. 14 ]1413

6

2

14



Constraint checking Arc-consistency: every combination of two adjacent variables 3-consistency, k-consistency (k n)

Constraint filtering, constraint checking, etc.. Eliminate non-acceptable tuples prior to search Warning: arc-consistency does not solve the problem

{ 1, 2, 3 } { 2, 3, 4 }

BA BA

{ 2, 3 } { 2, 3 }

)3()2( BA still is not a solution!



Systematic search Starting from a root node Consider all values for a variable V1

For every value for V1, consider all values for V2

etc..

For n variables, each of domain size d Maximum depth? fixed! Maximum number of paths? size of search space, size of CSP

S

v1 v4Var 1

Var 2

v3v2



Systematic search (I) Back-checking

• Systematic search generates dn possibilities• Are all possibilities acceptable?

Expand a partial solution only when it is consistent This yields early pruning of inconsistent paths

S

v1 v4Var 1

Var 2

v3v2

S

v1 v4Var 1

Var 2

v3v2



Systematic search (II) Chronological backtracking

What if only one solution is needed?

• Depth-first search & Chronological backtracking

• DFS: Soundness? Completeness?

Var 1 v1 v2

S

S

v1 v4Var 1

Var 2

v3v2



Systematic search (III) Intelligent backtracking

What if the reason for failure was higher up in the tree?

Backtrack to source of conflict !!

Backjumping, conflict-directed backjumping, etc.

Var 1 v1 v2

S S

v1 v2Var 1



Systematic search (IV) Ordering heuristics

• Which variable to expand first?

• Heuristics:– most constrained variable first (reduce branching factor)

– most promising value first (find quickly first solution)

},{},,,,{,,:2121 baDdcbaDVV VV Exp

)}(),{()}(),{(: 2121 bVcVaVcV andSol

dynamic

staticbecould

ordering value

ordering variableforStrategies

s

c

aV2 b a b a b

ba dV1

V1

s

V2 ba

ba c d



Systematic search (V) Back-checking

• Search tree with only backtrack search?Root node

1 2 4

1 3 41 2 3 4

1 2 3 4

1Q

3Q Q

2Q

1 2 3

1 2

2Q

4Q

1Q

3Q

Solution!

26 nodes visited.



Systematic search (VI) Forward checking

Root node

1Q

3Q Q 4

Domain Wipe Out V3

2Q

Domain Wipe Out V4

2Q

4Q

1Q

3Q

Solution!

8 nodes visited.

Search Tree with domains filter by Forward Check



Improving BT search

• General purpose methods for1. Variable, value ordering

2. Improving backtracking: intelligent backtracking avoids repeating failure

3. Look-ahead techniques: propagate constraints as variables are instantiated



Search-tree branching

• K-way branching– One branch for every value in the domain

• Binary branching– One branch for the first value– One branch for all the remaining values in the domain– Used in commercial constraint solvers: ILOG, Eclipse

• Have different behaviors (e.g., WRT value ordering

heuristics [Smith, IJCAI 95])



Summary of backtrack search• Constructive, systematic, exhaustive

– In general sound and complete

• Back-checking: expands nodes consistent with past• Backtracking: Chronological vs. intelligent• Ordering heuristics:

– Static– Dynamic variable– Dynamic variable-value pairs

• Looking ahead:– Partial look-ahead

• Forward checking (FC)• Directional arc-consistency (DAC)

– Full (a.k.a. Maintaining Arc-consistency or MAC)



CSP: a decision problem (NP-complete)

1. Modeling: abstraction and reformulation2. Preprocessing techniques:

• eliminate non-acceptable tuples prior to search

3. Systematic search:• potentially dn paths of fixed lengths• chronological backtracking• intelligent backtracking• variable/value ordering heuristics

4. Search ‘hybrids:’• Mixing consistency checking with search: look-ahead

strategies



Non-systematic search• Iterative repair, local search: modifies a

global but inconsistent solution to decrease the number of violated constraints

• Examples: Hill climbing, taboo search, simulated annealing, GSAT, WalkSAT, Genetic Algorithms, Swarm intelligence, etc.

• Features: Incomplete & not sound

– Advantage: anytime algorithm

– Shortcoming: cannot tell that no solution exists



Outline of CSP 101

• We have seenMotivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques

• We will move to (Implementing backtrack search)

Advanced solving techniquesIssues & research directions

Documents

CSCE 896-004, Fall 2009 cse.unl/~choueiry/F09-896-004/ Questions : [email protected]