Constraint Overview Satisfaction and • CSPs defined • Using standard search for CSPs Scheduling • Blindingly obvious improvements ƒ Backtracking search ƒ Forward Checking Andrew W. Moore ƒ Constraint Propagation Professor • Some example CSP applications School of Computer Science ƒ Overview Carnegie Mellon University ƒ Waltz Algorithm www.cs.cmu.edu/~awm ƒ Job Shop Scheduling [email protected] • Variable ordering 412-268-7599 • Value ordering

Note to other teachers and users of these slides. Andrew would be delighted if you found this source • Tedious Discussion material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit your own needs. PowerPoint originals are available. If you make use of a significant portion of these slides in your own lecture, please include this message, or the following link to the source repository of Slide 1 Slide 2 Andrew’s tutorials: http://www.cs.cmu.edu/~awm/tutorials . Comments and corrections gratefully received.

A Constraint Satisfaction Problem Formal Constraint Satisfaction Problem V 5 A CSP is a triplet { V , D , C }. A CSP has a finite set of variables V = { V , V V3 1 2 V1 .. V }. V2 N Each variable may be assigned a value from a domain D of values. C are binary constraints. Each member of C is a pair. The first member of each pair is a set of variables. The second element is a set of legal values which V6 R that set may take. Example: V4 G V = { V1 , V2 , V3 , V4 , V5 , V6 } D = { R , G , B }

C = { (V1,V2) : { (R,G), (R,B), (G,R), (G,B), (B,R) (B,G)}, Inside each circle marked V1 .. V6 we must assign: R, G or B. { (V1,V3) : { (R,G), (R,B), (G,R), (G,B), (B,R) (B,G)}, No two connected circles may be assigned the same symbol. : Notice that two circles have already been given an assignment. : }

Obvious point: Usually C isn’t represented explicitly, but by a function. Slide 3 Slide 4

How to solve our CSP? How to solve our CSP?

V5 V5 V3 V3 V1 V1 V2 V2

V6 R V6 R

V4 G V4 G

•How about using a search algorithm? START =(V1=? V2=? V3=? V4=? V5=? V6=?) •Define: a search state has variables 1 … k assigned. Values k+1 … n, succs(START) = as yet unassigned. (V1=RV2=? V3=? V4=? V5=? V6=?) •Start state: All unassigned. (V1=GV2=? V3=? V4=? V5=? V6=?) •Goal state: All assigned, and all constraints satisfied. (V1=BV2=? V3=? V4=? V5=? V6=?) •Successors of a stated with V1 … Vk assigned and rest unassigned are What search algorithms could we use? all states (with V1 … Vk the same) with Vk+1 assigned a value from D. It turns out BFS is not a popular choice. Why not? •Cost on transitions: 0 is fine. We don’t care. We just want any solution. Slide 5 Slide 6

1 DFS for CSPs Blindingly obvious improvement – Consistency Checking: V5 V3 V1 V2 “Backtracking Search”

V5 V3 V6 R V1 V2 V4 G

What about DFS? V6 R

Much more popular. At least it has a chance of finding an easy answer V4 G quickly. Don’t ever try successor which causes inconsistency with its neighbors. What happens if we do DFS with the order of assignments as B tried Again, what happens if we do DFS with the order of assignments as first, then G then R? B tried first, then G then R? This makes DFS look very, very stupid! What’s the computational overhead for this? Backtracking still looks a little stupid! Slide 7 Slide 8

Obvious improvement – Forward Constraint Propagation Checking V5 V3 V V1 5 V2 V3 V1 V2

V6 R V R 6 V4 G V4 G Forward checking computes the domain of each variable independently At start, for each variable, record the current set of possible legal values at the start, and then only updates these domains when assignments are for it. made in the DFS that are directly relevant to the current variable. When you assign a value in the search, update set of legal values for all Constraint Propagation carries this further. When you delete a value variables. Backtrack immediately if you empty a variable’s constraint from your domain, check all variables connected to you. If any of them set. change, delete all inconsistent values connected to them, etc… Again, what happens if we do DFS with the order of assignments as Often constraint propagation is only performed at the start. In our B tried first, then G then R? example it is useless. But… What’s the computational overhead? Slide 9 Slide 10

Constraint Propagation being non-useless Graph-coloring-specific Constraint

V5 Propagation V3 V V1 2 In the case of Graph Coloring, CP looks simple: after we’ve made a search step (instantiated a V R 6 node with a color), propagate the color at that V G 4 Extra Arc node. • In this example, constraint propagation solves the problem PropagateColorAtNode(node,color) without search … Not always that lucky! 1. remove color from all of “available lists” of our uninstantiated neighbors. • Constraint propagation can be done as a preprocessing 2. If any of these neighbors gets the empty set, it’s time to step. (Cheap). backtrack. • Or it can be maintained dynamically during the search. 3. Foreach n in these neighbors: if n previously had two or more available colors but now has only one color c, run Expensive: when you backtrack, you must undo some of PropagateColorAtNode(n,c) your additional constraints. Slide 11 Slide 12

2 Graph-coloring-specific Constraint A New CSP (where fancier Propagation propagation is possible) In the case of Graph Coloring, CP looks simple: • The semi magic square after we’ve made a search step (instantiated a • Each variable can have value 1, 2 or 3 node with a color), propagate the color at that node. This row must V1 V2 V3 PropagateColorAtNode(node,color) sum to 6 This row must 1. remove color from all of “available lists” of our V4 V5 V6 uninstantiated neighbors. sum to 6 nstraint 2. If any of these neighbors gets theem emptys, c oset, it’s time to This row must SP probl only V7 V8 V9 backtrack. eneral C ore than sum to 6 But for G much m e 3. Foreach n in othesen can neighbors: do if n previouslyts a uni hadqu two or opagati node ge morepr available colorswhe butn a now has only one color c, run This column This column This column This diagonal agating PropagateColorAtNode(n,c)prop must sum to 6 must sum to 6 must sum to 6 must sum to 6 value… Slide 13 Slide 14

General Constraint Propagation General Constraint Propagation Propagate(A , A ,… A ) Propagate(A , A ,… A ) 1 2 n 1 2 n A denotes the current set of possible values for finished = FALSE finished = FALSE i Specification: Takes a set of availability-lists variable i. This is call-by-reference. Some of the Ai while not finished for each and every node and uses all the while not finished sets may be changed by this call (they’ll have one finished = TRUE constraints to filter out impossible values that finished = TRUE or more elements removed) are currently in availability lists foreach constraint C foreach constraint C

Assume C concerns variables V1, V2 ,… Vk Assume C concerns variables V1, V2 ,… Vk

Set NewAV1 = {} , NewAV2 = {} , … NewAVk = {} Set NewAV1 = {} , NewAV2 = {} , … NewAVk = {}

Foreach assignment (V1=x1, V2=x2, … Vk=xk) in C Foreach assignment (V1=x1, V2=x2, … Vk=xk) in C

If x1 in AV1 and x2 in AV2 and … xk in AVk If x1 in AV1 and x2 in AV2 and … xk in AVk

Add x1 to NewAV1 , x2 to NewAV2 ,… xk to NewAVk Add x1 to NewAV1 , x2 to NewAV2 ,… xk to NewAVk for i = 1 , 2 … k for i = 1 , 2 … k

AVi := AVi intersection NewAVi AVi := AVi intersection NewAVi If AVi was made smaller by that intersection We’ll keep iteratingIf AVi untilwas we made do a smaller by that intersection finished = FALSE full iteration in whichfinished none of the= FALSE availability lists change. The If AVi is empty, we’re toast. Break out with “Backtrack” signal. If AVi is empty, we’re toast. Break out with “Backtrack” signal. Deta “finished” flag is just to record ils on next slide Slide 15 whether a change took place. Slide 16

General Constraint Propagation General Constraint Propagation NewA is going to be filled up If this test is satisfied that means that there’s at least one Propagate(A , A ,… A ) i Propagate(A , A ,… A ) 1 2 n with the possible values for 1 2 n value q such that we originally thought q was an finished = FALSE finished = FALSE variable Vi taking into account available value for Vi but we now know q is impossible. while not finished the effects of constraint C while not finished finished = TRUE finished = TRUE foreach constraint C foreach constraint C

Assume C concerns variables V1, V2 ,… Vk Assume C concerns variables V1, V2 ,… Vk

Set NewAV1 = {} , NewAV2 = {} , … NewAVk = {} Set NewAV1 = {} , NewAV2 = {} , … NewAVk = {}

Foreach assignment (V1=x1, V2=x2, … Vk=xk) in C Foreach assignment (V1=x1, V2=x2, … Vk=xk) in C

If x1 in AV1 and x2 in AV2 and … xk in AVk If x1 in AV1 and x2 in AV2 and … xk in AVk

Add x1 to NewAV1 , x2 to NewAV2 ,… xk to NewAVk Add x1 to NewAV1 , x2 to NewAV2 ,… xk to NewAVk for i = 1 , 2 … k After we’ve finished all the for i = 1 , 2 … k iterations of the foreach AVi := AVi intersection NewAVi AVi := AVi intersection NewAVi loop, NewAi contains the If AVi was made smaller by that intersectionfull set of possible values of If AVi was made smaller by that intersection

finished = FALSE variable Vi taking into finished = FALSE If A is empty, we’re toast. Break outaccount with “Backtrack”the effects of signal. If A is empty, we’re toast. Break out with “Backtrack” signal. Vi constraint C. Vi

Slide 17 If AVi is empty we’ve proved that there are no solutions for the Slide 18 availability-lists that we originally entered the function with

3 (V1,V2,V3) must be one of (1,2,3) Propagate on Semi-magic Square Propagate on Semi-magic(1,3,2) Square (2,1,3) (2,2,2) • The semi magic square • The semi magic square (2,3,1) (3,1,2) • Each variable can have value 1, 2 or 3 • Each variable can have value 1, 2 or 3 (3,2,1)

1 123 123 This row must 1 123 123 This row must sum to 6 sum to 6 123 123 123 This row must 123 123 123 This row must sum to 6 sum to 6 123 123 123 This row must 123 123 123 This row must sum to 6 sum to 6

This column This column This column This diagonal This column This column This column This diagonal must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6

Slide 19 Slide 20

(V1,V2,V3) must be one of (1,2,3) Propagate on Semi-magic(1,3,2) Square After doing first row constraint… •NewALV1 = { 1 } (2,1,3) •NewALV2 = { 2 , 3 } (2,2,2) (2,3,1) •NewAL• TheV3 semi= { 2 , magic3 } square (3,1,2) • Each variable can have value 1, 2 or 3 (3,2,1)

1 123 123 This row must 1 23 23 This row must sum to 6 sum to 6 123 123 123 This row must 123 123 123 This row must sum to 6 sum to 6 123 123 123 This row must 123 123 123 This row must sum to 6 sum to 6

This column This column This column This diagonal This column This column This column This diagonal must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6 must sum to 6

Slide 21 Slide 22

After doing all row constraints and And after doing diagonal column constraints… constraint… 1 23 23 This row must sum to 6 23 23 123 This row must sum to 6 This row must 1 23 23 23 123 23 This row must sum to 6 sum to 6 23 123 123 This row must sum to 6 This column This column This column This diagonal 23 123 123 This row must must sum to 6 must sum to 6 must sum to 6 must sum to 6 sum to 6

This column This column This column This diagonal CP has now iterated through all constraints once. must sum to 6 must sum to 6 must sum to 6 must sum to 6 But does it make further progress when it tries iterating through them again? Slide 23 Slide 24

4 And after doing another round of CSP Search with Constraint Propagation constraints…

This row must CPSearch(A , A ,… A ) 1 23 23 1 2 n sum to 6 Let i = lowest index such that Ai has more than one value foreach available value x in A 23 23 12 This row must i sum to 6 foreach k in 1, 2.. n Define A’ := A 23 12 23 This row must k k sum to 6 A’i := { x }

Call Propagate(A’1, A’2 ,… A’n) This column This column This column This diagonal If no “Backtrack” signal must sum to 6 must sum to 6 must sum to 6 must sum to 6 If A’1, A’2 ,… A’n are all unique we’re done! ide Recursively Call CPSearch(A’1, A’2 ,… A’n) sl xt YES! And this showed a case of a n e ges o n chan on constraint applying even when none of more s any on? il So.. erati ta the variables involved was down to a ext it e the n D unique value. Slide 25 Slide 26

CSP SearchSpecification: with Constraint Find out if there’s Propagation CSP Search with Constraint Propagation any combination of values in the combination of the given availability lists that satisifes all constraints.

CPSearch(A1, A2 ,… An) CPSearch(A1, A2 ,… An)

Let i = lowest index such that Ai has more than one value Let i = lowest index such that Ai has more than one value

foreach available value x in Ai At this point the A-primes are a copy foreach available value x in Ai foreach k in 1, 2.. n of the original availability lists except foreach k in 1, 2.. n A’i has committed to value x. Define A’k := Ak Define A’k := Ak A’i := { x } This call may prune away A’i := { x } Call Propagate(A’1, A’2 ,… A’n) some values in some of Call Propagate(A’1, A’2 ,… A’n) If no “Backtrack” signal the copied availability lists If no “Backtrack” signal

If A’1, A’2 ,… A’n are all unique we’re done! If A’1, A’2 ,… A’n are all unique we’re done!

Recursively Call CPSearch(A’1, A’2 ,… A’n) Recursively Call CPSearch(A’1, A’2 ,… A’n)

What’s the top-level call? Assuming that we terminate deep in the recursion if we find a solution, the CPSeach function only terminates normally if no Call with that Ai = complete set of possible values for Vi . solution is found. Slide 27 Slide 28

CSP Search with Constraint Propagation Semi-magic Square 123 123 123 CPSearch Tree 123 123 123 123 123 123 CPSearch(A1, A2 ,… An)

Let i = lowest index such that Ai has more than one value foreach available value x in Ai 1 23 23 2 123 123 3 12 12 foreach k in 1, 2.. n 23 23 12 123 123 123 12 12 23 Define A’k := Ak

A’i := { x } 23 12 23 123 123 123 12 23 12

Call Propagate(A’1, A’2 ,… A’n) If no “Backtrack” signal If A’ , A’ ,… A’ are all unique we’re done! 1 2 n 1 2 3 1 3 2 Recursively Call CPSearch(A’1, A’2 ,… A’n) 2 3 1 3 2 1 What’s the top-level call? 3 1 2 2 1 3

Call with that Ai = complete set of possible values for Vi . Slide 29 Slide 30

5 Semi-magic Square 123 123 123 Some real CSPs CPSearch Tree 123 123 123 • Graph coloring is a real, and useful, CSP. Applied to 123 123 123 problems with many hundreds of thousands of nodes. Not very AI-esque. 1 23 23 2 123 123 3 12 12 • VLSI or PCB board layout. 23 23 12 123 123 123 12 12 23 • Selecting a move in the game of “minesweeper”. 23 12 23 123 123 123 12 23 12 0 0 1 0 0 1 0 0 1 r 1 1 2 1 2 3 1 3 2 eve we n ese act, er th 2 3 1 3 2 1 In f nsid at n co stop eve we Which squares have a bomb? Squares with numbers don’t. Other squares 3 1 2 2 1 3 use beca ess might. Numbers tell how many of the eight adjacent squares have bombs. We succ first want to find out if a given square can possibly have a bomb…. Slide 31 Slide 32

“Minesweeper” CSP The Waltz algorithm

0 0 1 V1 One of the earliest examples of a computation posed as a CSP. 0 0 1 V2 The Waltz algorithm is for interpreting line drawings of solid polyhedra. 0 0 1 V3

1 1 2 V4

V8 V7 V6 V5

V = { V1 , V2 , V3 , V4 , V5 , V6 , V7 , V8 }, D = { B (bomb) , S (space) }

C = { (V1, V2) : { (B,S) , (S,B) }, (V1, V2, V3,) : { (B,S,S) , (S,B,S) , (S,S,B)},…} Look at all intersections. V1 What kind of intersection could this be? A V 2 concave intersection of three faces? Or V8 an external convex intersection? V V7 3 Adjacent intersections impose constraints on each other. Use CSP to V6 V4 find a unique set of labelings. Important step to “understanding” the

V5 image. Slide 33 Slide 34

Waltz Alg. on simple scenes 18 legal kinds of junctions Assume all objects: • Have no shadows or cracks • Three-faced vertices • “General position”: no junctions change with small movements of the eye. Then each line on image is one of the following: • Boundary line (edge of an object) (<) with right hand of arrow denoting “solid” and left hand denoting “space” • Interior convex edge (+) • Interior concave edge (-)

Given a representation of the diagram, label each junction in one of the above manners. The junctions must be labeled so that lines are labeled consistently at both ends. Can you formulate that as a CSP? FUN FACT: Constraint Propagation always Slide 35 works perfectly. Slide 36

6 Waltz Examples

Slide 37 Slide 38

Scheduling Job Shop scheduling

A very big, important use of CSP methods. At a job-shop you make various products. Each product is a “job” to be done. • Used in many industries. Makes many multi-million dollar decisions. E.G. • Used extensively for space mission planning. Job = Make a polished-thing-with-a-hole • Military uses. 1 Job2 = Paint and drill a hole in a widget People really care about improving scheduling algorithms! Each job requires several operations. E.G. Problems with phenomenally huge state spaces. But for which Operations for Job1: Polish, Drill Operations for Job : Paint, Drill solutions are needed very quickly. 2 Each operation needs several resources. E.G. Many kinds of scheduling problems e.g.: Polishing needs the Polishing machine ™ Job shop: Discrete time; weird ordering of operations possible; set Polishing needs Pat (a Polishing expert) of separate jobs. Drilling needs the Drill ™ Batch shop: Discrete or continuous time; restricted operation of Drilling needs Pat (also a Drilling expert) Or Drilling can be done by Chris ordering; grouping is important. Some operations need to be done in a particular order (e.g. Paint after you’ve ™ Manufacturing cell: Discrete, automated version of open job shop. Drilled)

Slide 39 Slide 40

Job Shop Formalized Job Shop Example

1 1 A Job Shop problem is a pair ( J , RES ) j1 = polished-hole-thing = { O 1 , O 2 } 2 2 J is a set of jobs J = {j1 , j2 , … jn} j2 = painted-hole-widget = { O 1 , O 2 }

RES is a set of resources RES = {R1 .. Rm} RES = { Pat,Chris,Drill,Paint,Drill,Polisher } 1 O 1 = polish-thing: need resources… 1 1 Each job jI is specified by: { R 11 = Pat , R 12 = Polisher } • a set of operations OI = {OI OI …OI } 1 1 2 n(I) O 2 = drill-thing: need resources… • and must be carried out between release-date rdI and due-date ddI. 1 1 1 1 { R 21 = (r 211=Pat or r 212=Chris), R 22 = Drill } I I I I • and a partial order of operations: (O i before O j), (O i’ before O j’), etc… 2 O 1 = paint-widget: need resources… 2 { R 11 = Paint } Each operation OI has a variable start time stI and a fixed duration duI and i i i O2 = drill-widget : need resources… requires a set of resources. e.g.: OI requires { RI , RI …}. 2 i i1 i2 2 2 2 2 { R 21 = (r 211=Pat or r 212=Chris), R 22 = Drill } 2 2 I Precedence constraints : O 2 before O 1. All operations take one time unit du i Each resource can be accomplished by one of several possible physical = 1 forall i,I. Both jobs have release-date rdI = 0 and due-date ddI = 1. I I I resources, e.g. R i1 might be accomplished by any one of {r ij1 , r ij2 , …}. Each I of the r ijks are a member of RES.

Slide 41 Slide 42

7 Job-shop: the Variables and A slightly bigger example Constraints O1 R before O1 R before O1 R Variables 1 1 2 2 3 3 I • The operation state times st i I • The resources R ij (usually these are obvious from the definition of 2 before 2 I O 1 R1 O 2 R2 O i. Only need to be assigned values when there are alternative physical resources available, e.g. Pat or Chris for operating the drill). Constraints: 3 before 3 before 3 I O 1 R3 O 2 R1 O 3 R2 • Precedence constraints. (Some O is must be before some other I O js). • Capacity constraints. There must never be a pair of operations with O4 R before O4 R overlapping periods of operation that use the same resources. 1 4 2 2 Example from [Sadeh and Fox, 96]: Norman M. Sadeh and Mark S. Fox, Variable and Value Ordering Heuristics for the Job Shop Scheduling Constraint Satisfaction Problem, Non-challenging question. Can you schedule our Job-shop? Artificial Intelligence Journal, Number Vol 86, No1, pages 1-41, 1996. Available from citeseer.nj.nec.com/sadeh96variable.html 4 jobs. Each 3 units long. All jobs have release date 0 and due date

Slide 43 15. All operations use only one resource each. Slide 44

General purpose Variable Ordering Heuristics Further CSP techniques 1. Most constrained variable. Let’s look at some other important CSP methods. Keep the 2. Most constraining variable. job-shop example in mind. V5 V3 Here’s another graph-coloring example (you’re now V1 V2 allowed R, G, B and Y)

V5 V6 R V3 V1 V2 V4 G B V7 V6 R Y

V4 G B V 7 Y

Slide 45 Slide 46

General purpose Value Ordering Heuristics General purpose CSP algorithm (From Sadeh+Fox) V5 1. If all values have been successfully assigned then stop, else go V3 V1 on to 2. V2 2. Apply the consistency enforcing procedure (e.g. forward-checking if feeling computationally mean, or constraint propagation if V6 R extravagant. There are other possibilities, too.)

V4 G 3. If a deadend is detected then backtrack (simplest case: DFS-type backtrack. Other options can be tried, too). Else go on to step 4. B V7 4. Select the next variable to be assigned (using your variable Y ordering heuristic). A good general purpose one is “least- 5. Select a promising value (using your value ordering heuristic). constrained-value”. Choose the value which 6. Create a new search state. Cache the info you need for backtracking. And go back to 1. causes the smallest reduction in number of available values for neighboring variables Slide 47 Slide 48

8 Job-shop example. Consistency Reactive CSP solutions enforcement • Say you have built a large schedule. • Disaster! Halfway through execution, one of the Sadeh claims that generally forward-checking is resources breaks down. We have to reschedule! better, computationally, than full constraint • Bad to have to wait 15 minutes for the scheduler to propagation. But it can be supplemented with a make a new suggestion. Job-shop specific TRICK. Important area of research: efficient The precedence constraints (i.e. the available schedule repair algorithms. times for the operations to start due to the ordering • Question: If you expect that resources may of operations) can be computed exactly, given a sometimes break, what could a scheduling program partial schedule, very efficiently. do to take that into account? • Unrelated Question: Why has none of this lecture Slide 49 used A*? Slide 50

Other approaches. And What You Should Know Other Approaches: ¾ Hill-climbing, Tabu-search, Simulated annealing, Genetic Algorithms. (to be discussed later) What you should know: 9 How to formalize problems as CSPs 9 Backtracking Search, Forward Checking, Constraint Propagation 9 The Waltz algorithm 9 You should understand and appreciate the way job-shop scheduling is formalized. It is an excellent representative example of how important well-studied constraint satisfaction problems are represented. 9 Understand examples of Variable ordering and Value ordering heuristics In those cases where your lecturer or these handouts are too incomprehensible, consult the Russell and Norvig. Winston’s “Artificial Intelligence” book also has good discussion of constraint satisfaction and Waltz algorithm. Slide 51

9