„Constraint programming represents one of the closest approaches computer science has yet made to Constraint propagation the Holy Grail of programming: the backtracking-&based search user states the problem, the backtracking-based search computer solves it.“
Eugene C. Freuder, Constraints, April 1997 Roman Barták Charles University, Prague (CZ)
[email protected] http://ktiml.mff.cuni.cz/~bartak
2
Holly Grail of Programming Tutorial outline
> Computer, solve the SEND, MORE, MONEY problem! Introduction history, applications, a constraint satisfaction problem > Here you are. The solution is [9,5,6,7]+[1,0,8,5]=[1,0,6,5,2] Depth-first search a Star Trek view backtracking, backjumping, backmarking incomplete and discrepancy search > Sol=[S,E,N,D,M,O,R,Y], Consistency domain([E,N,D,O,R,Y],0,9), domain([S,M],1,9), node, arc, and path consistencies 1000*S + 100*E + 10*N + D + k-consistency and global constraints 1000*M + 100*O + 10*R + E #= Combining search and consistency 10000*M + 1000*O + 100*N + 10*E + Y, all_different(Sol), look-back and look-ahead schemes labeling([ff],Sol). variable and value ordering Conclusions > Sol = [9,5,6,7,1,0,8,2] constraint solvers, resources today reality
3 4
A bit of history Scene labelling (Waltz 1975) feasible interpretation of 3D lines in a 2D drawing - - + + - + + + + + + - + +
IntroductionIntroduction Interactive graphics (Sutherland 1963, Borning 1981) geometrical objects described using constraints
Logic programming (Gallaire 1985, Jaffar, Lassez 1987) from unification to constraint satisfaction
6
1 Application areas Constraint technology All types of hard combinatorial problems: molecular biology based on declarative problem description via: DNA sequencing variables with domains (sets of possible values) determining protein structures e.g. start of activity with time windows
interactive graphic constraints restricting combinations of variables e.g. endA < startB web layout network configuration constraint optimization via objective function assignment problems e.g. minimize makespan personal assignment Why to use constraint technology? stand allocation understandable timetabling open and extendible scheduling proof of concept planning
7 8
CSP Two or more?
Constraint satisfaction problem consists of: Binary constraint satisfaction a finite set of variables only binary constraints domains - a finite set of values for each variable a finite set of constraints any CSP is convertible to a binary CSP constraint is an arbitrary relation over the set of dual encoding (Stergiou & Walsh, 1990) variables swapping the role of variables and constraints can be defined extensionally (a set of compatible v v tuples) or intentionally (formula) variables x1,…,x6 1 4 Booleanwith constraint domain {0,1} (0,0,1), (0,1,0),satisfactionR21 & R33 (0,0,0), (0,1,1), (1,0,0) (1,0,1) A solution to a CSP is a complete consistent only Booleanc1: x1+x2+x6=1 (two valued) domains assignment of variables. c2: x1-x3+x4=1 any CSPc3: x4+x5-x6>0is convertibleR11 to a BooleanR33 CSP R22 & R33 complete = a value is assigned to every variable c4: x2+x5-x6=0 SAT encoding consistent = all the constraints are satisfied Boolean variable indicates(0,0,1), (1,0,0), whether a given(0,1,0), (1,0,0),value is (1,1,1) (1,1,0), (1,1,1) assigned to the variable R31 v2 v3
9 10
Two or more?
Binary constraint satisfaction only binary constraints any CSP is convertible to a binary CSP dual encoding (Stergiou & Walsh, 1990) swapping the role of variables and constraints Depth-firstDepth-first search
Boolean constraint satisfaction only Boolean (two valued) domains any CSP is convertible to a Boolean CSP SAT encoding Boolean variable indicates whether a given value is assigned to the variable
11
2 Basic idea Chronological backtracking A recursive definition We are looking for a complete consistent assignment! procedure BT(X:variables, V:assignment, C:constraints) start with a consistent assignment (for example, empty one) if X={} then return V extend the assignment towards a complete assignment x ← select a not-yet assigned variable from X for each value h from the domain of x do Depth-first search is a technique of searching solution by extending a partial consistent assignment towards a if consistent(V+{x/h}, C) then complete consistent assignment. R ← BT(X-{x}, V+{x/h}, C) if R≠fail then return R assign values gradually to variables end for after each assignment test consistency of the constraints over the assigned variables return fail and backtrack upon failure end BT call BT(X, {}, C) Backtracking is probably the most widely used complete systematic search algorithm. complete = guarantees finding a solution or proving its non-existence Consistency procedure checks satisfaction of constraints whose variables are already assigned.
13 14
Weaknesses of backtracking Backjumping thrashing Backjumping is a technique for removing thrashing from backtracking. throws away the reason of the conflict How? Example: A,B,C,D,E:: 1..10, A>E 1) identify the source of the conflict (impossibility to assign a value) BT tries all the assignments for B,C,D before finding that A≠1 2) jump to the past variable in conflict Solution: backjumping (jump to the source of the failure) The same forward run like in backtracking, only the back-jump can be redundant work longer, and hence irrelevant assignments are skipped! unnecessary constraint checks are repeated How to find a jump position? What is the source of the conflict? Example: A,B,C,D,E:: 1..10, B+8 late detection of the conflict Graph-directed backjumping x constraint violation is discovered only when the values are known 1 2 34 5 Example: A,B,C,D,E::1..10, A=3*E Enhancement: use only the violated constraints the fact that A>2 is discovered when labelling E ª conflict-directed backjumping Solution: forward checking (forward check of constraints) 15 16 Conflict-directed backjumping Conflicting variable in practice N-queens problem in practice How to find out the conflicting variable? Situation: ABCDEFGH assume that the variable no. 7 is being assigned (values are 0, 1) 1 Queens in rows are allocated to columns. the symbol • marks the variables participating the violated 2 constraints (two constraints for each value) 3 6th queen cannot be allocated! Neither 0 nor 1 can be assigned to 4 1 • the seventh variable! 5 1. Write a number of conflicting 2 • • •2 queens to each position. 3 • •2 1. Find the closest variable in each 6 1 3,4 2,5 4,5 3,5 1 2 3 2. Select the farthest conflicting 4 • violated constraint (o). queen for each position. 7 5 • 2. Select the farthest variable from the above chosen variables for each 8 3. Select the closest conflicting Order of assignment 6 queen among positions. value (2). 7 • • • • 3. Choose the closest variable from Note: conflict conflict the conflicting variables selected for Graph-directed backjumping has no effect here (due to the complete graph)! with value 0 with value 1 each value and jump to it. 17 18 3 Consistency check (BJ) Gaschnig (1979) Algorithm backjumping In addition to the test of satisfaction of the constraints, the closest procedure BJ(Unlabeled, Labeled, Constraints, PreviousLevel) conflicting level is computed if Unlabelled = {} then return Labeled procedure consistent(Labeled, Constraints, Level) pick first X from Unlabelled J ← Level % the level to which we will jump Level ← PreviousLevel+1 Jump ← 0 NoConflict ← true % indicator of a conflict for each value V from DX do for each C in Constraints do C ← consistent({X/V/Level} ∪ Labeled, Constraints, Level) if all variables from C are Labeled then if C = fail(J) then if C is not satisfied by Labeled then Jump ← max {Jump, J} NoConflict ← false else Jump ← PreviousLevel J ← min {J, max{L | X in C & X/V/L in Labeled & L 19 20 Weakness of backjumping Ginsberg (1993) Dynamic backtracking The same graph (A,B,C,D,E), the same colours (1,2,3) example When jumping back the in-between assignment is lost! but a different approach. Backjumping Example: D + remember the source of the conflict colour the graph in such a way that the connected vertices have different + carry the source of the conflict colours C A B + change the order of variables E = DYNAMIC BACKTRACKING D node vertex A 1 B 1 node 1 2 3 node 1 2 3 node 1 2 3 B 2 21 A A • A • A • p C m B A • B A • CA • 1 2 ju 1 2 C ck CA • CA • B • A E D ba 1 2 1 2 3 DA B • D A B AB DA • E 1 2 3 1 2 3 EA B D EA B EA B • jump back jump back + carry the conflict source + carry the conflict source • selected colour + change the order of B, C During the second attempt to label C superfluous work is done AB a source of the conflict - it is enough to leave there the original value 2, the change of B The vertex C (and the possible sub-graph connected to C) is does not influence C. not re-coloured. 21 22 Ginsberg (1993) Algorithm dynamic BT Redundant work procedure DB(Variables, Constraints) What is redundant work in chronological backtracking? Labelled {}; Unlabelled Variables ← ← repeated computation whose result has already been obtained while Unlabelled ≠ {} do select X in Unlabelled Example: ValuesX ← DX - {values inconsistent with Labelled using Constraints} if ValuesX = {} then A,B,C,D :: 1..10, A+8 23 24 4 Haralick, Elliot (1980) Backmarking Backmarking Removes redundant constraint checks by memorizing negative and N-queens problem in practice positive tests: Mark(X,V) is the farthest (instantiated) variable in conflict with ABCDEFGH the assignment X=V 1. Queens in rows are allocated to 1 1 columns. BackTo(X) is the farthest variable to which we backtracked since the last attempt to instantiate X 2 1 1 1 2. Latest choice level is written next to chessboard (BackTo). At beginning 1s. Now, some constraint checks can be omitted: 3 1 2 1 2 1 3. Farthest conflict queen at each Mark 25 26 Consistency check (BM) Algorithm backmarking procedure BM(Unlabeled, Labeled, Constraints, Level) Only the constraints where any value is changed are re-checked, and the farthest conflicting level is computed. if Unlabelled = {} then return Labeled pick first X from Unlabelled % fix order of variables for each value V from D do procedure consistent(X/V, Labeled, Constraints, Level) X if Mark(X,V) ≥ BackTo(X) then % re-check the value for each Y/VY/LY in Labeled such that LY≥BackTo(X) do if consistent(X/V, Labeled, Constraints, Level) then % only possible changed variables Y are explored R ← BM(Unlabeled-{X}, Labeled ∪{X/V/Level}, Constraints, Level+1) % in the increasing order of LY (first the oldest one) if R ≠ fail then return R % solution found if X/V is not compatible with Y/VY using Constraints then end if Mark(X,V) ← LY end if It is not necessary return fail It is not necessary toto test test it it again again end for end if (it(it is is satisfied) satisfied) BackTo(X) ← Level-1 % jump will be to the previous variable end for 2 for each Y in Unlabelled do % tell everyone about the jump Mark(X,V) ← Level-1 BackTo(Y) ← min {Level-1, BackTo(Y)} 5 6 return true 12 34 end for return fail % return to the previous variable end consistent 1 BackTo end BM 27 28 Incomplete search Incomplete search BBS(8) A cutoff limit to stop exploring a (sub-)tree some branches are skipped → incomplete search When no solution found, restart with enlarged cutoff limit. 123456789 DBS(3) Bounded Backtrack Search (Harvey, 1995) restricted number of backtracks Depth-bounded Backtrack Search (Cheadle et al., 2003) 1324 57 68 restricted depth where alternatives are explored IB(2) Iterative Broadening (Ginsberg and Harvey, 1990) restricted breadth in each node still exponential! 1324 57 68 CS(7) Credit Search (Beldiceanu et al., 1997) 43creditcredit 22 2 1 limited credit for exploring alternatives 1 1 1 1 1 1 credit is split among the alternatives 1324 57 6 29 30 5 Heuristics in search Discrepancies Observation 1: Discrepancy The search space for real-life problems is so huge that it cannot be fully explored. = the heuristic is not followed Heuristics - a guide of search Basic principles of discrepancy search: they recommend a value for assignment quite often lead to a solution change the order of branches to be explored What to do upon a failure of the heuristic? prefer branches with less discrepancies BT cares about the end of search (a bottom part of the search tree) heuristic = go left so it rather repairs later assignments than the earliest ones thus BT assumes that the heuristic guides it well in the top part is before Observation 2: The heuristics are less reliable in the earlier parts of the search prefer branches with earlier discrepancies tree (as search proceeds, more information is available). heuristic = go left Observation 3: is before The number of heuristic violations is usually small. 31 32 Discrepancy search Limited Discrepancy Search (Harvey & Ginsberg, 1995) restricts a maximal number of discrepancies in the iteration 1 5 4 3 2 10 9 8 7 6 Improved LDS (Korf, 1996) Consistency restricts a given number of discrepancies in the iteration Consistency 1 23 4 567 8 Depth-bounded Discrepancy Search (Walsh, 1997) restricts discrepancies till a given depth in the iteration 1 2 34 56 78 … * heuristic = go left 33 Introduction to consistencies Node consistency (NC) So far we used constraints in a passive way (as a test) … Unary constraints are converted into variables’ domains. …in the best case we analysed the reason of the conflict. Definition: Cannot we use the constraints in a more active way? The vertex representing the variable X is node consistent iff every value in the variable’s domain Dx satisfies all the unary constraints Example: imposed on the variable X. CSP is node consistent iff all the vertices are node consistent. A in 3..7, B in 1..5 the variables’ domains A values are consistent (for example A=4, B=4 is not consistent) delete V from DX end for How to remove the inconsistent values from the end for variables’ domains in the constraint network? end NC 35 36 6 Arc consistency (AC) Arc revisions Since now we will assume binary CSP only How to make (Vi,Vj) arc consistent? i.e. a constraint corresponds to an arc (edge) in the constraint network. Delete all the values x from the domain Di that are inconsistent with all the values in Dj (there is no value y Definition: in Dj such that the assignment Vi = x, Vj = y satisfies all The arc (Vi,Vj) is arc consistent iff for each value x from the domain Di the binary constrains on Vi a Vj). there exists a value y in the domain Dj such that the assignment Vi =x and Algorithm of arc revision Vj = y satisfies all the binary constraintson Vi, Vj. procedure REVISE((i,j)) Note: The concept of arc consistency is directional, i.e., arc consistency DELETED ← false of (Vi,Vj) does not guarantee consistency of (Vj,Vi). for each X in Di do CSP is arc consistent iff every arc (Vi,Vj) is arc consistent (in both if there is no such Y in D such that (X,Y) is consistent, i.e., directions). j (X,Y) satisfies all the constraints on Vi, Vj then delete X from Di Example: DELETED ← true TheThe procedure procedure also also end if reports the deletion A 37 38 Mackworth (1977) Algorithm AC-1 What is wrong with AC-1? How to establish arc consistency among the constraints? If a single domain is pruned then revisions of all the arcs are Doing revision of every arc is not enough! repeated even if the pruned domain does not influence most Example: X in [1,..,6], Y in [1,..,6], Z in [1,..,6], X 39 40 Mackworth (1977) Algorithm AC-2 Mackworth (1977) Algorithm AC-3 A generalised version of the Waltz’s labelling algorithm. Re-revisions can be done more elegantly than in AC-2. In every step, the arcs going back from a given vertex are 1) one queue of arcs for (re-)revisions is enough processed (i.e. a sub-graph of visited nodes is AC) 2) only the arcs affected by domain reduction are added Algorithm AC-2 to the queue (like AC-2) procedure AC-2(G) Algorithm AC-3 for i ← 1 to n do % n is a number of variables Q ← {(i,j) | (i,j)∈arcs(G), j 7 Looking for a support Computing support sets Observation (AC-3): A set of values supported by a given value (if the value disappears then Many pairs of values are tested for consistency in every these values lost one support), and a number of own supports are kept. arc revision. procedure INITIALIZE(G) These tests are repeated every time the arc is revised. Q ← {} , S ← {} % emptying the data structures for each arc (Vi,Vj) in arcs(G) do 1. When the arc V2,V1 is revised, the for each a in Di do a a 1 a total ← 0 × value a is removed from domain of V2. b b b for each b in Dj do c c c 2. Now the domain of V3, should be if (a,b) is consistent according to the constraint Ci,j then d d d 2 explored to find out if any value total ← total + 1 × S ← S ∪ { Using support sets Mohr, Henderson (1986) Algorithm AC-4 Situation: The algorithm AC-4 has optimal worst case time complexity O(ed2)! we have just processed the arc (i,j) in INITIALIAZE Algorithm AC-4 counter(i,j),_ i j Sj,_ 2 a1 b1 45 46 Other AC algorithms Directional arc consistency (DAC) AC-5 (Van Hentenryck, Deville, Teng, 1992) Observation 1: AC has a directional character but a CSP is not directional. generic AC algorithm covering both AC-4 and AC-3 AC-6 (Bessière, 1994) Observation 2: improves AC-4 by remembering just one support AC has to repeat arc revisions; the total number of revisions AC-7 (Bessière, Freuder, Régin, 1999) depends on the number of arcs but also on the size of domains (while cycle). improves AC-6 by exploiting symmetry of the constraint AC-2000 (Bessière & Régin, 2001) Is it possible to weaken AC in such a way that every arc an adaptive version of AC-3 that either looks for a support or is revised just once? propagates deletions AC-2001 (Bessière & Régin, 2001) Definition: improvement of AC-3 to get optimality (queue of variables) CSP is directional arc consistent using a given order of variables iff every arc (i,j) such that i 47 48 8 Algorithm DAC-1 How to use DAC? 1) Consistency of an arc is required just in one direction. AC visibly covers DAC (if CSP is AC then it is DAC as well) 2) Variables are ordered So, is DAC useful? ª there is no directed cycle in the graph! DAC-1 is surely much faster than any AC-x 4 6 1 there exist problems where DAC is enough 3 1 2 345 Claim: If the constraint graph forms a tree then DAC is enough to 5 2 If arcs are explored in a „good“ order, no revision solve the problem without backtracks. has to be repeated! How to order the vertices for DAC? Algorithm DAC-1 How to order the vertices for search? procedure DAC-1(G) 1. Apply DAC in the order from for j = |nodes(G)| to 1 by -1 do the root to the leaf nodes. for each arc (i,j) in G such that i Relation between DAC and AC Is arc consistency enough? Observation: By using AC we can remove many incompatible values CSP is arc consistent iff for some order of the variables, the Do we get a solution? problem is directional arc consistent in both directions. Do we know that there exists a solution? Is it possible to achieve AC by applying DAC in both Unfortunately, the answer to both above questions is NO! primal and reverse direction? Example: {1,2} In general NO, but … X X≠Z X≠Y CSP is arc consistent Example: but there is no solution Z X in {1,2}, Y in {1}, Z in {1,2}, X≠Z,Y Path consistency (PC) Relation between PC and AC How to strengthen the consistency level? Does PC subsume AC (i.e. if CSP is PC, is it AC as well)? More constraints are assumed together! the arc (i, j) is consistent (AC) if the path (i,j,i) is consistent (PC) thus PC implies AC Definition: The path (V0,V1,…, Vm) is path consistent iff for every pair of values x∈D a y∈D satisfying all the binary constraints on V ,V Is PC stronger than AC (is there any CSP that is AC but 0 m 0 m it is not PC)? there exists an assignment of variables V1,…,Vm-1 such that all the binary constraints between the neighbouring variables Vi,Vi+1 are Example: satisfied. X in {1,2}, Y in {1,2}, Z in {1,2}, X≠Z, X≠Y, Y≠Z it is AC, but not PC (X=1, Z=2 cannot be extended to X,Y,Z) CSP is path consistent iff every path is consistent. V4 V2 V3 V V1 Some notes: 0 ??? AC removes incompatible values from the domains, only the constraints between the neighboring what will be done in PC? variables must be satisfied PC removes pairs of values it is enough to explore paths of length 2 (Montanary, 1974) PC makes constraints explicit (A 53 54 9 Path revision Mackworth (1977) Algorithm PC-1 Constraints represented extensionally via matrixes. How to make the path (i,k,j) consistent? Path consistency is realized via matrix operations Rij ← Rij & (Rik * Rkk * Rkj) Example: How to make a CSP path consistent? Repeated revisions of paths (of length 2) while any domain changes. A,B,C in {1,2,3}, B>1 A 55 56 How to improve PC-1? Influenced paths Is there any inefficiency in PC-1? t Because Yji = Y ij it is enough to revise only the paths (i,k,j) where i≤j. just a few „bits“ Let the domain of the constraint (i,j) be changed when revising (i,k,j): it is not necessary to keep all copies of Yk one copy and a bit indicating the change is enough Situation a: i Z ← Yij &(Yik *Ykk *Ykj) Situation b: i=j If the domain is pruned if Z=Yij then return false If the domain is pruned all the paths containing i in the middle of the path are re-revised Y ← Z thenthen the the influenced influenced ij paths will be revised. but the paths (i,i,i) and (k,i,k) are not revised again return true paths will be revised. Sb = {(p,i,m) | 1 ≤ m ≤ n & 1 ≤ p ≤ m} - {(i,i,i),(k,i,k)} end REVISE_PATH | Sb | = n*(n-1)/2 - 2 57 58 Mackworth (1977) Algorithm PC-2 Other PC algorithms Paths in one direction only (attention, this is not DPC!) PC-3 (Mohr, Henderson 1986) After every revision, the affected paths are re-revised based on computing supports for a value (like AC-4) Algorithm PC-2 If the pair (a,b) at the arc (i,j) is not supported by procedure PC-2(G) another variable, then a is removed from Di and b is n ← |nodes(G)| removed from D . Q ← {(i,k,j) | 1 ≤ i ≤ j ≤ n & i≠k & j≠k} j while Q non empty do this algorithm is not sound! select and delete (i,k,j) from Q if REVISE_PATH((i,k,j)) then PC-4 (Han, Lee 1988) Q ← Q ∪ RELATED_PATHS((i,k,j)) correction of the PC-3 algorithm end while end PC-2 based on computing supports of pairs (b,c) at arc (i,j) PC-5 (Singh 1995) uses the ideas behind AC-6 procedure RELATED_PATHS((i,k,j)) if i 59 60 10 Drawbacks of PC Half way between AC and PC memory consumption Can we make a consistency algorithm: because PC eliminates pairs of values, we need to keep all the compatible pairs extensionally, e.g. using {0,1}-matrix stronger than AC, without drawbacks of PC (memory consumption, bad ratio strength/efficiency changing the constraint network)? PC removes more (or same) inconsistencies than AC, but the strength/efficiency ratio is much worse than for AC Restricted path consistency (Berlandier 1995) modifies the constraint network based on AC-4 (uses the support sets) PC adds redundant arcs (constraints) and thus it changes as soon as a value has only one support in another connectivity of the constraint network variable, PC is evoked for this pair of values this complicates using heuristics derived from the structure of the e f constraint network (like density, graph width etc.) e f × PC is still not a complete technique ≠ 1,2,3 1,2,3 a A,B,C,D in {1,2,3} a × c ≠ ≠ ≠ c b A≠B, A≠C, A≠D, B≠C, B≠D, C≠D ≠ b d is PC but has no solution × 1,2,3 ≠ 1,2,3 d 61 62 k-consistency Strong k-consistency = = Is there a common formalism for AC and PC? 1,2 1,2 1,2,3 3-consistent graph AC: a value is extended to another variable = but not 2-consistent graph! PC: a pair of values is extended to another variable … we can continue Definition: CSP is strongly k-consistent iff it is j-consistent for every j≤k. Definition: Visibly: strong k-consistency ⇒ k-consistency CSP is k-consistent iff any consistent assignment of Moreover: strong k-consistency ⇒ j-consistency ∀j≤k (k-1) different variables can be extended to a consistent In general: ¬ k-consistency ⇒ strong k-consistency assignment of one additional variable. ≠ NC = strong 1-consistency = 1-consistency AC = (strong ) 2-consistency 1,2,3 ≠≠≠1,2,3 1,2,3 4 PC = (strong ) 3-consistency sometimes we call NC+AC+PC together strong path consistency ≠ 4-consistent graph 63 64 What k-consistency is enough? Think globally Assume that the number of vertices is n. What level of CSP describes the problem locally: consistency do we need to find out the solution? the constraints restrict small sets of variables Strong n-consistency for graphs with n vertices! + heterogeneous real-life constraints n-consistency is not enough - see the previous example - missing global view strong k-consistency where k local (arc) consistency deduces And what about this graph? X1 a b ≠ no pruning < ≠ (D)AC is enough! but some values can be XXa b c X3 1,2 = 1,2,3 1,2,3 1,2,3 Because this a tree.. ≠ removed X2 a b < 65 66 11 Régin (1994) Inside all-different a set of binary inequality constraints among all variables X1 ≠ X2, X1 ≠ X3, …, Xk-1 ≠ Xk all_different({X1,…, Xk}) = {( d1,…, dk) | ∀i di∈Di & ∀i≠jdi ≠ dj} better pruning based on matching theory over bipartite graphs X a Initialisation: 1 1) compute maximum matching SearchSearch and and X2 b 2) remove all edges that do not × × belong to any maximum matching X c PropagationPropagation 3 Propagation of deletions (X ≠a): X × a 1 1 1) remove discharged edges 2) compute new maximum matching X2 × b 3) remove all edges that do not X3 c belong to any maximum matching 67 How to solve CSPs? Core search procedure - DFS So far we have two separate methods: The basic constraint satisfaction technology: depth-first search label the variables step by step complete (finds a solution or proves its non-existence) the variables are marked by numbers and labelled in a given order too slow (exponential) ensure consistency after variable assignment explores “visibly” wrong valuations A skeleton of search procedure consistency techniques procedure Labelling(G) usually incomplete (inconsistent values stay in domains) return LBL(G,1) end Labelling pretty fast (polynomial) Share advantages of both approaches - combine them! procedure LBL(G,cv) if cv>|nodes(G)| then return nodes(G) label the variables step by step (backtracking) for each value V from Dcv do maintain consistency after assigning a value if consistent(G,cv) then R ← LBL(G,cv+1) Do not forget about traditional solving techniques! if R ≠ fail then return R end if Linear equality solvers, simplex … end for A „hook“ for consistency such techniques can be integrated to global constraints! return fail procedure There is also local search. end LBL 69 70 Look back techniques Forward checking “Maintain” consistency among the already labelled variables. It is better to prevent failures than to detect them only! „look back“ = look to already labelled variables Consistency techniques can remove incompatible values for future What’s result of consistency maintenance among labelled variables? (=not yet labelled) variables. a conflict (and/or its source - a violated constraint) Forward checking ensures consistency between the currently labelled Backtracking is the basic look back method. variable and the variables connected to it via constraints. Backward consistency checks Forward consistency checks procedure AC-BT(G,cv) procedure AC-FC(G,cv) Q ← {(Vi,Vcv) in arcs(G),i select and delete any arc (Vk,Vm) from Q select and delete any arc (Vk,Vm) from Q consistent ← not REVISE(Vk,Vm) if REVISE(Vk,Vm) then When a value is deleted, end while When a value is deleted, consistent ← not empty Dk return consistent thethe domain domain is is empty empty end if Empty domain implies end AC-BT end while Empty domain implies inconsistencyinconsistency return consistent Backjumping & comp. uses information about the violated constraints. end AC-FC 71 72 12 Partial look ahead Full look ahead We can extend the consistency checks to more future variables! Knowing more about far future is an advantage! The value assigned to the current variable can be propagated to all Instead of DAC we can use a full AC (e.g. AC-3). future variables. Full look ahead consistency checks Partial lookahead consistency checks procedure AC3-LA(G,cv) procedure DAC-LA(G,cv) Q ← {(Vi,Vcv) in arcs(G),i>cv} % start with arcs going to cv for i=cv+1 to n do consistent ← true for each arc (Vi,Vj) in arcs(G) such that i>j & j≥cv do while consistent & Q non empty do select and delete any arc (V ,V ) from Q if REVISE(Vi,Vj) then k m if empty D then return fail if REVISE(Vk,Vm) then i Q Q {(V ,V ) | (V ,V ) in arcs(G),i k,i m,i>cv} end for ← ∪ i k i k ≠ ≠ consistent not empty D end for ← k end if return true end while end DAC-LA return consistent Notes: end AC3-LA In fact DAC is maintained (in the order reverse to the labelling order). Notes: Partial Look Ahead or DAC - Look Ahead The arcs going to the current variable are checked exactly once. It is not necessary to check consistency of arcs between the future The arcs to past variables are not checked at all. variables and the past variables (different from the current variable)! It is possible to use other than AC-3 algorithms (e.g. AC-4) 73 74 Comparison Constraint propagation at glance 4 queens Past (already labelled) variablescv Future (free) variables Backtracking is not very good 19 attempts 1 2 3 456 7 8 backtracking forward checking look ahead Propagating through more constraints remove more inconsistencies (BT < FC < PLA < LA), of course it increases complexity of the step. Forward checking is better Forward Checking does no increase complexity of backtracking, the 3 attempts constraint is just checked earlier in FC (BT tests it later). And the winner is Look Ahead When using AC-4 in LA, the initialisation is done just once. 2 attempts Consistency can be ensured before starting search Algorithm MAC (Maintaining Arc Consistency) AC is checked before search and after each assignment It is possible to use stronger consistency techniques (e.g. use them once before starting search). 75 76 Consistency and Search Variable ordering Variable ordering in labelling influence significantly efficiency of Consistency techniques are (usually) incomplete. solvers (e.g. in a tree-structured CSP). ª We need a search algorithm to resolve the rest! What variable ordering should be chosen in general? FAIL-FIRST principle Labeling „select the variable whose instantiation will lead to a failure“ depth-first search it is better to tackle failures earlier, they can be become even harder assign a value to the variable prefer the variables with smaller domain (dynamic order) propagate = make the problem locally consistent a smaller number of choices ~ lower probability of success backtrack upon failure the dynamic order is appropriate only when new information appears during solving (e.g., in look ahead algorithms) X in 1..3 ≈ X=1 ∨ X=2 ∨ X=3 (enumeration) „solve the hard cases first, they may become even harder later“ prefer the most constrained variables In general, search algorithm resolves remaining disjunctions! it is more complicated to label such variables (it is possible to X=1 ∨ X≠1 (step labeling) assume complexity of satisfaction of the constraints) X<3 ∨ X≥3(bisection) this heuristic is used when there is an equal size of the domains X a static heuristic that is useful for look-back techniques 77 78 13 Backtrack-free search Value ordering Definition: Order of values in labelling influence significantly efficiency (if we choose the CSP is solved using backtrack-free search if for some order of right value each time, no backtrack is necessary). variables we can find a value for each variable compatible with the What value ordering for the variable should be chosen in general? values of already assigned variables. SUCCEED FIRST principle < „prefer the values belonging to the solution“ = 1, 2 1, 2 1, 2, 3 1, 2, 3 if no value is part of the solution then we have to check all values 1 2 34 < if there is a value from the solution then it is better to find it soon Note: SUCCEED FIRST does not go against FIRST-FAIL ! How to find out a sufficient consistency level for a given graph? prefer the values with more suppors Some observations: this information can be found in AC-4 variable must be compatible with all the “former” variables prefer the value leading to less domain reduction i.e., across the „backward“ edges this information can be computed using singleton consistency for k „backward“ edges we need (k+1)-consistency let m be the maximum of backward edges for all the vertices, prefer the value simplifying the problem then strong (m+1)-consistency is enough solve approximation of the problem (e.g. a tree) the number of backward edges is different for different variable Generic heuristics are usually too complex for computation. order It is better to use problem-driven heuristics that propose the value! of course, the order minimising m is looked for 79 80 Golomb ruler A ruler with M marks such that distances between any two marks are different. The shortest ruler is the optimal ruler. ExampleExample 01 4 9 11 Hard for M≥16, no exact algorithm for M ≥ 24! Applied in radioastronomy. Solomon W. Golomb Professor University of Southern California http://csi.usc.edu/faculty/golomb.html 82 Golomb ruler Golomb ruler CSP model some results A base model: What is the effect of different constraint models? Variables X1, …, XM with the domain 0..M*M size base model base model base model X = 0 ruler start + symmetry + symmetry 1 + implied constraints X1< X2<…< XM no permutations of variables 7 220 80 30 8 1 462 611 190 ∀i 14 Constraint solvers It is not necessary to program all the presented techniques from scratch! Use existing constraint solvers (packages)! provide implementation of data structures for modeling variables’ domains and constraints provide a basic consistency framework (AC-3) provide filtering algorithms for many constraints (including ConclusionsConclusions global constraints) provide basic search strategies usually extendible (new filtering algorithms, new search strategies) Some systems with constraint satisfaction packages: Prolog: CHIP, ECLiPSe, SICStus Prolog, Prolog IV, GNU Prolog, IF/Prolog C/C++: CHIP++, ILOG Solver Java: JCK, JCL, Koalog Mozart 86 Resources Summary Books Constraints P. Van Hentenryck: Constraint Satisfaction in Logic Programming,MIT arbitrary relations over the problem variables Press, 1989 express partial local information in a declarative way E. Tsang: Foundations of Constraint Satisfaction, Academic Press, 1993 Basic constraint satisfaction framework: K. Marriott, P.J. Stuckey: Programming with Constraints: An local consistency connecting filtering algorithms for individual constraints Introduction, MIT Press, 1998 depth-first search resolves remaining disjunctions T. Frühwirth, S. Abdennadher: Essentials of Constraint Programming, Springer Verlag, 2003 local search can also be used R. Dechter: Constraint Processing, Morgan Kaufmann, 2003 Problem solving using constraints: Journal declarative modeling of problems as a CSP Constraints, An International Journal. Kluwer Academic Publishers (Springer) dedicated algorithms can be encoded in constraints special search strategies On-line materials On-line Guide to Constraint Programming (tutorial) It is easy to state combinatorial problems in terms of a CSP http://kti.mff.cuni.cz/~bartak/constraints/ … but it is more complicated to design solvable models. Constraints Archive (archive and links) http://4c.ucc.ie/web/archive/index.jsp We still did not reach the Holy Grail of computer programming (the user Constraint Programming online (community web) states the problem, the computer solves it) but CP is close. http://www.cp-online.org/ 87 88 15