From: AAAI Technical Report FS-01-04. Compilation copyright © 2001, AAAI (www.aaai.org). All rights reserved. Local Search and Backtracking vs Non-Systematic Backtracking Steven Prestwich Department of Computer Science University College, Cork, Ireland s. prestwich@cs,ucc. ie Abstract exploit structure, and is quite unsuitable for certain prob- lems. This situation has motivated research into hybrid ap- This paperaddresses the followingquestion: what is the es- proaches, with the aim of combiningfeatures of both classes sential differencebetween stochastic local search (LS)and systematicbacktracking (BT) that gives LSsuperior seal- of algorithmin order to solve currently intractable problems. ability? Onepossibility is LS’s lack of firm commitment Whatis the essential difference betweenlocal search and to any variable assignment.Three BT algorithms are mod- backtracking, that sometimesenables local search causes to ified to have this feature by introducingrandomness into scale far better than backtracking? Answeringthis question the choiceof backtrackingvariable: a forwardchecker for may lead to interesting new hybrids. The issue was de- n-queens, the DSATURgraph colouring algorithm, and a bated in (Freuder et al. 1995) and somesuggestions were Davis-Logemann-Lovelandprocedure for satisfiability. In local search’s locality, randomness,incompleteness, differ- each case the modifiedalgorithm scales like LSand some- ent search space, and ability to follow gradients. Wepropose timesgives better results. It is arguedthat randomisedback- that the essential difference is backtrackers’ strong commit- trackingis a formof local search. ment to early variable assignments,whereas local search can in principle (given sufficient noise) reassign any variable Introduction any point during search. Thoughthis seems rather an obvi- ous point, it has led to a newclass of algorithms that some- A variety of algorithms have been devised to solve constraint times out-performboth backtracking and local search. satisfaction and optimisation problems, two contrasting ap- proaches being systematic backtrackingand stochastic local Non-systematic backtracking search. Backtrackers are complete and can therefore prove unsolvability or find all solutions. Theycan also prove the The new approach has already been described in (Prest- optimality of a solution to an optimisation problemby fail- wich 2000a; Prestwich 2000b; Prestwich 2001a; Prestwich 2001b) but it is summarisedhere. It begins like a standard ing to find a better solution. They often use techniques backtrackerby selecting a variable using a heuristic, assign- such as constraint propagation, value and variable order- ing heuristics, branch-and-boundand intelligent backtrack- ing a value to it, optionally performingconstraint propaga- ing. Backtrackers are sometimescalled constructive because tion, then selecting another variable; on reaching a dead-end they construct solutions from partial consistent assignments (a variable that cannot be assigned any value without con- of domainvalues to variables. In contrast, local search algo- straint violation or, whereconstraint propagation is used, rithms are typically non-constructive, searching a space of domainwipe-out) it backtracks then resumes variable selec- total but inconsistent assignments.They attempt to minimise tion. The novel feature is the choice of backtracking vari- able: the variables selected for unassignmentare chosen ran- constraint violations, and whenall violations have been re- moveda solution has been found. domly, or using a heuristic with randomtie-breaking. As Neither backtracking nor local search is seen as adequate in DynamicBacktracking (Ginsberg 1993) only the selected for all problems¯ Real-world problems may have a great variables are unassigned, without undoinglater assignments. deal of structure and are often solved most effectively by Because of this resemblance we call the approach Incom- backtrackers, which are able to exploit structure via con- plete DynamicBacktracking (IDB). The search is no longer straint propagation. Unfortunately, backtrackers do not al- complete, so we must sometimesforce the unassignment of ways scale well to very large problems, even when aug- morethan one variable. Wedo this by adding an integer pa- mentedwith powerfulconstraint propagation and intelligent rameter 13 _> 1 to the algorithm and unassigning B variables backtracking. For large problemsit is often moreefficient at each dead-end. to use local search, whichtends to have superior scalabil- A complication arises when we wish to combine IDB ity. However,non-constructive local search cannot fully with forward checking. Suppose we have reached a dead- end after assigning variables vl ... Vk, and vk+l ... vn re- Copyright(~) 2001,American Association for Artificial Intelli- main unassigned. Wewould like to unassign some arbi- gence(www.aaai.org). All rights reserved. trary variable v~, where 1 _< u _< k, leaving the domains 109 in the state they would have been in had we assigned only n=lO n = 100 n = 1000 vl...v~,-1,vu+l...Vk. Howcan we do this efficiently? alg steps succ steps succ steps succ One way to characterise forward checking is as follows: a DFS1 81.0 100% 9929 value x is in the domainof a currently unassignedvariable v DFS2 25.4 100% 7128 39% 98097 3% if and only if addingthe assignmentv = x wouldnot cause a DFS3 14.7 100% 1268 92% 77060 24% constraint violation. In backtrackingalgorithms this is often IDB1 112 100% 711 100% 1213 100% used to update the domains of unassigned variables as as- IDB2 33.0 100% 141 100% 211 100% signments are added and removed. Wegeneralise this idea IDB3 23.8 100% 46.3 100% 41.2 100% by associating with each value x in the domainof each vari- IDB4 13.0 100% 8.7 100% 13.3 100% able v a conflict count Cv,x. The value of Cv,x is the number IDB5 12.7 100% 8.0 100% 12.3 100% of constraints that wouldbe violated if the assignmentv = x MCHC 57.0 100% 55.6 100% 48.8 100% wereadded. If Cv,x = 0 then x is currently in the domainof MCBT 46.8 100% 25.0 100% 30.7 100% v. Wemake a further generalisation by maintaining the con- flict counts of all variables, assigned and unassigned. The Figure 1: Results on n-queens meaningof a conflict count for a currently assigned variable is: Cv,x is the numberof constraints that wouldbe violated if the variable v were reassigned to x. Nowon assigning (Minton et al. 1992) comparedthe performance of back- or unassigning a variable, we incrementally update conflict tracking and local search on n-queens problems up to n = counts in all other variable domains.This is clearly moreex- 106. They executed each algorithm 100 times for various pensive than standard forward checking, which only updates values of n, with an upper bound of 100n on the numberof the domainsof unassigned variables. But the state of each steps (backtracks or repairs), and reported the meannum- variable domainis nowindependent of the order in which ber of steps and the success rate as a percentage. Were- assignments were made, and we can unassign variables ar- produce the experiment up to n = 1000, citing their re- bitrarily. The scheme can also be extended to non-binary sults for the Min-Confiictshill climbing algorithm (denoted constraints (see below). here by MCHC)and a backtracker augmentedwith the Min- A value ordering heuristic has been found useful in sev- Conflicts heuristic (denoted by MCBT).We compute results eral applications. It reassigns each variable to its previ- for depth-first search (DFS1), DFS1 with forward checking ous assigned value wherepossible, otherwise to a different (DFS2), and DFS2with dynamicvariable ordering based value. This speeds up the rediscovery of consistent partial minimumdomain size (DFS3). In DFS1and DFS2the vari- assignments. However,to avoid stagnation it attempts to use able to be assigned is selected randomly.(Our results differ a different (randomly-chosen)value for just one variable af- from those of Mintonet al., possibly because of algorithmic ter each dead-end. details such as randomtie-breaking during variable order- ing.) Wealso obtain results for these three algorithms with Experiments DFSreplaced by IDB(denoted by IDB1, IDB2, IDB3); also In this section IDBis applied to three problems: n-queens, for IDB3plus a backtracking heuristic that unassigns vari- graph colouring and satisfiability. Theexperiments are per- ables with maximumdomain size l (IDB4), and for IDB4 formed on a 300MHzDEC Alphaserver 1000A 5/300 under plus the value ordering heuristic described above (IDB5). Unix. For readers wishing to normalise our execution times ThelDB parameter is set to B = lforn = 1000 and to other platforms, the DIMACS(Johnson and Trick 1996) n = 100, and B = 2 for n = 10 (B = 1 sometimes causes benchmarkprogram dfmax r500.5 takes 46.2 seconds on the stagnation whenn = 10). Alphaserver. The results in Figure 1 show that replacing DFSby IDB greatly boosts scalability in three cases: the simple back- The n-queens problem tracking algorithm, backtracking with forward checking, Wefirst evaluate IDB on the n-queens problem. Though and the same algorithm with dynamic variable ordering. fashionable several years ago n-queensis no longer consid- Even the basic IDBalgorithm scales muchbetter than all ered a challenging problem. However,large instances still the DFS algorithms, and IDB3 performs like MCHC.The defeat mostbacktrackers and it is therefore of interest. Con- additional backtrackingand value ordering heuristics further sider a generalised chess board, which is a