University of Leeds

SCHOOL OF COMPUTER STUDIES

RESEARCH REPORT SERIES

Rep ort

A Tutorial on Constraint Programming

by

Barbara M Smith

Division of Articial Intelligence

April

Abstract

A constraint satisfaction problem CSP consists of a set of variables for

each variable a nite set of p ossible values its domain and a set of con

straints restricting the values that the variables can simultaneously take A

solution to a CSP is an assignment of a value from its domain to every vari

able in such a way that every constraint is satised Many problems arising

in OR in particular scheduling timetabling and other combinatorial prob

lems can b e represented as CSPs Constraint programming to ols now exist

which allow CSPs to b e expressed easily and provide standard strategies for

nding solutions This tutorial is intended to give a basic grounding in con

straint satisfaction problems and some of the algorithms used to solve them

including the techniques commonly used in constraint programming to ols

In particular it covers arc and path consistency simple backtracking and

forward checking as examples of search algorithms and the use of heuristics

to guide the search A simple example is considered in detail to show the

eect of dierent choices of formulation and search strategy Finding optimal

solutions to CSPs is also discussed

Intro duction

Constraint satisfaction problems CSPs have b een a sub ject of research in

AI for many years It has b een recognised that CSPs have practical sig

nicance b ecause many problems arising in OR in particular scheduling

timetabling and other combinatorial problems can b e represented as CSPs

This tutorial is intended to intro duce the research into CSPs which is neces

sary to make use of constraint programming to ols it is not intended to b e a

comprehensive survey of the eld

Briey a constraint satisfaction problem CSP consists of

a set of variables X fx x g

1 n

for each variable x a nite set D of p ossible values its domain

i i

and a set of constraints restricting the values that the variables can

simultaneously take

Note that the values need not b e a set of consecutive integers although

often they are they need not even b e numeric

A solution to a CSP is an assignment of a value from its domain to every

variable in such a way that every constraint is satised We may want to

nd

just one solution with no preference as to which one

all solutions

an optimal or at least a go o d solution given some ob jective function

dened in terms of some or all of the variables

As describ ed b elow solutions to CSPs are found by searching systemati

cally through the p ossible assignments of values to variables usually guided

by heuristics The reasons for cho osing to represent and solve a problem as

a CSP rather than say as a mathematical programming problem are two

fold Firstly the representation is often much closer to the original prob

lem the variables of the CSP directly corresp ond to problem entities and

the constraints can b e expressed without having to b e translated into lin

ear inequalities This makes the formulation simpler the solution easier to

understand and the choice of go o d heuristics to guide the solution strategy

more straightforward Secondly although CSP algorithms are essentially

very simple they can sometimes nd solutions more quickly than if integer

programming metho ds are used

In recent years with the development of constraint programming to ols it

has b ecome easier to express and solve constraint satisfaction problems The

rst commercial constraint programming to ols were those based on exten

sions of Prolog in particular CHIP Constraint Handling in Prolog Cur

rently ICLs DecisionPower incorp orates CHIP and it is also available from

Cosytec PrologI I I is an alternative constraint programming language based

on Prolog available from Prologia Constraint logic programming ie the

extension of logic programming languages like Prolog to supp ort the handling

of constraints is now a signicant area of research in the logic programming

community

Logic programming is not however an essential basis for constraint pro

gramming and other to ols have b een built notably Solver from ILOG which

is a C library It is worth mentioning that the French computer com

pany Bull which had its own constraint programming to ol called Charme

develop ed from CHIP has now abandoned this in favour of a version of

ILOG Solver Where necessary examples in this tutorial will refer to ILOG

Solver which is the to ol that I am most familiar with However examples

could equally well have b een presented in terms of CHIP or another to ol

Constraints

The constraints of a CSP are usually represented by an expression involving

the aected variables eg

x x

1 2

x x x

1 2 3

x x x

1 2 3

or in ILOG Solver

CtNeqxx

CtEqx x x

CtLtx x x

It is useful also to b e aware of the formal denition of a constraint a

p ossible constraint C b etween the variables x x x is any subset of

ij k i j k

1

the p ossible combinations of values of x x x The subset sp ecies the

i j k

combinations of values which the constraint allows

For example if variable x has the domain f g and variable y has the

domain f g then any subset of f g is a

valid constraint b etween x and y The constraint x y would b e represented

by the subset f g

Although the constraints of real problems are not represented this way in

practice the denition do es emphasise that constraints need not corresp ond

to simple expressions and in particular they need not b e linear inequalities

or equations although they can b e

A constraint can aect any numb er of variables from to n n is the

numb er of variables in the problem The numb er of aected variables is the

arity of the constraint It is useful to distinguish two particular cases

Unary constraints aect just one variable The constraint can b e used

to remove any value which do es not satisfy the constraint from the domain

of the variable at the outset For instance if there is a constraint x

1

the value can b e removed from the domain of x and the constraint will

1

then b e satised Since unary constraints are dealt with by prepro cessing

the domains of the aected variable they can b e ignored thereafter

Binary Constraints aect two variables If all the constraints of a CSP

are binary the variables and constraints can b e represented in a constraint

graph the no des of the graph represent the variables and there is an edge

joining two no des if and only if there is a constraint b etween the corresp ond

ing variables

1

The p ossible combinations of values of x x x are given by the crosspro duct of

i j k

the domains D  D  D so C is a p ossible constraint if

i j k ij k

C  D  D  D 

ij k i j k

In fact any constraint of higher arity can b e expressed in terms of binary

constraints although in practice this is not likely to b e worth doing Hence

in some sense binary CSPs are representative of all CSPs

An individual binary constraint b etween two variables with domain sizes

m and m can b e represented by an m m matrix of values where

1 2 1 2

signies that the constraint allows the corresp onding pair of values and

that it do es not Although this is not a practical way of dening real

constraints it means that a random matrix can represent a binary con

straint and this allows exp erimenters who need a large numb er of CSPs to

pro duce randomlygenerated problems The pairs of variables which have

constraints b etween them are also selected randomly describ es in more

detail two p ossible mo dels for randomlygenerated CSPs A great deal of

exp erimental work on CSPs comparing the p erformance of dierent algo

rithms for instance has b een done using p opulations of randomlygenerated

binary CSPs

Example

Cryptarithmetic puzzles like the following can b e expressed as CSPs Each

letter in the following sum stands for a dierent digit nd their values

D O N A L D

G E R A L D

R O B E R T

The variables are the letters D O N A L G E R B T and their

domains are the set of digits f g except that D G and R cannot b e

This expressed in ILOG Solver by creating a set of constrained integer

variables

CtIntVar DONAL

GERBT

It is convenient to form an array containing p ointers to these variables

CtIntVar AllVars

DONALGERBT

The constraints are

the ten variables must all b e assigned a dierent value Technically

this means that there is a notequal constraint b etween every pair of

variables giving binary constraints In practice to ols such as ILOG

Solver allow an easy way of stating that all variables in a sp ecied set

must set have dierent values

CtAllNeq AllVars

the sum given must work out

CtEqDONALD

GERALD

ROBERT

There are alternative ways of formulating this problem which are b etter

from the p oint of view of nding a solution This will b e discussed later

Arc Consistency

If there is a binary constraint C b etween the variables x and x then the

ij i j

2

arc x x is arc consistent if for every value a D there is a value b D

i j i j

such that the assignments x a and x b satisfy the constraint C

i j ij

Any value a D for which this is not true ie no such value b exists can

i

safely b e removed from D since it cannot b e part of any consistent solution

i

removing all such values makes the arc x x arc consistent Note that we

i j

have only checked the values of x there may still b e values in the domain

i

of x which could b e removed if we reverse the op eration and make the arc

j

x x arc consistent

j i

Figure a shows the original domains of x and y In b x y has

b een made arc consistent in c b oth x y and y x have b een made arc

consistent

Note that if a binary constraint is represented by a matrix in the manner

describ ed earlier making the constraint arc consistent in b oth directions

eectively removes any values from the domains of the two variables for

which the corresp onding row or column in the matrix is all zeros

If every arc in a binary CSP is made arc consistent then the whole

problem is said to b e arc consistent Making the problem arc consistent is

2

The arc x x has a direction attached to it so that it is distinct from the arc x x

i j j i

The edge joining x and x on the other hand is undirected

i j x x < y - 2 y

{1..5} {1..5} (a)

x x < y - 2 y

{1,2} {1..5} (b)

x x < y - 2 y

{1,2} {4,5}

(c)

Figure Making x y and y x arc consistent

often done as a prepro cessing stage reducing the sizes of some domains

should make the problem easier to solve

A numb er of algorithms for making a CSP arc consistent have b een pro

p osed the most commonlyused ones app ear to b e AC and AC although

variations up to at least AC exist each supp osedly b etter than its prede

2

cessors in some way AC has worst case time complexity O d c where d

is the maximum domain size and c is the numb er of binary constraints the

2

space complexity is similarly O d c

Since it is reasonably cheap to make a problem or just the arcs cor

resp onding to binary constraints arc consistent constraint programming

to ols normally include an arc consistency algorithm In ILOG Solver for

instance arc consistency is established as constraints are dened and main

tained whenever any change is made eg to the domains of variables This

is done automatically

Example Supp ose we have a set of activities each with a sp ecied

duration There are precedence constraints b etween the activities so that if

task A precedes task B then

startA durationA startB

Task Duration Precedes

A BC

B D

C D

D

We can express this as a binary CSP by intro ducing variables representing

the start time of each activity and others representing the start and nish of

the pro ject Since the pro ject need not take longer than the total duration

of all the activities this gives a p ossible limit for each variable

Variable Initial domain

start fg

startA fg

startB fg

startC fg

startD fg

nish fg

Arc consistency reduces the domains of the variables as shown b elow

Any value remaining in any variables domain is part of a consistent solution

to the whole problem which will allow the pro ject to nish by time as

shown in Figure

startB

{3..7}

start startA startD finish

{0} {0..2} {7..9} {9..11}

startC

{3..5}

Figure Simple pro ject network

Note that if we want to minimise the pro ject completion time and now

set finish to its minimum p ossible value further values will b e removed

from the domains This is equivalent to the Critical Path Metho d

Hence in a pro ject planning problem with precedence constraints only

and no resource constraints for example arc consistency is sucient by

itself to remove all values from the domains which cannot b e part of any

solution This is not generally true and in order to nd a solution if there

is one we need to search for one

Path Consistency and Beyond

In Figure the problem is arc consistent but it is clear that the variable x

cannot have the value In general even when a problem has b een made y {4,5}

x < y-2 y < z

z >= 5x x z {1,2} {6..10}

w >= 5x w+z < 20

w

{6..10}

Figure An arc consistent constraint network

arc consistent it is p ossible to make further deductions from the constraints

short of searching for a complete solution The next step still with binary

constraints is to consider triples of variables in which two pairs of variables

have a nontrivial constraint b etween them A trivial constraint here is one

that allows every pair of values

xj

x x

i k

Figure A triangle of constraints

In Figure supp ose there are nontrivial constraints b etween x and x

i j

and b etween x and x

j k

The path x x x is path consistent i for every pair of values v

i j k i

D and v D allowed by the constraint C there is a value v D

i k k ij j j

such that v v C and v v C If there is no such value v

i j ij j k j k j

then v v should b e removed from the constraint C ie the constraint

i k ik

should b e tightened In other words if no value can b e found for x which

j

is simultaneously consistent with x v and x v then we cannot allow

i i k k

these values to b e simultaneously assigned to x and x

i k

In the example of Figure making x w z path consistent would show

that the constraint b etween x and z has to b e tightened to exclude the

simultaneous assignment of x and z making the problem arc

consistent again would show that the value must b e removed from the

domain of x So in this case path consistency would allow the problem to

b e considerably simplied

It is of course p ossible to check every triple of variables in a binary CSP

and tighten the constraints where p ossible Clearly the numb er of p ossible

triples is greater than the numb er of pairs of variables that need to b e checked

to make the problem arc consistent and the b est algorithm has worst case

3 3

time complexity O d n This is one reason why path consistency algorithms

are not in common use Another is that since constraints are not generally

expressed as allowed tuples of values it is not easy to remove individual pairs

of values in order to tighten a binary constraint

In practice the constraints required for path consistency are often easy

to see when the problem is formulated For instance in the triangle of

constraints in Figure path consistency shows that the constraint b etween

x and z must b e x z a constraint such as this which must eectively

exist but may not b e sp ecied in the original statement of the problem is

called an induced constraint See for further discussion of this

y

x y y z

x z

Figure Example of an induced constraint b etween x and z

It is also p ossible to consider groups of or more variables and attempt to

induce new constraints which will in general b e nonbinary but this is still

more timeconsuming in practice prepro cessing rarely go es b eyond making

a problem arc consistent

Search Algorithms

Most algorithms for solving CSPs search systematically through the p ossible

assignments of values to variables Such algorithms are guaranteed to nd

a solution if one exists or to prove that the problem is insoluble but they

may take a very long time to do so

This pap er is not intended to b e a comprehensive survey of constraint sat

isfaction algorithms I shall consider only two systematic search algorithms

a simple backtracking algorithm and the forward checking algorithm which

is the default search algorithm in the constraint programming to ols that I

am aware of

Both algorithms instantiate ie assign a value to each variable in turn

and build up a partial solution consisting of the variables already considered

with their assigned values these are termed the past variables The variables

which are so far uninstantiated are the future variables

In the backtracking algorithm the current variable is assigned a value

from its domain This assignment is then checked against the current partial

solution if any of the constraints b etween this variable and the past variables

is violated the assignment is abandoned and another value for the current

variable is chosen If all values for the current variable have b een tried the

algorithm backtracks to the previous variable and assigns it a new value If a

complete solution is found ie a value has b een assigned to every variable

the program may terminate if only one solution is required or carry on to

nd new solutions If there are no solutions the algorithm terminates when

all p ossibilities have b een considered

If the CSP has n variables each with m p ossible values the maximum

depth of any branch in the search tree is n and up to m branches are cre

n

ated from each no de There are up to m p ossible assignments of values to

variables many of which will not b e allowed by the constraints of course

n

Hence the tree has up to m leaf no des

An example of a search tree built by the backtracking algorithm is shown

in Figure using the queens problem The nqueens problem requires

placing n queens on an n n chessb oard in such a way that no queen can

take any other hence no two queens can b e on the same row the same column

or the same diagonal of the b oard As a CSP this problem has variables

representing the rows of the chessb oard and each variable has domain fg

representing the columns However it is easier to follow the progress of the

search if the chessb oard representation is used a Q on a particular square

should b e taken as meaning that the variable corresp onding to that row has

b een assigned the value corresp onding to that column Deadends where the

algorithm has to backtrack to a previous choice are marked by crosses and

the solution eventually found is marked by a tick

The backtracking algorithm only checks the constraints b etween the cur

rent variable and the past variables The forward checking algorithm on the

other hand checks the constraints b etween the current and past variables

and the future variables When a value is assigned to the current variable

any value in the domain of a future variable which conicts with this assign

ment is temp orarily removed from the domain The advantage of this is

that if the domain of a future variable b ecomes empty it is known immedi Q Q

Q Q Q Q QQQ Q QQQ Q QQQQ

QQQQQQQQ Q Q Q Q QQQ Q Q Q Q Q QQQQQQ Q

Q Q Q Q Q Q Q QQQQQQQ QQQQQQQ

Q Q Q Q Q Q Q

Figure Search tree for queens using simple backtracking

ately that the current partial solution is inconsistent and as b efore either

another value for the current variable is tried or the algorithm backtracks to

the previous variable the state of the domains of future variables as they

were b efore the assignment which led to failure is restored With simple

backtracking this failure would not have b een detected until the future vari

able was considered and it would then have b een discovered that none of its

values were consistent with the current partial solution Forward checking

therefore allows branches of the search tree that will lead to failure to b e

pruned earlier than with simple backtracking

This can again b e illustrated using the queens problem If we start by

placing a queen on the rst row then none of the other queens can b e placed

in the same column or on the same diagonal and the values corresp onding

to the squares attacked by this queen can b e removed from the domains of

the variables representing the queens in rows to unless and until this

branch leads to a dead end and the rst row queen has to b e moved The

full search tree built by forward checking for this problem is shown in Figure

Squares with crosses denote values removed from the domains of future

variables by the past and current assignments

Note that whenever a new variable is considered all its remaining values

are guaranteed to b e consistent with the past variables so that checking

an assignment against the past assignments is no longer necessary If all

the constraints are binary then only the constraints b etween the current Q Q

Q Q Q Q Q Q

Q Q Q Q Q Q

Q Q Q

Q

Figure Search tree for queens using forward checking

variable and future variables need b e checked it is only if there are constraints

of higher arity that the past variables may need to b e involved A k ary

constraint k needs to b e checked if and only if the current variable

is one of the variables involved in the constraint and all but one of the

other variables in the constraint have already b een instantiated the past

instantiations then eectively reduce the constraint to a binary constraint

b etween the current variable and a future variable

Forward checking do es more work when each assignment is added to the

current partial solution in order to reduce the size of the search tree and

so hop efully reduce the overall amount of work done In fact in queens

forward checking do es ab out the same amount of work as the backtracking

algorithm in checking for consistency there is not enough scop e in such a

small search tree for early pruning of large branches However the follow

ing articial example will show that forward checking can save an arbitrary

amount of work compared with simple backtracking supp ose we have vari

ables x x x x where x x x all have domain fg and the con

1 2 3 n 1 2 n

straints on these three variables are that they should all have dierent values

Clearly this subproblem and thus the whole problem is infeasible It do es

not matter what the domains of the remaining variables x x are or

3 n1

what the constraints on these variables are we assume that this part of the

problem can b e solved without any diculty The backtracking algorithm

will instantiate variables x and x to and resp ectively and then assign

1 2

values to x x in turn b efore discovering that there is no value for x

3 n1 n

which is consistent with the rst two assignments It will then backtrack to

x and try all the alternative assignments for this variable then backtrack

n1

to x and so on even though these variables are not part of the subprob

n2

lem which is causing the diculty It could take a very long time to discover

that the problem has no solution Forward checking on the other hand

will discover that there is no remaining value in the domain of x as so on

n

as values have b een assigned to x and x it will never consider assigning

1 2

values to the remaining variables Forward checking is not the only way of

avoiding this kind of stupidity and it can get into diculties itself on the

whole however it p erforms reasonably well compared with other algorithms

when combined with go o d heuristics as describ ed in the next section and it

is almost always a much b etter choice than simple backtracking

A search strategy which do es still more work in lo oking ahead when an

assignment is made combines forward checking with maintaining arc consis

tency Whenever a new subproblem is created by removing values from the

domains of future variables which are inconsistent with the current assign

ment the subproblem is made arc consistent This will remove further values

from the domains of future variables and as with forward checking itself the

hop e is that in doing additional work at the time of the assignment there

will b e an overall timesaving Forward checking combined with maintaining

arc consistency is the default algorithm used in ILOG Solver for instance

Variable Ordering

A tree search algorithm for constraint satisfaction requires the order in which

variables are to b e considered to b e sp ecied Even if no particular order

is sp ecied the algorithm must have some default ordering to fall back on

probably the order in which the variables were dened The ordering may b e

either a static ordering in which the order of the variables is sp ecied b efore

the search b egins and is not changed thereafter or a dynamic ordering in

which the choice of next variable to b e considered at any p oint dep ends on

the current state of the search

Dynamic ordering is not feasible for all tree search algorithms for in

stance with simple backtracking there is no extra information available dur

ing the search that could b e used to make a dierent choice of ordering from

the initial ordering However with forward checking the current state in

cludes the domains of the variables as they have b een pruned by the current

set of instantiations and so it is p ossible to base the choice of next variable

on this information

A common variable ordering heuristic is based on what Haralick and El

liott termed the failrst principle which they explained as To succeed

try rst where you are most likely to fail In forward checking this principle

is implemented by cho osing next the variable with fewest remaining values in

its domain on the assumption that any value is equally likely to participate

in a solution so that the more values there are the more likely it is that one

of them will b e a successful choice

Calling this the failrst principle is it seems to me slightly misleading

or at least onesided after all we do not want to fail so it seems at rst

sight p erverse to delib erately cho ose the variable that is most likely to lead

to failure The reasoning is that if the current partial solution will not lead

to a complete solution so that the current branch will eventually prove to b e

a dead end then the so oner we discover this the b etter Hence encouraging

early failure if failure is inevitable is b enecial in the long run On the

other hand if the current partial solution can b e expanded to a complete

solution then every remaining variable must b e instantiated and the one

with smallest domain is likely to b e the most dicult to nd a value for

instantiating other variables rst may further reduce its domain and lead to

a failure Hence the principle could equally well b e stated as Deal with hard

cases rst they can only get more dicult if you put them o So whether

the current partial solution will lead to a complete solution in which case

we want to get to the solution straightaway without further backtracking or

will not lead to a solution and we want to nd that out as so on as p ossible

cho osing the variable with smallest remaining domain makes sense

This heuristic should reduce the average depth of branches in the search

tree by triggering early failure Hence even if there is no solution so that

a complete search is required or if all solutions are required the size of the

search tree explored is less than if a static ordering is used

When all variables have the same numb er of values which is the case

in some problems at the start then the failrst principle indicates that we

should still try to cho ose the variable which is likely to b e most dicult

to instantiate and a go o d choice is the variable which participates in most

constraints in the absence of more sp ecic information on which constraints

are likely to b e dicult to satisfy for instance

A word of warning Most exp eriments with CSPs have b een done with

randomlygenerated problems in which every variable has the same domain

size initially and all constraints are equally dicult to satisfy For these

problems cho osing the variable with smallest domain works extremely well

For real problems to o it is often a go o d choice but sometimes needs a

little thought For instance describ es a rostering problem in which

the variables represent tasks and the values p eople who can do those tasks

Some of the variables have very small domains initially not b ecause they

are dicult to assign but b ecause they represent tasks which particular

individuals can do if there is nothing of higher priority available So in this

case cho osing the variable with smallest domain rst would b e wrong

Value Ordering

Having selected the next variable to assign a value to a search algorithm has

to select a value to assign As with variable ordering unless values are to

b e assigned simply in the order in which they app ear in the domain of each

variable we should decide how to cho ose the order in which values should b e

assigned A dierent value ordering will rearrange the branches emanating

from each no de of the search tree This is an advantage if it ensures that

a branch which leads to a solution is searched earlier than branches which

lead to dead ends provided that only one solution is required If all solutions

are required or if the whole tree has to b e searched b ecause there are no

solutions then the order in which the branches are searched is immaterial

Supp ose we have selected a variable to instantiate how should we cho ose

which value to try rst It may b e that none of the values will succeed

we are in fact exploring what will turn out to b e a dead end and we shall

have to backtrack to the previous variable In that case every value for the

current variable will eventually have to b e considered and the order do es

not matter On the other hand if we can nd a complete solution based on

the past instantiations we want to cho ose a value which will lead to such a

solution a go o d general principle then is to cho ose a value which is likely

to succeed and unlikely to lead to a conict if we can detect such a value

Some value ordering heuristics based on this principle have b een prop osed

for use with forward checking eg In b oth cases in order to select

a value for the current variable the state of the domain of future variables

which would result from each choice is found ie forward checking is done for

each value in turn Keng Yun suggest then calculating the p ercentage

of values in future domains which will no longer b e usable as a measure of

the cost of making this choice the b est choice would b e the value with lowest

cost Geelen suggests instead calculating the promise of each value that

is the pro duct of the domain sizes of the future variables after cho osing this

value this is an upp er b ound on the numb er of p ossible solutions resulting

from the assignment the value with highest promise should b e chosen

Unfortunately there is a great deal of work involved in forward checking

from each p ossible value in turn For randomlygenerated problems and

probably in general the work involved in assessing each value is not worth

the b enet of cho osing a value which will on average b e more likely to lead

to a solution than the default choice

In particular problems on the other hand there may b e information

available which allows the values to b e ordered according to the principle of

cho osing rst those most likely to succeed

Example

The cryptarithmetic problem discussed earlier was formulated with a single

constraint representing the required sum

CtEqDONALD

GERALD

ROBERT

This is sucient to allow ILOG Solver to nd a solution using its de

fault strategy of forward checking combined with arc consistency However

it takes a relatively long time The p erformance of the algorithm can b e

measured by the numb er of times it detects a failure and has to backtrack

With the constraint stated as ab ove it takes fails to nd the solution

The diculty lies in the formulation of the sum as a single constraint

involving all the variables If we tried solving the puzzle by hand considering

the sum as a whole in this way would not give an obvious p oint of attack

if we consider how forward checking works we can see that it also causes

diculties for the algorithm

The p ower of the forward checking algorithm lies in the fact that each

instantiation of a variable can b e used to reduce the domains of future vari

ables To ols such as Solver also continually make sure that the remaining

subproblem is arc consistent having reduced the domains of two future vari

ables which have a constraint b etween them arc consistency may show that

further reductions to the domain of one or other have b ecome necessary

Both of these metho ds of reducing future domains can only make use of

binary constraints or constraints which have eectively b ecome binary b e

cause all but two of the aected variables have already b een instantiated

The pruning eect of higher arity constraints therefore has to b e p ostp oned

until sucient variables have b een instantiated to make them into binary

constraints Hence the forward checking algorithm combined with main

taining arc consistency is b est at solving binary CSPs If there is a choice

b etween two formulations of a problem as a CSP b oth of which represent

problem entities in a natural way then in general one that has constraints of

3

low arity should b e chosen

With this in mind an alternative formulation representing the sum col

umn by column which has additional variables representing the quantities

carried into the next column is preferable

CtIntVar

CCCCC

CtEqD C T

CtEqL C C R

CtEqA C C E

CtEqN R C C B

CtEqE C C

CtEqD G C R

This nds the solution with fails a dramatic improvement

So far nothing has b een said ab out variable and value ordering heuristics

It is hard to see any reason for preferring one value to another so we shall not

sp ecify a value ordering for the variable ordering the heuristic of cho osing

the variables with smallest remaining domain from amongst the original

variables seems a go o d choice Note that if we cho ose the variable with

smallest remaining domain from amongst all the variables the carries would

b e chosen rst as they start o with smaller domains In this case cho osing

the carries rst gives go o d results but in general it may not b e a go o d idea to

allow subsidiary variables dened entirely in terms of the original variables

to drive the search strategy With this heuristic the solution is found with

only fails

This is still not the b est that can b e done however the nal improvement

seems to dep end on giving Solver a piece of additional information that it

cannot sp ot for itself It is clear from lo oking at the original sum that T must

b e even however arc consistency do es not allow Solver to eliminate the o dd

values from the domain of T If we do this by hand as well as making all the

previous mo dications the solution can b e found without any backtracking

3

It has b een mentioned that any CSP can in theory b e expressed as a binary CSP

However this is done by intro ducing new variables which represent tuples of the original

variables so that the new formulation is likely to b e much more cumb ersome to deal with

See for details

Symmetries

In many problems if there are any solutions at all there are classes of equiv

alent solutions For instance in timetabling problems it may b e p ossible to

interchange the allo cations to the time slots and still have a feasible solution

in rostering problems a group of sta may have the same skills and the same

availability and so b e interchangeable in the roster Such symmetries in the

problem may cause diculties for a search algorithm if the problem turns

out to b e insoluble or the algorithm is exploring a branch of the search tree

which do es not lead to a solution then all symmetrical assignments will b e

explored in turn This is a waste of eort b ecause if one such assignment is

infeasible then they all are Such symmetries should b e avoided if p ossible

by including additional constraints in the formulation which will allow only

one solution from each class of equivalent solutions

It is dicult to give general advice on how to do this b ecause it dep ends

on the particular problem As an example in the rostering problem we

could numb er the sta in the group and the tasks which the memb ers of the

group can b e assigned to in the rst timep erio d covered by the roster and

insist that if i j then the task assigned to p erson i must have a smaller

numb er than the task assigned to p erson j This means that the sta in the

group are no longer interchangeable thus ruling out the equivalent solutions

and avoiding doing unnecessary work For instance if there is no solution

with p erson assigned to task then there will b e no solution with p erson

assigned to task either For more information on avoiding symmetries

by adding constraints see An example of a problem with symmetries is

describ ed in

Optimization Problems

Constraint programming to ols adopt the same general approach to attempt

ing to nd an optimal solution create a constrained variable which repre

sents the ob jective function nd an initial solution then intro duce a new

constraint that the value of the ob jective variable must b e b etter than in the

initial solution Rep eatedly solve the new problem and tighten the constraint

on the ob jective variable in this way until the problem b ecomes insoluble

the last solution found is then the optimal solution

For instance ILOG Solver has a builtin function CtMinimize and an

other CtMaximise but we shall assume that we have a minimization problem

which can replace the usual function which nds just one solution and takes

an extra parameter which is the variable representing the ob jective function

say cost It is the programmers resp onsibility to see that cost is dened

in terms of the other variables in the problem and that whenever a solution

to the problem is found cost is thereby assigned a value Eectively what

then happ ens though this is taken care of internally by CtMinimize is that

whenever a new solution is found the value of cost is saved say in best

and a new constraint is added

cost best

The new problem is then solved and this is rep eated until the constraint on

cost has b een tightened to the extent that the problem cannot b e solved

the last constraint is then removed and the previous solution which is now

known to b e optimal is found again

This is clearly somewhat inecient in that the optimal solution is found

twice However it works well provided that the problem is small With

large problems as the constraint on cost gets tighter it can get extremely

dicult to nd a solution Whereas go o d heuristics may b e able to nd a

solution quickly if there are many p ossible solutions this gets more dicult

if there are few solutions and of course in order to prove that a problem has

no solutions to show that the last solution was optimal the entire search

tree must b e explored So for large problems it may not b e practicable to get

anywhere near the optimal solution still less to prove optimality This partly

dep ends on particular circumstances in some cases proving optimality is

straightforward b ecause reducing cost b elow its optimal value results in a

problem which is obviously infeasible Even then nding an optimal solution

may not b e easy

In many cases therefore it is necessary to rely on go o d heuristics and

allow the program to search for improvements on the initial solution for as

long as is practicable b efore accepting the current solution as hop efully

go o d enough The advantage in that case over heuristics which will simply

construct a solution is that the builtin backtracking may nd signicant

improvements over the initial solution b efore the search has to b e abandoned

Conclusions

It is probably clear from the foregoing that the search algorithms available

for solving CSPs are relatively unsophisticated compared for instance to

mathematical programming techniques On the other hand it is b ecause of

this that the constraints can b e much more expressive and therefore more

p owerful than is allowed in mathematical programming

Because algorithms like forward checking search systematically for solu

tions they are unlikely to b e able to handle large problems without go o d

heuristics to guide them Go o d variable and value ordering heuristics are

often crucial and can make the dierence b etween nding a solution very

quickly and failing to nd a solution at all

An area which needs further investigation is the comparison b etween

mathematical programming and constraint programming Integer program

ming problems can b e expressed as CSPs but when is it worthwhile to do

so In I describ e a problem where constraint programming succeeded in

nding a solution when integer programming had failed and discuss reasons

for the dierence in p erformance in this case

Further Reading

For a thorough overview of constraint satisfaction problems concentrating

esp ecially on search algorithms and achieving dierent levels of consistency

in CSPs see Edward Tsangs b o ok

Van Hentenrycks b o ok is sp ecically on CHIP and so contains a

go o d deal on how Prolog has b een extended to pro duce CHIP It do es also

however contain quite a lot of useful material on tackling sp ecic problems

which would apply to most constraint programming to ols Other useful pa

p ers on applying CHIP to sp ecic problems and comparing this approach

with traditional OR are and

Pap ers on Solver are available from the ILOG WorldWide Web site

httpwwwilogfrilogproductssolversolverhtml including an

overview of the system and the pap er on symmetries already mentioned

References

M Dincbas H Simonis and P van Hentenryck Solving a Cutting

Sto ck problem in constraint logic programming In R Kowalski and

K Brown editors Logic Programming pages

M Dincbas H Simonis and P van Hentenryck Solving the car

sequencing problem in constraint logic programming In Proceedings

ECAI pages

P A Geelen Dual Viewp oint Heuristics for Binary Constraint Satis

faction Problems In B Neumann editor Proceedings ECAI pages

R Haralick and G Elliott Increasing tree search eciency for con

straint satisfaction problems Articial Intel ligence

N Keng and D Y Yun A PlanningScheduling Metho dology for the

Constrained Resource Problem In Proceedings IJCAI pages

JF Puget On the Satisability of Symmetrical Constrained Satisfac

tion Problems In Proceedings of ISMIS

JF Puget A C Implementation of CLP In Proceedings of

SPICIS Singapore International Conference on Intel ligent Systems

B M Smith How to Solve the Zebra Problem or Path Consistency the

Easy Way In B Neumann editor Proceedings ECAI pages

A longer version is available as Research Rep ort Scho ol of

Computer Studies University of Leeds

B M Smith Lo cating the Phase Transition in Constraint Satisfaction

Problems Research Rep ort Scho ol of Computer Studies Univer

sity of Leeds May to app ear in Articial Intel ligence

B M Smith and S Bennett Combining Constraint Satisfaction and

Lo cal Improvement Algorithms to Construct Anaesthetists Rotas In

Proceedings of CAIA the th IEEE Conference on Articial Intel li

gence Applications pages Mar

B M Smith S C Brailsford P M Hubbard and H P Williams The

Progressive Party Problem Integer Linear Programming and Constraint

Programming Compared Research Rep ort Scho ol of Computer

Studies University of Leeds Mar

E Tsang Foundations of Constraint Satisfaction Academic Press

P van Hentenryck Constraint Satisfaction in Logic Programming MIT

Press