sign in sign up
<<
Home , Algorithm

Genetic for Combinatorial Optimization The Assembly

Line Balancing Problem

Michael Ferris Edward J Anderson

Computer Sciences Department University of Cambridge

University of Wisconsin Judge Institute of Management Studies

Madison Wisconsin Mill Lane Cambridge CB RX England

ferriscswiscedu ejaphxcamacuk

August

Revised June

Revised January

Abstract

Genetic algorithms are one example of the use of a random element within an for combi

natorial optimization We consider the application of the to a particular problem

the Assembly Line Balancing Problem A general description of genetic algorithms is given and

their sp ecialized use on our testb ed problems is discussed We carry out extensive computational

testing to nd appropriate values for the various parameters asso ciated with this genetic algorithm

These exp eriments underscore the imp ortance of the correct choice of a scaling parameter and

mutation rate to ensure the go o d p erformance of a genetic algorithm We also describ e a parallel

of the genetic algorithm and give some comparisons b etween the parallel and se

rial Both versions of the algorithm are shown to b e eective in pro ducing go o d

solutions for problems of this type with appropriately chosen parameters

Subject classications Programming combinatorial optimization parallel pro cessing Pro duc

tionScheduling assembly line balancing

Keywords Genetic algorithms combinatorial optimization parallel pro cessing assembly line

balancing

Introduction

Algorithms based on genetic ideas were rst used to solve optimization problems more than twenty

[4]

years ago following the development of the fundamental ideas of genetic algorithms by John

Holland at the University of Michigan During the s this work continued but was largely

unknown In the last few years however there has b een increasing interest in genetic algorithms

[25 , 6] [7, 8, 13]

see for example the conference pro ceedings and the b o oks

A of researchers have lo oked at the application of genetic algorithms to optimization of

nonlinear functions our interest however is in the application of this technique to combinatorial

optimization problems There has b een a variety of work done in this area much of it considering

[19, 29, 16]

the application of genetic algorithms to the solution of the travelling salesman problem

This is not surprising given the imp ortance of the travelling salesman problem and its frequent use

as a vehicle for testing new metho ds in combinatorial optimization However there are particular

diculties with the use of a genetic algorithm for the travelling salesman problem and it is our

b elief that the usefulness of the metho d for other combinatorial optimization problems cannot b e

fairly assessed on the basis of its p erformance on the travelling salesman problem alone

It is appropriate to start with an outline description of the way in which a genetic algorithm

works The metho d op erates with a set of p otential solutions This is referred to as a population

and members of the p opulation are sometimes called individuals The p opulation changes over

time but always has the same size N say Each individual is represented by a single string of

characters At every of the algorithm a tness value f ii N is calculated for

each of the current individuals Based on this tness a number of individuals are selected

as p otential parents These form what is sometimes called a mating p o ol The mating p o ol will

have N members but some will b e duplicates of each other so that it contains several copies of some

individuals in the current p opulation and no copies of others Individuals which are not selected

for the mating p o ol are lost

Two new individuals can b e obtained from two parents in the mating p o ol by choosing a random

p oint along the string splitting b oth strings at that p oint and then joining the front part of one

parent to the back part of the other parent and vice versa Thus parents ABCABCABC

and AABBCCCBA might pro duce ospring ABCBCCCBA and AABABCABC

when mated This pro cess is called crossover It is not necessary for every member of the mating

p o ol to b e involved in crossover some will b e left unchanged Individuals in the mating p o ol

may also change through random mutation when characters within a string are changed directly

Normally each character is given such a small probability of b eing changed that most of the time

individuals are left unaltered The pro cesses of crossover and mutation are collectively referred

to as recombination The end result is a new p opulation the next generation and the whole

pro cess rep eats Over time this leads to convergence within a p opulation with fewer and fewer

dierences b etween individuals When a genetic algorithm works well the p opulation converges to

a go o d solution of the underlying and the b est individual in the p opulation

after many generations is likely to b e close to the global optimum

A mo del algorithm is given in Exhibit I

Exhibit I Mo del Algorithm

rep eat

for each individual i do evaluate tness f i

create mating p o ol of size N based on tness values f i

for i to N do

remove pairs of individuals fj k g from mating p o ol

recombine using individual s j and k

until p opulation variance is small

A more detailed description of some elements of the metho d is given in the next section It is

imp ortant to realize that we have chosen one simple metho d of implementing a genetic algorithm

as has b een describ ed by Gol There are many other versions which have b een suggested some

of which do not op erate on a generation by generation basis and others of which use dierent

mechanisms for repro duction At the heart of all these metho ds however is the idea that go o d

new solutions can b e obtained by using as building blo cks parts of previously existing solutions

There has b een an increasing interest in the use of search techniques which contain a sto chas

tic element as metho ds for the solution of hard combinatorial optimization problems Simulated

[23 , 9, 14] [12, 17]

Annealing has this prop erty as do es in some implementations There

has also b een some recent work which has considered techniques which combine lo cal search and

[1]

the genetic algorithm ideas outlined ab ove For example Aarts et al have shown that such a

technique is comp etitive with for some job shop scheduling problems Also

[20]

recent careful computational work by Johnson et al has demonstrated that simulated annealing

for example can b e comp etitive with the b est available metho ds for certain very large

combinatorial optimization problems This is the context in which the genetic algorithm approach

should b e evaluated An optimistic assessment of the p otential of this metho d is that for at least

some dicult combinatorial problems a suitably tuned genetic algorithm may b e comp etitive with

the b est available heuristic techniques

There are a number of similarities b etween a genetic algorithm and other sto chastic search

techniques The running time of these metho ds will dep end on certain parameter settings with a

greater likelihoo d that an optimal or nearoptimal solution is found if the algorithm is allowed to

run for a long time Also such metho ds have applicability across a wide range of problem domains

part of their attraction is that they hold out the promise of eectiveness without b eing dep endent

on a detailed knowledge of the problem domain One of the ma jor dierences b etween genetic

algorithms and other sto chastic search metho ds is that genetic algorithms op erate using a whole

p opulation of individuals and in this sense they have a kind of natural parallelism not found in

Simulated Annealing or Tabu Search techniques

It is not easy to assess the eectiveness of this type of algorithm For any particular problem

there are likely to b e sp ecial purp ose techniques and which will outp erform a more general

purp ose metho d In a sense this may not b e imp ortant One advantage of simulated annealing for

example is that for many problems a very small amount of programming eort and a single long

will suce to nd as go o d a solution as would b e obtainable with a much faster and

more sophisticated metho d which might take weeks to understand and program Might something

similar b e true for genetic algorithms

The primary aim of this pap er is to demonstrate that genetic algorithms can b e eective in the

solution of combinatorial optimization problems and to give some guidance on the implementation

of the genetic algorithm for b oth serial and parallel architectures Many previous implementations

of this metho d have b een guided by exp eriments initially carried out by and later extended by

a number of other authors for example In the great ma jority of these exp eriments nonlinear

function optimization problems have b een solved rather than combinatorial optimization problems

We b elieve that there is much more to b e learnt ab out the b est implementation of genetic algorithms

for combinatorial problems

The pap er consists of two parts In the rst part we describ e a fairly standard implementation

of the genetic algorithm technique for the Assembly Line Balancing Problem A careful discussion

is given of the appropriate setting for the various parameters involved and some exp eriments are

rep orted to indicate the relative imp ortance of the genetic op erators of crossover and selection

Provided that an appropriate scaling of tness values is carried out coupled with a relatively high

mutation rate the genetic algorithm we have implemented is an eective solution pro cedure for

this problem In the second part of the pap er we describ e an alternative parallel version of the

algorithm for use on a message passing system Some computational comparisons are carried out

The Assembly Line Balancing Problem

We will lo ok at the application of genetic algorithms to a particular problem o ccurring in Op erations

Management namely the Assembly Line Balancing Problem ALBP It is helpful to have a sp ecic

problem class in mind when discussing the details of a genetic algorithm implementation and we

b elieve that the issues which arise in dealing with the ALBP are in many cases the same issues

which arise in tackling other combinatorial optimization problems As we shall see the ALBP has

a natural co ding scheme which makes it attractive as a vehicle for testing genetic algorithms The

problem can b e describ ed as follows

Supp ose we wish to a manufacturing line using a given number of stations n At each

station someone will p erform a number of op erations on each item b eing made b efore passing it on

to the next station in the line The problem is to assign the op erations to the n stations in such a

way as to pro duce a balanced line given the time that each op eration will take The total output of

the line will b e determined by the slowest station which is the station with the most work assigned

to it Our aim is to minimize the amount of work assigned to this station and thus to maximize

the total throughput of the line Thus far we have describ ed a common type of balancing problem

in a scheduling context this would b e equivalent to minimizing makespan on identical parallel

machines The crucial feature of the ALBP however is that certain op erations must b e p erformed

b efore others can b e started If we have this type of precedence relation b etween op erations A and

B for example then we cannot assign op eration B to a station earlier than op eration A though

b oth op erations may b e assigned to the same station Figure gives an example of an ALBP with

arrows used to show the direction of precedence constraints A p ossible solution to the problem

with three stations is shown Station is assigned op erations and and has a total pro cessing

time of minutes Stations and have total pro cessing times of and minutes resp ectively

The ALBP has attracted the attention of many researchers Both heuristic and exact metho ds

have b een prop osed for its solution For a review of some of these metho ds see the pap ers

Note that the ALBP is sometimes p osed with the total op eration time for each station constrained

by some upp er b ound the desired cycle time and the number of stations as the variable to b e 13 8 9 2 5 8 15 15 7 6 10 10 4 6 7 1 18 12 3 9

Station 1 Station 2 Station 3

station time = 38 station time = 42 station time = 33

Figure An example of a precedence graph

minimized

Note that we do not exp ect a genetic algorithm to b e as eective as some of the sp ecial pur

p ose heuristic metho ds for the ALBP As we mentioned ab ove our aim is not to demonstrate the

sup eriority of a genetic algorithm for this particular problem but rather to give some indication of

the p otential of this technique for other combinatorial optimization problems

There are a number of issues to b e resolved in implementing a genetic algorithm for the ALBP

We will now deal with the ma jor issues in turn At this p oint we are concerned with the underlying

mechanisms which should b e used rather than the most computationally ecient implementation

Co ding

There are two asp ects to the co ding scheme for a genetic algorithm One is the way that a solution

is related to the elements of the string which co des for it As we mentioned ab ove a natural co ding

is available for the ALBP In this co ding a solution is represented by a string each element of which

is a number The number in the ith place in the string is the station to which the ith op eration

is to b e assigned This still leaves one asp ect of the co ding undetermined the numbering of the

op erations This numbering will b e imp ortant since the crossover op eration will b e less likely to

separate two pieces of information genes in the genetic description if they are close together

on the string It seems sensible to put the op erations into an order consistent with the precedence

relations so that if one op eration is necessary b efore starting another its assignment will come

in an earlier p osition in the string For the example of Figure we might choose to order the

op erations within the string with the op eration given Then the assignment shown would

b e co ded as

It is p erhaps appropriate to note that the ma jority of genetic algorithm research has b een

carried out on strings using a binary alphab et so that each element in a string is either a or a

We do not b elieve that translating the co ding scheme we have used into a binary alphab et would

b e sensible for this problem

Achieving feasible solutions

Many of the p ossible assignments of op erations to station are infeasible b ecause they break one

or more of the precedence constraints It is a characteristic of the genetic algorithm that for this

and many other problems infeasible solutions are often generated by b oth crossover and mutation

op erators Some authors have advocated the use of nonstandard forms of crossover and mutation

in order to guarantee that only feasible solutions are pro duced We have chosen to use the the

standard genetic op erators and with this restriction there are three ways in which the problem

can b e dealt with One metho d is to use some p enalty function to drive the solutions towards

feasibility A second approach is to force each string generated to corresp ond to a feasible solution

Thus after crossover and mutation have b een carried out a correction routine needs to b e invoked

which adjusts the string so that it b ecomes feasible The third approach is similar and can b e

describ ed as a deco ding of the string The individual string remains unaltered in the p opulation

but it is deco ded using rules which guarantee a feasible assignment Thus the genetic information

within an infeasible string remains present in the p opulation but this is not expressed in the current

individual For this problem we have carried out some limited exp eriments using these dierent

metho ds and the p enalty approach app ears to b e the most eective

Fitness and selection for the mating p o ol

At the heart of a GA is some of tness for each member of the p opulation This will

determine how likely it is that an individual survives into the next generation or is selected for

mating For the ALBP the tness will include an element corresp onding to the total time for the

op erations assigned to the slowest station In addition as mentioned ab ove we also wish to assign

some p enalty cost to any solutions which are infeasible b ecause of precedence constraints In fact

we dene the value of a solution as

maxS k N

i v

i

where S is the total time for op erations assigned to station i N is the number of precedence

i v

violations and k is a constant which we set equal to the largest single op eration time

Given this denition we now wish to minimize the ob jective function Notice that the value

of the ob jective is not a measure of tness since we consider the maximization of tness not its

minimization There are obviously a number of ways in which we can pro ceed One approach is to

take as the tness value some large constant minus the ob jective value The constant is chosen in

order that the tness values are p ositive A second idea is to take tness as the recipro cal of the

ob jective value assuming that all ob jective values are p ositive Yet another technique is to choose

the tness by

f exp hv

i i

where h is chosen to make the tnesses lie in a particular range The drawback of b oth the

latter schemes is that they alter the relative tnesses of dierent individuals in quite a complicated

way while the rst scheme involves the choice of an arbitrary constant These diculties can b e

overcome by taking into account the scaling of tness values more explicitly Many researchers have

recognized the usefulness of a scaling pro cedure in order to achieve some degree of control over the

sp eed of convergence of a genetic algorithm

Supp ose we are given the values of our current individuals and we wish to determine a tness

distribution If we have a maximization problem we dene the tness of individual i f as the

i

value of the ob jective for that individual v If we have a minimization problem then we take as

i

the tness v This has the advantage of making as little change as p ossible to the of

i

the original ob jective function

We now p erform a linear scaling on the values with the aim of pro ducing a tness distribution

F i N with the following prop erties

i

P

N

F

i

i=1

P

N

max F F N

i i i

i=1

F for all i

i

where is a scale factor chosen in order to achieve a particular sp eed of convergence It is not

always p ossible to achieve all three aims in which case we choose to vary the choice of the scale

factor

Dene the linear scaling by F mf c We use the rst two criterion ab ove in order to choose

i i

m and c We obtain

m

P

N

N max f f

i i i

i=1

and

P

N

N max f f

i i i

i=1

c

P

N

N N max f f

i i i

i=1

However this will not guarantee that each F is nonnegative We choose to mo dify the scale

i

parameter in such a way as to guarantee that all F are nonnegative We therefore require

i

P

N

N max f f f

i i i k

i=1

P P

N N

N max f f N N max f f

i i i i i i

i=1 i=1

for all k A simple calculation shows that this gives the choice of as

min f g

g 

where is the given value of scaling required and the maximum value p ossible for is

g 

N

X

max f f

i i k

min f f N

P

 k i

N

k

f N f

i k

i=1

i=1

We have carried out some limited exp eriments comparing this approach to the problems of tness

and scaling with some other more complicated approaches We found it to b e more eective than

the other metho ds we tried and we have used this metho d in all the exp eriments we rep ort here

We also tried a number of dierent metho ds to select individuals for the mating p o ol The

underlying approach is to select an individual with a probability prop ortional to its tness Thus

each time a selection is made individual i is chosen with probability given by F the scaled tness

i

value It is found to b e helpful to alter this simple approach in such a way as to limit the variability

inherent in this scheme It is b etter to ensure that any individual with ab ove average tness is

automatically selected The metho d we use to achieve this is called Sto chastic Universal Sampling

[5]

and is describ ed in detail by Baker In brief the pro cedure can b e describ ed as follows Take

the members of the p opulation and reorder them randomly Assign to each individual an interval

prop ortional to its tness and scaled so that the total length of all the intervals is N Consider

the intervals laid end to end along the line from to N Cho ose a random number x uniformly

on and put the individuals corresp onding to the intervals in which the p oints x x x

x N lie into the mating p o ol Pairs of individuals are then removed from the mating p o ol and

recombined using crossover and mutation We made some comparisons with another commonly

used metho d called remainder sto chastic sampling without replacement and found this to b e

similar in p erformance to Sto chastic Universal Sampling

Crossover and mutation

We turn next to consider the op erations of crossover and mutation which together make up the

pro cess of recombination We have made some limited exp eriments with dierent crossover mecha

nisms with the aim of incorp orating some problem sp ecic knowledge This has b een shown to b e

[16]

eective in some previous studies For our problem however these approaches do not app ear

to lead to any substantial improvements over the more standard crossover mechanism describ ed in

the introduction When a pair of individual s has b een selected for mating crossover o ccurs at a

single random p oint with probability p and with probability p the ospring are identical to

c c

the parents

Following mating the two ospring pro duced then undergo mutation Though we have carried

out some exp eriments with dierent forms of mutation there seems to b e little if any disadvantage

in adopting the standard approach as it app ears in the literature of genetic algorithms This involves

the random change of allo cations of op erations to stations For each op eration in turn with some

small probability p we change the station to which it is assigned either to the station immediately

m

b efore it or to the station immediately after it Thus we change particular elements in the string

by plus or minus one Actual implementation of this mutation op eration is most eciently carried

out by generating a to determine the next p osition on the string to which mutation

will b e applied

There are two ways in which we have adjusted the standard pro cedures for repro duction Firstly

we have inserted into a random p osition in the p opulation an individual from the previous generation

with the b est tness This has the eect of guaranteeing that the nal generation contains the b est

solution ever found which is desirable from the p oint of view of assessing the relative p erformance

of dierent versions of the genetic algorithm This pro cedure called elitism has b een used by a

number of other researchers The second change we have made has as far as we are aware not

b een investigated previously When ospring are pro duced after crossover and mutation or just

one of these if either of the ospring has a worse tness than the worst member of the previous

generation then that individual is not retained and instead one of the parents is allowed to continue

unchanged into the next generation The eect of this is that the worst member of the p opulation

is never worse than the worst in the previous generation Our exp eriments show that this can

substantially improve the convergence b ehavior of the algorithm Clearly by forcing the worst

individual in each generation to b e no worse than that of the previous generation we will sp eed

up the convergence of the algorithm In some circumstances this would not b e an advantage if

convergence is to o fast we are likely to end up at a p o or quality solution In this case however

we can adjust the rate of convergence using the scaling mechanism describ ed ab ove so that the

overall sp eed of convergence is unchanged This turns out to give a signicant net b enet in terms

of the algorithm p erformance

Exp erimental results from the serial implementation

We have carried out a number of exp erimental tests First to nd appropriate settings for the

parameters involved in the genetic algorithm and second to assess the eectiveness of the genetic

algorithm in comparison with other approaches There are a number of diculties involved in car

rying out such exp eriments and it may b e helpful to make some general comments b efore describing

our results in detail

First note that we have not made any attempt to optimize the co de used to implement the

genetic algorithm There are a number of areas where we could exp ect to improve on computation

times by using relatively more sophisticated programming techniques Our aim in these tests is to

nd go o d solutions with a relatively small number of function evaluations and we will lo ok only at

the total number of generations in the genetic algorithm rather than the computation time In fact

for this problem the work in function evaluation which includes counting the number of violated

constraints was in any case the dominant part of the computation

Second we should note that it is imp ortant to carry out comparisons b etween dierent metho ds

only when the number of generations used is roughly the same By changing the scaling factor it is

p ossible to obtain convergence in a greater or lesser number of generations Runs in which a large

number of generations were necessary to achieve convergence will typically have considered many

more individual solutions and thus are for this reason alone likely to do b etter than runs which

converge more quickly if all runs are allowed to continue till convergence There is a danger that

some change to the parameter settings app ears to b e making an improvement when in fact it is a

change in the sp eed of convergence Thus we choose to make comparisons by xing the number

of generations in a run and lo oking at the b est individual in the nal p opulation We can use the

scaling parameter to adjust the sp eed of convergence Our exp erimental results rep orted b elow

show that it is b est to adjust the scaling factor to ensure convergence at ab out the same time as

the run will come to an end

To understand the reason for this it is helpful to see the b ehavior of the algorithm shown

graphically The graphs in Figure show how the value of the worst and the b est in the p opulation

plotted against generation number for three dierent scaling factors These are taken for single

runs of the algorithm on one problem but are reasonably representative One can see that with

unlimited computational time a low scaling factor would b e b est but if runs were to b e stopp ed

after generations for example it would b e b est to choose a scaling factor leading to convergence

at ab out this p oint or a little later

Thirdly it is not clear how the genetic algorithm p opulation should b e initialized We exp er

imented with two kinds of starting p opulation The rst scheme generated the initial p opulation

entirely at random In order that one can eectively program a genetic co de quickly this would

b e the metho d of preference for generating initial solutions In the second scheme we generated a

[3]

set of initial solutions using a metho d due to Arcus This is extremely eective In a signicant

prop ortion of cases the initial set of Arcus solutions contains at least one which is never improved

up on by the GA and in the other cases the b est of the Arcus solutions is never far from the b est

solution found Interestingly however the p erformance of the GA scheme was not as go o d from

this preselected starting p opulation as it was from a random start It seems that there is premature

convergence of the metho d around the few individuals which are generated by the Arcus scheme

with insucient variability in the p opulation the GA is unable to work well Consequently the

results we rep ort here all start with a random initial p opulation

Exp eriments were p erformed on randomly generated problems having op erations these are to

b e assigned to stations Problems were generated so that op eration times are uniformly distributed 300

270 Scaling factor 1.2 Scaling factor 1.5 Scaling factor 2.0 240

F 210 U N C 180 T I O 150 N

V A 120 L U E 90

60

30

0 0 100 200 300 400 500 600

NUMBER OF

Figure Eects of scaling

b etween and We construct the precedence relations using the upp er triangular part

of a matrix W Task i precedes j if and only if W i j where W is a matrix determined as

n(n1)

nonzero entries where is the density of precedence normally set to follows W has

2

Two distinct indices from a uniform distribution are generated and the corresp onding entry of

the upp er triangular part of W is made nonzero This pro cess is rep eated until W has the correct

number of entries

We b egin by describing the results of exp eriments we carried out to nd appropriate values for

the three parameters probability of crossover p probability of mutation p and desired scale

c m

factor We arbitrarily chose generations as the limit for each run the parameter values that

g

we choose would b e markedly dierent if the length of a run was altered We have not carried out

extensive tests on p opulations with dierent sizes all our exp eriments have used a p opulation size

of What little exp erimentation we have carried out leads us to b elieve that this is a reasonable

value for the size of problem we have tackled It is also appropriate for the parallel implementation

for which we use a hypercub e architecture with pro cessors

The results we obtained are shown in Tables and for probabilities of crossover of

and resp ectively The probability of mutation of each element of the string is chosen

from the set of values and The scaling factor is

chosen from the set of values and a value which tries to force a

scaling of lab eled big since in many cases this scaling is not achieved due to the nonnegativity

requirement We chose to carry out exp eriments using randomly generated problems for each of

which we carried out runs of the genetic algorithm using dierent random number seeds For each

set of parameter values we thus have the results of runs These are summarized in the tables by

giving a the average of the p ercentage deviation of the b est solutions in the nal generation from

the b est solution ever found for each problem b the average p ercentage of individuals in the nal

p opulation which had the same value as the b est in that generation and c the rst generation at

which an average of or more of the individuals in the p opulation have the same value as the

b est in the p opulation or left blank if this still has not o ccurred by generation

Hence the rst number given is a measure of the quality of the solutions obtained while the

second and third numbers ect the sp eed of convergence To make the tables easier to read we have

omitted entries for parameter values which lead to convergence either much earlier than generation

or much later To b e precise we have left out entries either where convergence measured by

of the individuals having the same value o ccurs b efore generation or where on average

the number of individuals in the nal p opulation having the same b est value is less than

Interpretation of these results is not entirely clearcut Notice that the lowest value that o ccurs

in any table is that of with p p and scaling factor big The use of a big

m c

scaling factor however can lead to erratic results and for this reason we would recommend the

values p p scale factor if runs are stopp ed after generations These

m c

choices are not particularly critical Notice that this represents a very much higher level of mutation

than has b een recommended in previous studies Since each individual is represented by a string

of length the probability that an individual do es not receive any mutation is with this

value of p The reason that we can use such a large probability of mutation is that towards the

m

end of the run many of the mutations generate individuals worse than the worst current individual

and are thus ignored In eect our metho d b enets from a higher mutation rate early in the run

without this interfering with the convergence of the algorithm at the later stages This approach

thus lends supp ort to the suggestion see for example that a in mutation during the

course of a run may b e b enecial in a similar way to the reduction in temp erature which is used

in simulated annealing metho ds

Table p

c

big

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

Table p

c

big

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

Table p

c

big

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

Table selection only replace if improved

The genetic algorithm involves a selection op erator which can b e introduced without the use of

crossover We carried out some exp eriments to estimate the prop ortion of the improved p erformance

of the genetic algorithm compared with straightforward lo cal search which was accounted for by

the selection op erator There are two implementations of this approach which are of interest First

we can carry out selection on a p opulation in which mutation only takes place if the previous

solution is improved This is similar to a simple lo cal search technique except that we throw away

some solutions that are not progressing well and replace them with copies of solutions which are

progressing b etter There are now two parameters to choose p and the scale factor Table

m g

gives the p ercentage deviation from optimality for various choices of these parameters using this

scheme The b est choices of these were found to b e p and at which values the

m g

b est solution after generations was on average away from the b est ever found Note that

the p ercentage deviation from optimality is much greater than in Tables or

An alternative implementation of the same basic idea is to allow the mutation op erator to

generate a solution which is worse than the current solution and use the selection op erator to obtain

improvements in the overall quality of the solutions represented in the p opulation This is closer

to the spirit of the genetic algorithm and allows the p ossibility of jumping out of a lo cal optimum

In implementing this approach we incorp orated the elitist approach of inserting the previous b est

solution back into the p opulation and also the elimination of any mutations which pro duced a

solution worse than the worst member of the previous p opulation With this implementation the

values of p and which worked b est were and With these parameter settings we then

m g

achieved b est solutions after generations which were on average away from the b est ever

found see Table

We note that the results given ab ove supp ort the conclusions given in that the combination

of selection and crossover is far more eective for optimization than an exclusive use of either

technique The choice of a suitable weighting of these op erators may b e problem sp ecic but their

combined usage is of great imp ortance in pro ducing an eective metho d for the given problem

Table selection only always replace

big

A Parallel Implementation Using Neighborho o ds

Up to this p oint we have only b een concerned with a comparison of the genetic algorithm with

other schemes used to solve our problem In the remainder of this pap er we will lo ok at the

parallelism asso ciated with the genetic algorithm A genetic algorithm would app ear to b e ideal for

parallel architectures since evaluation and repro duction can b e p erformed concurrently For a large

p opulation of individuals the use of many pro cessors would seem enticing However this ignores

the problem of communication and synchronization which are inherent in the selection mechanism

describ ed ab ove In this section we discuss how selection can b e p erformed in a way which reduces

the message passing required in a parallel architecture For a parallel implementation we assume

that each individual in the p opulation resides on a pro cessor with one or more individuals assigned

to each pro cessor and communication is carried out by message passing For descriptions of other

[11, 19, 24, 26, 28]

implementations of parallel genetic algorithms see the references

We consider rst the mechanisms that would b e involved in an eective parallelization of the

genetic algorithm we have describ ed ab ove First of all a global sum op eration should b e used

to compute the sum of all the scaled tness values after which each individual could compute

its scaled tness value Using a parallel prex op eration the partial sums of the scaled tness

values could b e calculated on each pro cessor These values could b e used by the pro cessors in

order to sample the individuals residing on that pro cessor The mating p o ol is thus formed but

is distributed over all the pro cessors The pairs of individuals used for mating are then selected

from this p o ol by some pro cess of random requests In the serial case this is achieved by randomly

p ermuting the mating p o ol and then taking pairs of individuals from the top of the p o ol In

the parallel case the p ermutation could b e generated in parallel so that the individual s required

for mating at a given pro cessor would b e known Each pro cessor would also need to know the

pro cessor holding the ith entry of the mating p o ol so that it could request this individual from that

pro cessor This information could b e determined by an alltoall p ersonalized communication in

fact just the number of individuals in the mating p o ol residing on each pro cessor is needed

Though the parallelization outlined ab ove is clearly feasible there are obvious communication

costs and some diculties in sp ecifying the precise way in which some parts of the pro cess are

carried out We have chosen instead to make comparisons with a dierent form of genetic algorithm

using a parallel scheme in which each individual resides at a lo cation In the sequel we shall

asso ciate the lo cation and the unique individual that is residing there We need to introduce

the notion of a lo cation b ecause in practice the required by each individual will

b e carried out on a single pro cessor It is convenient however to distinguish b etween pro cessors

and lo cations b ecause a single pro cessor may handle more than one individual Each lo cation

is connected to a small subset of the other lo cations which we will call its neighborho o d The

assignment of lo cations to pro cessors may b e optimized to reduce communication costs Though

this approach will mean some substantial changes in the basic algorithm it do es lead to a metho d

which is very well suited to parallel pro cessing Recently this type of approach has b een successfully

used by a number of researchers see for example the work of H M uhlenbein and M Gorges

Schleuter who have developed the ASPARAGOS system to implement an asynchronous parallel

[11, 24]

genetic algorithm

A mo del algorithm for a scheme in which tness information is only compared lo cally is displayed

in Exhibit I I

Exhibit I I Synchronous Neighborho o d Algorithm

rep eat

for each individual i do

evaluate fi

broadcast fi in the neighborho o d of i

receive fj for all individuals j in the neighborho o d

select individual s j and k to mate from the neighborho o d based on tness

request individuals j and k

synchronize

repro duce using individuals j and k

remove individual i from its lo cation and

replace it with one of the ospring of j and k

until p opulation variance is small

The rst question that arises is how to dene the neighborho o d of each lo cation This may b e

dictated to some extent by the architecture of the parallel pro cessor we will wish to avoid passing

messages which require transmission through large numbers of pro cessors We have carried out

[2]

exp eriments with a number of neighborho o d without coming to a rm conclusion In

the exp eriments that we rep ort b elow we have used a neighborho o d structure we call ring This

is dened by stating that lo cations i and j are considered neighboring if

i nbdj ji j j

This can b e viewed as every lo cation b eing on a ring with neighboring lo cations no further than

four links away Strictly then we need to replace ji j j by

minji j j ji N j j ji N j j

The neighborho o d size is eight

In the form given ab ove for each lo cation the algorithm selects two individuals j and k from

the neighborho o d of that lo cation based on tness In order to reduce communication costs it

is p ossible to make one of these selections b e the individual already residing at the lo cation In

this case we only need to select one individual from the neighborho o d with which to mate We

have carried out some exp eriments which show that this technique p erforms at least as well and

frequently b etter than the original scheme and we have adopted it for all the results rep orted

b elow Since we only need to select one individual at each lo cation the metho ds we discussed in

the context of a serial implementation are no longer relevant and selection is carried out in the

simplest way so that individual i is chosen from the neighborho o d with probability given by

X

f f

i j

j 2nbd(i)

Repro duction pro duces two ospring Our was to replace the current individual with

its b est ospring provided this ospring is b etter than the worst individual in the neighborho o d

The question arises whether it is p ossible to use the other ospring We have carried out some

[2]

exp eriments in order to answer this and it seems that the p erformance of the algorithm is not de

graded by throwing the less t ospring away This is the p olicy which is adopted in the exp eriments

rep orted b elow

This metho d eectively deals with communication p enalties provided that care is taken to use

a neighborho o d structure which is appropriate for the machine architecture The synchronization

issue remains largely unsolved but the asynchronous scheme see Exhibit I I I has proven very

eective in practice

Exhibit III Asynchronous Neighborho o d Algorithm

rep eat

for each individual i do

evaluate fi

broadcast fi in the neighborho o d of i

receive fj for all individuals j in the neighborho o d

select an individual j to mate from neighborho o d based on tness

request individual j

repro duce using individuals i and j

until p opulation variance is small

Note that since we have removed the synchronization step it may happ en that we request

an individual based on its tness but in fact receive an individual which has replaced the one

requested In practice this do es not seem to matter

Exp erimental Results for the Parallel Implementation

Our aim in this section of the pap er is to compare the p erformance of the parallel implementation

of the algorithm with a serial version First we try to understand the b ehavior of the synchronous

by implementing this version on the Intel Hyp ercub e iPSC with pro cessors

This will enable us to assess the eect of moving to a parallel implementation which as we have

already seen is quite dierent in the way that selection is carried out We tested the pro cedure on

the same set of test problems that were used in the serial implementation For the ALBP using

Table sy nchr onous p

c

big

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

the notion of neighborho o ds the computational timings are entirely dominated by the evaluation

of tness values which includes a check on the feasibility of newly generated solutions

As for the serial case it is necessary to set three parameters Similar tests were carried out to

those rep orted in tables and The b est choice of the probability of crossover turned out to b e

and Table gives the results obtained with this value The only dierence to the exp erimental

arrangement and that for the serial exp eriments relates to the p opulation size in a run which is

here set to This is only half the number used in the serial implementation as at each generation

a total of N individuals are lo oked at two at each pro cessor Thus using half the p opulation

size will imply the same number of function evaluations leaving aside any consideration of the

saving in function evaluations when an individual is passed through to the next generation without

undergoing either crossover or mutation

It seems that go o d choices for the parameters are p and a scale factor of The

m

results however are not as go o d as for the sequential algorithm Notice that the convergence

is substantially slowed with many fewer cases of complete convergence As we have p ointed out

earlier premature convergence can b e detrimental to the quality of the nal solution It is also

clear from these results that the p erformance of the parallel algorithm is less sensitive to the correct

choice of scaling factor than the serial algorithm

Table asy nchr onous p

c

big

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

a

b

c

The asynchronous scheme has also b een implemented on the Intel Hyp ercub e iPSC The

b ehavior of this version of the algorithm is consistently b etter than the synchronous scheme The

corresp onding results are given in Table Nevertheless the quality of the solutions remains worse

than the b est results obtained using the serial algorithm

Conclusions

There are a number of conclusions that can b e drawn from these exp eriments We b egin by

noting that we have b een quite successful with a version of the standard form of the genetic

algorithm For example we have not tried to combine ideas from genetic algorithms with lo cal

search pro cedures The signicant changes that we have made to the usual genetic algorithm are

as follows First we reject individuals who would decrease the tness of the worst member of a

p opulation Second we carefully control scaling to ensure the b est b ehavior for the algorithm and

nally we use much higher rates of mutation than has b een usual in previous work We recommend

that others wishing to implement a genetic algorithm for combinatorial optimization problems

adopt all three of these measures

The comparison of the standard genetic algorithm with a neighborho o d search scheme with

multiple restarts shows that the genetic algorithm outp erforms this metho d and invariably pro duces

b etter solutions

The exp erimental parallel implementation that we use shows that it is still imp ortant within

this framework to allow the GA sucient time to converge Certainly it is b etter to consider a

smaller p opulation over a longer time span than a larger p opulation over a shorter span The lo cal

neighborho o d scheme do es slow convergence which may b e a desirable feature if very go o d solutions

are required Furthermore the convergence of the asynchronous scheme is slightly slower and in

general leads to b etter solutions

Acknowledgement

The authors would like to thank Mr Menglin Cao for his help in testing the algorithm This

material is based on research supp orted by NSF Grant CCR and Air Force Oce of

Scientic Research Grant AFOSR

References

EHL Aarts PJM van Laarhoven and NLJ Ulder Lo calsearchbased algorithms

for job shop scheduling CQMNot Centre for Quantitative Metho ds Nederlandse Philips

Bedrijven BV PO Box NL MD Eindhoven

EJ Anderson and MC Ferris A genetic algorithm for the assembly line balancing

problem In Proceedings of the Programming Combinatorial Optimization Conference

Waterloo Ontario Canada May University of Waterloo Press

AL Arcus COMSOAL A metho d of sequencing op erations for assembly

lines In ES Bua editor Readings in Production and Operations Management John Wiley

Sons New York

JD Bagley The Behaviour of Adaptive Systems which employ Genetic and Correlation

Algorithms PhD thesis University of Michigan

JE Baker Reducing and ineciency in the In Grefenstette pp

RK Belew editor Proceedings of the Fourth International Conference on Genetic

Algorithms Morgan Kaufmann Publishers Inc San Mateo California

LD Davis editor Genetic Algorithms and Simulated Annealing Pitman London

LD Davis The Handbook of Genetic Algorithms Van Nostrand Reinhold

RW Eglese Simulated annealing A to ol for op erational research European Journal

of

TC Fogarty Varying the probability of mutation in the genetic algorithm In Schaeer

pp

M GeorgesSchleuter Asparagos An asynchronous parallel genetic algorithm In

Schaeer pp

F Glover Tabu search part ORSA Journal on

DE Goldb erg Genetic Algorithms in Search Optimization and

AddisonWesley Reading MA

BL Golden and CC Skiscim Using simulated annealing to solve routing and lo cation

problems Naval Research Logistics Quarterly

JJ Grefenstette editor Genetic Algorithms and their Applications Proceedings of

the Second International Conference on Genetic Algorithms Lawrence Erlbaum Asso ciates

Hillsdale New Jersey

JJ Grefenstette Incorp orating problem sp ecic knowledge into genetic algorithms In

Davis

A Hertz and D de Werra Using Tabu search techniques for graph coloring Computing

P Jog JY Suh and D Van Gucht The eects of p opulation size heuristic crossover and

lo cal improvement on a genetic algorithm for the travelling salesman problem In Schaeer

pp

P Jog JY Suh and D Van Gucht Parallel genetic algorithms applied to the travelling

salesman problem SIAM Journal on Optimization

DS Johnson CR Aragon LA McGeo ch and C Schevon Optimization by simulated

annealing An exp erimental evaluation part I graph partitioning Operations Research

R Johnson Optimally balancing large assembly lines with FABLE Management Science

KA De Jong An Analysis of the Behavior of a Class of Genetic Adaptive Systems

PhD thesis University of Michigan Dissertation Abstracts International B

M Lundy and A Mees Convergence of an annealing algorithm Mathematical Program

ming

H M uhlenbein Parallel genetic algorithms p opulation genetics and combinatorial op

timization In Schaeer pp

JD Schaeer editor Proceedings of the Third International Conference on Genetic

Algorithms Morgan Kaufmann Publishers Inc San Mateo California

JY Suh and D Van Gucht Distributed genetic algorithms Technical Rep ort

Computer Science Department Indiana University Blo omington

FB Talbot JH Patterson and WV Gehrlein A comparative evaluation of heuristic

line balancing techniques Management Science

R Tanese Parallel genetic algorithms for a hypercub e In Grefenstette pp

D Whitley T Starkweather and D Fuquay Scheduling problems and travelling sales

men The genetic edge recombination op erator In Schaeer pp