Simulated Annealing

Appendix A Simulated Annealing A.1 Introduction Simulated annealing can be considered as an extension of local optimization meth- ods because: • This approach can be applied to criteria that are neither continuous nor con- tinuously differentiable. It is just necessary to be able to compute the value of the criterion for any feasible solution. Thus, the criterion may be given by any type of function or even by an algorithm that returns a numerical value starting from the values of the parameters that define a solution. • The variables can be of a qualitative nature, needing only to be able to derive from their “values” a quantitative value of the criterion. Indeed, simulated annealing easily applies to combinatorial optimization prob- lems. A problem P that belongs to the field of application of simulated annealing is expressed as any optimization problem: Find *∈ Ss such that )*( = Opt sfsf )( ∈Ss where: • S is the set of feasible solutions. A feasible solution is one that satisfies all the constraints. We will see some examples in the next section. • f is a “function” in the broadest sense, as explained above. This function is the criterion. • Opt refers either to minimization or to maximization, depending on the type of problem. 450 A Simulated Annealing A.2 Basic Requirements In order to apply simulated annealing, we must be able to: • Compute the value of the criterion for any feasible solution. • Define an initial feasible solution. • Derive a neighboring feasible solution from any current solution. The criterion depends on the problem to be solved. An initial feasible solution may be difficult to find. A heuristic is often used to generate such a solution. Another possibility is to start with a solution that is not feasible and to penalize the criterion in order to shrug off this solution as soon as possible. A neighboring solution is usually obtained by slightly altering the solution at hand. Again, the way the current solution is altered depends on the type of problem considered. Let us illustrate the basic requirements with the following four examples. A2.1 Traveling Salesman Problem A salesman has to visit shops located in n different towns. The objective is to find the shortest circuit passing once through each town. This circuit starts from the salesman’s office and ends in the same office that is located in the n + )1( -th town. For this problem, the criterion is the length of the circuit. Any circuit passing once through each town is a feasible solution. Note that the number of feasible solutions is equal to n! since n + 1 towns are concerned. Indeed, this problem is combinatorial. A neighboring solution of a given circuit is obtained by permuting 2 towns of the circuit, these towns being chosen at random among the n towns to visit. A2.2 Balancing a Paced Assembly Line We refer to the notations introduced in Chapter 7. We consider that the cycle time C is given as well as a partial order over the set of operations and the operation times. A feasible solution is the assignment of the operations among N stations such that the partial order is satisfied and the sum of the operation times in any station is less than or equal to C. A.2 Basic Requirements 451 The criterion could be (7.1) or (7.2) defined in Chapter 7, or any other criterion derived from the distribution of the operations among the stations. A way to obtain a neighboring solution has already been explained in Chapter 7. It consists either in permuting two operations located in two consecutive stations or in moving one operation to the next (or the previous) station, assuming that the capacity constraints and the partial order are satisfied by this new solution. A2.3 Layout Problem This layout problem consists in arranging the resources (machines, equipment, etc.) in a shop in order to minimize a criterion that is often the sum of the products of the average flows between machines by the distance covered by these flows. A feasible solution should satisfy various constraints such as, for instance: • The resources must be located on a surface that is limited by the walls of the shop. • Some resources must be located near the entrance (or the exit) of the shop, for practical reasons. • The location of some resources is fixed. This is the case when the machines are particularly heavy and require a floor that is reinforced. • Some pairs of resources must be close to each other since they work together. • Some pairs of machines must be far from each other. One reason could be that a machine emits vibrations that disturb the functioning of another. These constraints are some of the most frequent when solving this kind of problem. A neighboring solution of a given solution is obtained either by shifting a re- source to an idle location or by permuting two resources. An example is developed in Section 10.2.5.2. A2.4 Establishing a School Timetable Several criteria may be proposed in such a problem such as, for instance: • Minimize the total time students have to stay at school. This criterion can be expressed as the sum over the classes and the days of the week of the difference between the time students leave school in the evening and the time they arrive at school in the morning. • Minimize the sum of the idle periods between the first and the last course of each day, for all the teachers and days of the week. 452 A Simulated Annealing • Minimize the total number of courses that are taught outside a “normal” activity period. The “normal” activity period could be the period between 8 a.m. and 5 p.m. Thus, we can use one of these criteria or a weighted sum of two of them or all of them. A feasible timetable verifies constraints like: 1. Some special courses must be taught in specific rooms (chemistry, physics, language labs, etc.). 2. A professor cannot teach more than one course at a time. 3. The timetable should fit with the courses that must be taught in the classes. 4. Each teacher should teach courses corresponding to her/his specialty. 5. Each teacher should teach a given number of hours every week. A neighboring solution of a given solution can be obtained by: • moving a course to a free period chosen at random; • permuting two courses (with the corresponding teachers); • permuting two classes that are concerned by the same course, without permuting the teachers. The above lists are not exhaustive. A.3 Simulated Annealing Algorithm A3.1 A Brief Insight into the Simulated Annealing Approach The basic idea behind a simulated annealing algorithm is to generate step by step a sequence of solutions, without requiring an improvement of the solution at each step. Simulated annealing can keep a solution that is worse than the previous one with a probability. This probability diminishes when the deterioration of the criterion grows and when the number of solutions already generated increases. The goal of this approach is to avoid being entrapped in a subset of feasible solutions, like when using a gradient method for a multimodal function. Consider Figure A.1, for instance, that corresponds to a minimization problem. There are three levels of contour lines visible. Each type of contour line corresponds to a value of the criterion: the thick lines represent solutions having a criterion value equal to 1000, the dotted lines represent solutions having a criterion value equal to 800, etc. A.3 Simulated Annealing Algorithm 453 X5 X2 X1 X4 X0 X3 Value 1000 Value 800 Value 600 Sequence of solutions Figure A.1 The search path in the set of feasible solutions The local minimum values are X1 to X5. If X0 represents the initial solution, then a gradient method would generate a sequence of solutions that tends towards the local minimum X3. Using the simulated annealing approach we can visit several “basins” and possibly find a solution whose criterion is better than X3. Now, we will investigate how to decide if we should keep or reject a solution that is generated as a neighboring solution of the previous one. A3.2 Accepting or Rejecting a New Solution Let Sn be the previous solution and SU n )( the corresponding value of the criterion. A neighboring solution Sn+1 has been derived at random from Sn and SU n+1 )( is the corresponding criterion value. If Sn+1 is “better” than Sn , that is to say if n+1 ≤ SUSU n )()( in the case of a minimization problem and n+1 ≥ SUSU n )()( in the case of a maximization problem, we keep the solution Sn+1 as the next current solution of the sequence. If Sn+1 is “worse” than Sn , then we take the solution Sn+1 as the next current solution of the sequence depending on the probability calculated as follows: pn = − Δ Tnn )/(exp (A.1) where: n =Δ ()()n+1 − SUSU n 454 A Simulated Annealing Tn is a decreasing function of the rank n of the solution. Tn is called the “tem- perature”. Algorithm A.1 is used to accept or reject Sn+1 when this solution is “worse” than Sn . Algorithm A.1. 1. Generate at random a real number x on the interval [ 1,0 ] (uniform probability density). 2. Compute pn according to (A.1). 3. If n ≥ xp , then keep Sn+1 as the next current solution of the sequence, otherwise reject Sn+1 and keep Sn as the next current solution.

Load more