NEW TOOLS FOR LARGE-SCALE COMBINATORIAL OPTIMIZATION PROBLEMS
By OLEG V. SHYLO
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2009
1 °c 2009 Oleg V. Shylo
2 Dedicated to my family
3 ACKNOWLEDGMENTS I would like to thank my adviser, Dr. Panos M. Pardalos, for all of his encouragement and support. His knowledge, experience and sense of humor were crucial for this work.
Also, I would like to acknowledge collaborators Clayton Commander, Valeriy Ryabchenko, Alkis Vazacopoulos, Oleg Prokopyev, Ashwin Aruselvan, Ilias Kotsireas and Timothy Middelkoop, who have been a pleasure to work with. I am grateful to my friends, my family and my girlfriend Erika for their love, moral support and all the fun we had together.
4 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...... 4
LIST OF TABLES ...... 7
LIST OF FIGURES ...... 8
ABSTRACT ...... 9
CHAPTER 1 RESTART STRATEGIES IN OPTIMIZATION: PARALLEL AND SERIAL CASES ...... 10 1.1 Introduction ...... 10 1.2 Motivation ...... 11 1.3 Literature Overview ...... 12 1.4 Optimal Restart Strategies ...... 13 1.4.1 Serial Algorithm ...... 13 1.4.2 Parallel Algorithm ...... 16 1.5 Algorithm Portfolio and Restart Strategies ...... 20
2 GLOBAL EQUILIBRIUM SEARCH ...... 25 2.1 Algorithm Description ...... 25 2.2 Approximation Formula ...... 29
3 UNCONSTRAINED BINARY QUADRATIC PROBLEM ...... 32 3.1 Introduction ...... 32 3.2 Description of the Algorithm ...... 34 3.2.1 Search Procedures ...... 35 3.2.2 Generation Procedure ...... 35 3.2.3 Computational Experiments ...... 35 3.3 Concluding Remarks ...... 41 4 PERIODIC COMPLEMENTARY BINARY SEQUENCES ...... 42
4.1 Introduction ...... 42 4.2 PAF and Quadratic Forms ...... 42 4.3 A Combinatorial Optimization Formalism for PCS ...... 45 4.4 Applications of PCS and Related Sequences ...... 46 4.5 Optimization Algorithm ...... 47 4.6 Results ...... 48 4.6.1 PCS(50, 2) ...... 48 4.6.2 Consequences ...... 49 4.7 Conclusion ...... 49
5 5 WEIGHTED MAX-SAT PROBLEM ...... 58 5.1 Introduction ...... 58 5.2 Algorithm for the Weighted MAX-SAT ...... 59 5.2.1 Generation Procedure ...... 59 5.2.2 Local Search for the Weighted MAX-SAT ...... 59 5.3 Computational Experiments ...... 61 5.3.1 Benchmarks ...... 61 5.3.2 Algorithms and Software ...... 65 5.3.3 Results and Conclusions ...... 65
6 JOB SHOP SCHEDULING PROBLEM ...... 67 6.1 Introduction ...... 67 6.2 Mathematical Model and Notations ...... 68 6.3 Algorithm Description ...... 69 6.3.1 Encoding and Generation Procedure ...... 72 6.3.2 Local Search ...... 74 6.3.3 Acceleration of Move Cost Estimation ...... 76 6.4 Computational Results ...... 77 6.5 Concluding Remarks ...... 80 6.6 Benchmark Problems ...... 82 REFERENCES ...... 89
BIOGRAPHICAL SKETCH ...... 96
6 LIST OF TABLES Table page 3-1 Results for Beasley instances [1] ...... 39
3-2 Results for Palubeckis’ instances [2] ...... 40
3-3 Results for the Maximum Independent Set instances arising in coding theory [3] 41
5-1 Results for jnh benchmarks ...... 64
5-2 Results for rndw1000b benchmarks ...... 64 5-3 Results for rndw1000c benchmarks ...... 64 6-1 Computational results by GES for ORB1-ORB10 ...... 82
6-2 Computational results for LA01-LA40 ...... 83 6-3 Results for TA1-TA80 ...... 84
6-4 Results for TA1-TA80 (continued) ...... 85 6-5 Comparison with TSSB ...... 86 6-6 Comparison with other algorithms ...... 86 6-7 Results for DMU1-DMU80 ...... 87 6-8 Results for DMU1-DMU80 (continued) ...... 88
7 LIST OF FIGURES Figure page 1-1 Tabu algorithm and job shop scheduling problem. Average running time as a function of restart parameter ...... 11
1-2 The average running time as a function of restart parameter R, ξ has log-normall f(x) distribution with µ = 1 and σ = 2; h(x) = 1−P r{ξ≤x} is hazard rate function ... 15 1-3 The algorithmic speed-up as a function of the number of processors; ξ has log-normal distribution with µ = 1 and σ = 2; the actual speed-up is obtained by comparing the optimal serial restart version to the optimal parallel restart version; the plot for the linear speed-up is for comparison purpposes only...... 19 1-4 The efficiency of parallelization (linear speed-up has efficency koefficient equal to 1), ξ has log-normall distribution with µ = 1 and σ = 2; ...... 20 1-5 Example of improved performance when combining two algorithms with log-normally distributed running time...... 21 1-6 Another example of improved performance when combining two algorithms with log-normally distributed running time...... 22
2-1 GES method (general scheme) ...... 28 3-1 Local Search TABU ...... 36 3-2 Generation procedure ...... 37
4-1 Graphical representations of the four symmetric matrices M1,M2,M3,M4 .... 44 5-1 Generation procedure ...... 60 5-2 Local Search 1-opt ...... 61 5-3 Local Search k-opt ...... 62 5-4 Local Search TABU ...... 63
6-1 GES for JSP ...... 70
6-2 Procedure that generates solution ...... 73 6-3 Improvement procedure ...... 76
6-4 Estimation procedure ...... 78
8 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NEW TOOLS FOR LARGE-SCALE COMBINATORIAL OPTIMIZATION PROBLEMS By Oleg V. Shylo August 2009 Chair: Panos M. Pardalos Major: Industrial and Systems Engineering Many traditional algorithmic techniques used in combinatorial optimization have reached the computational limits of their scope. A relatively low cost and availability of parallel multi-core clusters offer a potential opportunity to shift these limits. Unfortunately, most existing serial algorithms are not easily adaptable to parallel computing systems. The objective of this work is to provide new algorithmic methods that can be easily and effectively scaled to parallel systems with large numbers of processing units. We concentrate on a scalability that comes from running a set of independent algorithms in parallel. We investigate the relationship between the parallel acceleration and properties of serial algorithms. Some of these results revealed that the best serial algorithm is not necessarily the best choice in parallel setting. Additionally, we observed that the combination of different algorithms (some of them may have poor performance characteristics!) can be more beneficial then using any particular algorithm on its own. The limitations of this approach are addressed. In this dissertation, we present the algorithms based on Global Equilibrium Search method and Tabu Search method for classical optimization problems. These algorithm demonstrate the state-of-the-art performance comparing to the latest published work in the field: Unconstrained Binary Quadratic Problem, Weighted MAX-SAT problem and
Job Shop Scheduling Problem.
9 CHAPTER 1 RESTART STRATEGIES IN OPTIMIZATION: PARALLEL AND SERIAL CASES
1.1 Introduction
Let P be a class of problems and A be a randomized algorithm that is able to solve correctly any instance of P in finite time. An algorithm A is of the Las Vegas type, if
its running time on a given instance p ∈ P is a random variable tA(p). A restart version of A(p) can be implemented by stopping the algorithm after certain time and running it
from the beginning: i.e. run A(p) for time R or until the correct solution is found; restart A(p) from the beginning and run it for time R or until the correct solution is found; and so on. In other words, each run of A(p) is independent from all the others and limited to time R. The current paper investigates the running time distribution, such that the
average running time of AR(p) is smaller then the running time of A(p). This type of distributions is ofen observed in practical application, and thus it is crucial to understand
the underlying properties (see Section 1.3 for examples). In addition, the random nature of Las Vegas algorithms allows for a simple parallelization, which consists in running several independent copies of the same algorithm in parallel and halting the execution whenever the correct answer is found. This paper addresses the problem of minimizing the average running time of Las Vegas algorithm, both in serial and parallel setups. The conditions that are necessary for the existence of effective restart strategy are presented (i.e. allowing for acceleration of the algorithm). We show that the optimal restart strategy for the parallel case might be different from the optimal restart strategy of the serial algorithm. This result asserts that the most effective serial algorithm is not necessarily the best choice for parallelization. Our results can be used to create the approximation models describing the behaviour of complex algortihms allowing for effective restart strategies. A brief overview of the related work is presented in Section 1.3. Our main results and their proofs are described in Section 1.4.
10 1.2 Motivation
The idea of restarting the random algorithm in order to improve its performance has been widely used in optimization algorithms. For example, one of the best up-to-date solvers for the job shop scheduling problem is based on tabu search [4]. This algorithm incorporates the simple restart strategy: the tabu algorithm is restarted after every 200000 iterations. However, there is no explanation or discussion for such choice. This is a common situation for the research papers on optimization algorithms and the lack of
clarity involving the concept of restart strategies needs to be addressed. In Figure 1.2,
the computational experiments involving tabu search with different restart parameters are presented for a single benchmark problem instance. It is clear, that the choice of restart parameter provided in [4] is near-optimal and was based on careful tuning. Additionally, the choice of the restart parameter is crucial for the algorithm’s performance.
Figure 1-1. Tabu algorithm and job shop scheduling problem. Average running time as a function of restart parameter
11 1.3 Literature Overview
In [5], the idea of restart strategies was formulated, allowing for acceleration of general Las Vegas-type algorithms. The restart strategy was defined as an infinite set
of time intervals S = {t1, t2,...} after which a Las Vegas algorithm is re-initiated (i.e. restarted with a new random seed). The domain of restart strategies was extended to the case when ti’s are random variables, and runs may be suspended and restarted at a later time. The authors have shown that for the optimal restart strategy ti’s are constant:
∗ ∗ t1 = t2 = ... = t (uniform strategy). To obtain the optimal restart period t , one requires the full knowledge of the running time distribution. The universal restart strategy was proposed to overcome this issue. Such strategy is only a logarithmic factor slower than the optimal restart strategy and does not require any prior knowledge of Las Vegas
algorithm. In [6], these results were extended to the parallel environment. Similar to the serial case, a universal parallel restart strategy was proposed that is only a logarithmic factor slower than the optimal parallel restart strategy. The examples were provided for which non-uniform parallel restart schedule performs better than the uniform strategy. The authors showed that the optimal uniform parallel restart strategy (i.e. when each
processor runs independent copy of Las Vegas algorithm and is restarted after fixed time period) is within a constant factor from an optimal parallel restart strategy. The concept of restart strategies is also applicable in the domain of deterministic algorithms. In [7], it was shown that by introducing randomness and restart strategies into deterministic techniques (complete search algorithms), one can achieve algorithmic speedup for practical instances of constraint satisfaction problems and propositional satisfiability. The existence of effective restart strategies for enumeration algorithms was formally explained and justified in [8]. The authors introduced the concept of
restart-distribution, which is the distribution of running time, such that there exists an effective optimal restart strategy (i.e. allowing acceleration of Las Vegas algorithm). A
search tree model is provided in [9] allowing for a formal proof of heavy-tailed behaviour
12 (imbalanced search trees). The authors suggest that the heuristics with large variability in running times can outperform more uniformly performing techniques by utilizing restart strategies. Such behaviour was observed for a range of state-of-the-art randomized
backtrack search methods. The existence of restart-distributions in the domain of Internet browsers was reported
in [10]. The loading process of Internet page can be accelerated by applying restart strategy (for example, by simply clicking refresh button in the browser). The authors provide algorithms to calculate the moments of completion time for Las Vegas algorithms under the optimal restart strategy. The presence of efficient restart strategies for the tabu search algorithm was reported
in [11]. It is worth noting that the concept of restart strategies is widely used in the
algorithmic design of metaheuristics (see for example [12], [13], [14]). However, its impact on the performance of the algorithm is often underestimated and merely mentioned.
1.4 Optimal Restart Strategies
Let A be the Las Vegas type algorithm that is used to solve problem p. The running time that is necessary to obtain the correct answer for p is a random variable ξ. A restart
version of A can be described as a series of independent executions: run A for time R or until the correct solution is found; restart A from the beginning and run it for time R or
until the correct solution is found; and so on. AR stands for a restart version of A and R > 0 denotes its restart period. In the reminder of this paper the following notation is used: P r stands for the probability of the event; E stands for the expected value of the random variable given in square brackets; symbol | is used to define conditional probabilities and expectations; f is the probability density function of ξ. 1.4.1 Serial Algorithm
The average running time that is required by AR to find the correct answer is given by: 1 − P r{ξ ≤ R} T (R) = R · + E[ξ|ξ ≤ R] (1–1) P r{ξ ≤ R}
13 In [8], the authors introduced the notion of a restart-distribution, which, in other words, is the distribution that has potential for acceleration (by implementing a restart version of underlying algorithm):
Definition 1. The distribution of a random varibale ξ is a restart-distribution if there exist R > 0, such that:
1 − P r{ξ ≤ R} R · + E[ξ|ξ ≤ R] < E[ξ] P r{ξ ≤ R}
Troughout this paper, the cumulative distribution function (CDF) of the algortihm’s running time is suposed to be continuous and differentiable (first and second derivative).
For other cases, a continuous and differentiable approximation of original CDF can be used. Proposition 1. The first derivative of T (R) with respect to restart period R is
dT (R) 1 − P r{ξ ≤ R} f(R) = − · T (R) (1–2) dR P r{ξ ≤ R} P r{ξ ≤ R}
Proof. µ ¶ dT (R) 1 − P r{ξ ≤ R} −f(R) 1 − P r{ξ ≤ R} = + R · − · f(R) + dR P r{ξ ≤ R} P r{ξ ≤ R} (P r{ξ ≤ R})2 Z 1 −f(R) R +R · f(R) + xf(x)dx = P r{ξ ≤ R} (P r{ξ ≤ R})2 µ 0 ¶ 1 − P r{ξ ≤ R} f(R) 1 − P r{ξ ≤ R} = − R · + E[ξ|ξ ≤ R] = P r{ξ ≤ R} P r{ξ ≤ R} P r{ξ ≤ R} 1 − P r{ξ ≤ R} f(R) = − · T (R) P r{ξ ≤ R} P r{ξ ≤ R}
∗ Corollary 1. Let R be the optimal restart period for AR, then the expected running time
of AR∗ is 1 − P r{ξ ≤ R∗} T (R∗) = (1–3) f(R∗)
14 Example Let ξ be a random variable with log-normal distribution with parameters σ = 2 and µ = 1. The graph of the average running time depending on the restart parameter R is presented in Figure 1.4.1.
Figure 1-2. The average running time as a function of restart parameter R, ξ has f(x) log-normall distribution with µ = 1 and σ = 2; h(x) = 1−P r{ξ≤x} is hazard rate function
∗ 0 ∗ ∗ Proposition 2. If R is the optimal restart period for AR, then f (R ) < 0 and T (R ) ≥ f(R∗) −f 0(R∗) .
Proof. µ ¶ d2T (R) d −f(R) 1 − P r{ξ ≤ R} = · T (R) + = dR2 dR P r{ξ ≤ R} P r{ξ ≤ R} Ã µ ¶ ! f 0(R) f(R) 2 f(R) dT (R) T (R) · − + − · − P r{ξ ≤ R} P r{ξ ≤ R} P r{ξ ≤ R} dR f(R) P r{ξ ≤ R}2
15 The second derivative at the point R∗ is ¯ d2T (R)¯ −f0(R∗) f(R∗) ¯ = T (R∗) − 2 ¯ ∗ ∗ dR R∗ P r{ξ ≤ R } P r{ξ ≤ R }
∗ ∗ f(R∗) Since the second derivative at minimum point R is non-negative, T (R ) ≥ −f 0(R∗) and f 0(R∗) < 0.
Theorem 1. The hazard rate function of a restart-distribution is non-increasing on some interval containing optimal restart period.
Proof. Suppose that hazard rate function h(x) is increasing on some interval [x1; x2],
∗ 1 1−P r{ξ≤R} such that R ∈ [x1; x2]. Then h(x) = f(R) is decreasing on this interval. Hence, its derivative is negative: · ¸ d 1 − P r{ξ ≤ R} 1 − P r{ξ ≤ R} −f 0(R) = −1 + · < 0. dR f(R) f(R) f(R)
Using Corollary 1, at optimal point R∗:
1 − P r{ξ ≤ R∗} f(R∗) = T (R∗) < , f(R∗) −f 0(R∗) which contradicts Proposition 2.
1.4.2 Parallel Algorithm
In this section, the trivial parallelization of Las Vegas algorithms is considered. Let
n AR denote a parallel algortihm that consists of n independent copies of AR running in
n parallel. The algorithm AR halts whenever one of n underlying algortims finds the correct
n n solution. Let random varibale ξmin be a running time fo AR; let Tn(R) denote the average
n n running time of AR and let fmin denote its probability density function. The average running time of the algorithm applying restart strategy in parallel environment is given by:
(1 − P r{ξ ≤ R})n T (R) = R · + E[ξn |ξn ≤ R] (1–4) n 1 − (1 − P r{ξ ≤ R})n min min
16 Proposition 3. The first derivative of Tn(R) with respect to restart period R is
dT (R) (1 − P r{ξ ≤ R})n n(1 − P r{ξ ≤ R})n−1f(R) n = − T (R) · (1–5) dR 1 − (1 − P r{ξ ≤ R})n n 1 − (1 − P r{ξ ≤ R})n
Proof. µ dT (R) (1 − P r{ξ ≤ R})n −n(1 − P r{ξ ≤ R})n−1f(R) n = + R · − dR 1 − (1 − P r{ξ ≤ R})n 1 − (1 − P r{ξ ≤ R})n
¶ n(1 − P r{ξ ≤ R})n(1 − P r{ξ ≤ R})n−1f(R) + (1 − (1 − P r{ξ ≤ R})n)2 Z n(1 − P r{ξ ≤ R})n−1f(R) R · xf n (x)dx = (1 − (1 − P r{ξ ≤ R})n)2 min µ 0 −n(1 − P r{ξ ≤ R})n−1f(R) R · (1 − P r{ξ ≤ R})n × + 1 − (1 − P r{ξ ≤ R})n 1 − (1 − P r{ξ ≤ R})n Z R ¶ n n 1 R · fmin(R) xfmin(x)dx · n + n + 0 1 − (1 − P r{ξ ≤ R}) 1 − (1 − P r{ξ ≤ R}) (1 − P r{ξ ≤ R})n −n(1 − P r{ξ ≤ R})n−1f(R) + R · = 1 − (1 − P r{ξ ≤ R})n 1 − (1 − P r{ξ ≤ R})n (1 − P r{ξ ≤ R})n n(1 − P r{ξ ≤ R})n−1f(R) − T (R) · 1 − (1 − P r{ξ ≤ R})n n 1 − (1 − P r{ξ ≤ R})n
∗ n Corollary 2. Let Rn be the optimal restart period for AR, then the expected running time
n of AR is ∗ ∗ (1 − P r{ξ ≤ Rn}) Tn(Rn) = ∗ (1–6) f(Rn) · n ∗ n 0 ∗ Proposition 4. Let Rn be the optimal restart period for AR, then f (Rn) < 0. The proof of Proposition 4 is similar to the proof of Proposition 2 and is left to the reader.
∗ ∗ n Theorem 2. If R and Rn are optimal restart periods for AR and AR, respectively, then T (R∗) ∗ ≤ n. Tn(Rn)
T (R∗) Proof. Suppose that ∗ > n, then consider a folowing algorithm: run 1st copy of Tn(Rn) algorithm for period of time ∆ and suspend it if the correct solution was not found; run
17 2nd copy of algorithm for period of time ∆ and suspend it if the correct solution was not found; and so on until n copies are initiated; now, continue with 1st copy for time ∆ and suspend it if the correct solution was not found; and continue this cycle until the correct solution is found. By decreasing ∆, one can achieve the average running time that is better then T (R∗). However, this contradicts the theorem from [5], which states that the optimal restart strategy is uniform (i.e. only one copy is running and restart interval is constant).
Theorem 3. If the hazard rate function of the running time distriution is unimodal and
T (R∗) ∗ ∗ ∗ < n, then R < R . Tn(Rn) n
Proof. Using Corollary 1 and Corrollary 2:
∗ ∗ ∗ T (R ) 1 − P r{ξ ≤ R } nf(Rn) ∗ = ∗ ∗ < n Tn(Rn) f(R ) 1 − P r{ξ ≤ Rn} or ∗ ∗ 1 − P r{ξ ≤ R } f(Rn) ∗ ∗ < 1 f(R ) 1 − P r{ξ ≤ Rn} Hence, the result follows from Theorem 1 and unimodality of hazard rate function.
Example Let ξ be a random variable with log-normal distribution with parameters
σ = 2 and µ = 1. The optimal speed-up achieved by parallelization is the relation between the running time of serial algorithm and parallel algorithm. The graph of the algorithmic speed-up with respect to the number of processes is presented in Figure 1.4.2. The restart parameters of the serial and parallel algortihms are different: the optimal restart parameter is monotonious increasing function of the number of processors. The efficiency of parallelization is a relationship of the running time of the serial algorithm to the running time of parallel algortihm multiplied by the number of processors. The graph of the efficiency of parallelization for log-normal distribution is shown in Figure 1.4.2. Remark Las Vegas algorithms can be easily parallelized by running the independent copies of the serial version on different processors. The parallel algorithgm provides the
18 Figure 1-3. The algorithmic speed-up as a function of the number of processors; ξ has log-normal distribution with µ = 1 and σ = 2; the actual speed-up is obtained by comparing the optimal serial restart version to the optimal parallel restart version; the plot for the linear speed-up is for comparison purpposes only. super-linear speedup, if it utilizes up to n processors and provides the speed-up that is larger than n. In the literature, one can find the papers that report “super-linear” speed-up when using such trivial parallelization. Theorem 2 shows that if the super-linear speed-up is achieved, then the underlying distribution of the running time is a restart- distribution. In some sense, the trivial parallelization takes advantage of the restart-distribution properties without implementing any restart strategy. Furthermore, the performance of the algorithm can be improved by applying an effective restart strategy explicitly. In other words, the super-linearity is mainly due to inefficiences of the serial algorithm. Since, the efficiency of parallelization quickly degrades with increasing number of processors (see Figure 1.4.2), in order to achieve the best utilization of available resources, one should combine the trivial parallelization (running independent copies in parallel) with other types of parallelizing: bit-level parallelism, instruction-level parallelism, data parallelism, task parallelism.
19 Figure 1-4. The efficiency of parallelization (linear speed-up has efficency koefficient equal to 1), ξ has log-normall distribution with µ = 1 and σ = 2;
Intuitively, one can assume that the best serial algorithm is the best candidate for parallelizing. However, Theorem 2 together with example presented above shows that the optimal restart parameter for the optimal serial algorithm can be non-optimal in parallel environment. In other words, the stable performance in the serial case do not guarantee the efficient parallel performance. On the conrary, an algorithm, which does not perform very efficient in a serial case, can outperform more effective serial search procedures when run in parallel.
1.5 Algorithm Portfolio and Restart Strategies
The framework discussed above is based on a use of a single algorithm with properties that allow for an efficient restart strategy. We considered the situation in which a combination of algorithms with different performance characteristics is used. The primal question was whether or not combining the algorithms can improve the performance with respect to the use of a single algorithm. As we showed above, the efficiency of parallel algorithms is closely related to the notion of restart-distributions.
20 Figure 1-5. Example of improved performance when combining two algorithms with log-normally distributed running time.
Consequently, we investigated the combination of algorithms with the running times following restart-distributions. The log-normal distribution was used as a basis for our studies since it belongs to the family of restart distributions. We considered a pairs of
algorithms with log-normally distributed running times to optimal solution (different means and variances). In Figure 1.5, we present the results for the mix of two algorithms (log-normal running times). Three data sets correspond to the following setups:
• Running a single algorithm (given mean and standard deviation of the running time to optimal solution) on each process
• Running a best performing mix of two algorithms (best out of all possible combinations). For example, on 6 processors we run 5 copies of the algorithm with smaller variance and 1 copy of the algorithm with a large variance. In Figure 1.5, we present another example of effective combination. In this section we will address the limitations of this approach with respect to algorithms implementing restart strategies.
21 Figure 1-6. Another example of improved performance when combining two algorithms with log-normally distributed running time.
Let us consider two algorithms A1 and A2 with restart parameters R1 and R2 and random running times ξ1 and ξ2 respectively. Assume that R1 = kR2, where k is integer. The running time of these algorithms or their combination will be considered to be a multiple of their restart parameters. In other words, the algorithms can be terminated only at their restart time points (after the solution was found) and the largest termination time is the total running time of algorithm portfolio. These assumptions are made to simplify the discussion and the general case will be addressed later.
Let p1 = P r{ξ1 ≤ R1} and p2 = P r{ξ2 ≤ R2}. Firstly, let us show that under the assumptions described above, the best algorithm portfolio consists of a single algorithm, i.e. mixing algorithms is not a profitable strategy. Let N be a total number of available parallel processors, and the optimal mixed portfolio consists of n2 copies of A2. Suppose that the mixed portfolio is better then a single algorithm portfolio. Then, the running time of the mix should be better then the running time of the single algorithm portfolio consisting of solely of A1:
22 1 1 R1 N−n n k < R1 N 1 − (1 − p1) 2 (1 − p2) 2 1 − (1 − p1)
N−n2 n2k N (1 − p1) (1 − p2) < (1 − p1)
k (1 − p2) < (1 − p1) (1–7)
Additionally, the running time of the mix should be better then the running time of
the single algorithm portfolio consisting of solely of A2:
1 1 R1 N−n n k < R2 N 1 − (1 − p1) 2 (1 − p2) 2 1 − (1 − p2)
N N−n2 n2k k(1 − p2) < 1 − (1 − p1) (1 − p2)
Using (1–7),
N (N−n2)k n2k k(1 − p2) < 1 − (1 − p2) (1 − p2)
N Nk k(1 − p2) < 1 − (1 − p2)
Nk 1 − (1 − p2) k < N (1 − p2) or, using formula for geometric series,
Xk−1 Ni k < (1 − p2) i=0
Hence, we arrived to contradiction, since (1 − p2) < 1 and under the assumptions described above, the best algorithm portfolio consists of a single algorithm.
Now, let us consider a case when R1 > R2,R1 = R2k, but k is not necessarily integer.
Consider a case when A1 is a better choice for a single algorithm portfolio. Then,
23 1 1 R1 N < R2 N 1 − (1 − p1) 1 − (1 − p2)
N R1 1 − (1 − p2) k = < N ≡ ∆ R2 1 − (1 − p1)
Let the average running time of the optimal mixed combination is Em, and suppose that it provides a better performance than a single algorithm portfolio. Then,
k 1 1 R2 N < Em < R1 N dke 1 − (1 − p2) 1 − (1 − p1) or N 1 R1 1 − (1 − p2) dke R1 N /Em < N < < 2 1 − (1 − p1) R2 1 − (1 − p1) k The last inequality shows that in the best case by implementing the mixed portfolio strategy one can achieve only acceleration by factor 2. These bounds hold for the rest of the cases and can be easily verified.
Example Let R1 = 7, R2 = 3, p1 = 1, p2 = 0.25 and N = 2. Then the average running
time for A1 running on two processors is 7, the average running time for A2 running on 2 processors is 6.85, and the average running time of their combination is 5.81. In this
6.85 example, the acceleration with respect to the single algorithm portfolio is 5.81 = 1.17 The results presented in this section assert that even though the mixed algortihm portfolio can outperform the parallel algorithm comprised of a single algorithm, the acelleration is limited by factor 2 if the efficient restart strategy is implemented.
24 CHAPTER 2 GLOBAL EQUILIBRIUM SEARCH
2.1 Algorithm Description
The numerous amount of discrete optimization problems that arise in practice have such various characteristics, that development of a general purpose solution method is clearly impracticable. One way of tackling this issue is to develop a library of suitable solution methods, allowing the practitioner to choose the most suitable for
his problem under his time constraints and quality requirements. In a recent decades,
heuristic approaches (such as tabu search [15], simulated annealing (SA) [16], etc.) have gained considerable amounts of attention from the scientific community for being the only practical tool that can be applied to a wide range of difficult problems. Global Equilibrium Search offers another highly effective tool for solving large scale optimization
problems [17]. The GES method shares ideas similar to those that inspired the SA technique, while providing, in practice, faster asymptotic convergence to the optimal solution on a wide
class of optimization problems. Moreover, the GES method can be used in an ensemble with other techniques, which makes it more versatile than most of its predecessors. Consider a discrete optimization problem of the following form:
min{f(x): x ∈ S : S ⊆ {0, 1}n} (2–1) where f is some quality function. Let us introduce a random binary vector ξ, that takes a values from a feasible set S according to the Boltzmann’s distribution with µ ≥ 0 being
the temperature parameter:
exp(−µf(x)) P {ξ(µ) = x} = P (2–2) x∈S exp(−µf(x))
25 Consider the SA method applied to problem (2–1). It can be shown easily that under certain conditions (i.e. symmetric neighbourhood structure) the stationary probabilities of
the Markov chain associated with the SA method are given by (2–2).
The set S can be split into two subsets in such a way that one of them contains the feasible solutions for which the j-th component is 1, and another set will contain
1 0 the solution with the jth component equal to 0. Let us name these two sets Sj and Sj .
1 0 Obviously, Sj ∪ Sj = S. Then the probability of the jth component of ξ being 1 can be
expressed as: P 1 exp(−µf(x)) x∈Sj pj(µ) ≡ P {ξj(µ) = 1} = P . (2–3) x∈S exp(−µf(x)) The idea of the GES method is to use some subset of known solutions Sb to generate new solutions in the successive stages of the algorithm. The distribution (2–3) (or its approximation) can be used for such a generation (substituting S with Sb in the formula):
P x∈Sb1 exp(−µf(x)) b j pbj(µ) ≡ P {ξj(µ) = 1} = P . (2–4) x∈Sb exp(−µf(x)) If arg min{f(x): x ∈ Sb} is unique, then the average Hamming distance between newly generated solutions and the best solution in the set Sb converges to zero as µ goes to infinity. However, the speed of such convergence is not the same for different components of the generated solutions, i.e. the speed of convergence of the jth component depends
1 0 on the quality of the solutions Sj versus the quality of solutions in Sj . Simply put, the temperature parameter in (2–3) controls the level of similarity of the newly generated solutions to high quality solutions in Sb. The uniqueness of the best solution x∗ in Sb
mentioned above should be maintained at all stages of the algorithm. One of the limitations of the strategy described above is that in order to implement it, there should exist an easy way of generating random solutions from S with the
distribution given by (2–4). Unfortunately, for some problems, the structure of the set
S would make this hard to achieve. For such cases, the local search based techniques (i.e.
SA method, Tabu method, GES method) are not easily applicable.
26 Another issue with generating the random solution x from S using (4) is that the components of the random solution x are not independent random variables. However, for the simplicity of an algorithm, this is usually ignored because the convergence property is
more important for the performance of the algorithm. Whenever the new solution is added to the set Sb, it is easy to recalculate the
probabilities pbj if the denominator and numerator in (2–4) are stored separately. Therefore, there is no need to store the whole set Sb in the memory! The notion of Sb is used below mainly for the simplicity of discussion. The performance of any GES based algorithm is dependant on the choice of the temperature schedule. As with the SA method, there is no basic recipe to provide an optimal schedule for the GES. The general advice here is to choose the sequence of
increasing values µ0 = 0, µ1, µ2 = µ1α, . . . , µK = µK−1α,(K is a number of temperature stages and α > 0) in such a manner that the algorithm will find the best solution from the b set S almost for sure when generating solutions with temperature parameter µK . However, there is no need to provide a separate cooling schedule for each problem solved. Simple scaling of the cost function (f0(x) = C · f(x), C > 0) can make one temperature schedule suitable for a wide range of problems from the same class. The choice of scaling factor can be made, for example, in the initial stage of algorithm, when µ = 0. Additionally, if we
multiply the denominator and numerator of (2–4) by exp(µf(x∗)), where x∗ is the best solution from Sb, then the convergence to the best solution from Sb is less dependant on the absolute values of solution costs. The general scheme of the GES method is presented in Figure 2-1. There are some elements that are included in the scheme, but were not discussed above: elite solutions set, prohibition of certain solutions and restarting the search. These elements are not necessary for success of the GES method and can be easily excluded. However, for some
classes of problems they can provide a significant performance improvement.
27 Require: µ – vector of temperature values, K – number of temperature stages, maxnfail – restart parameter, ngen – # of solutions generated during each stage b 1: xbest ← construct random solution; S=E={xbest} 2: while stopping criterion = FALSE do 3: if Sb = ∅ then 4: x ← construct random solution 5: xmax = x b 6: S = {xmax} (set of known solutions) 7: E = {xmax} (set of elite solutions) 8: end if 9: for nfail = 0 to nfail∗ do 10: xold = xmax 11: for k = 0 to K do k b 12: calculate generation probabilities(p , S, µk) 13: for g = 0 to ngen do k 14: x ← generate solution(xmax, p ) 15: R ← search method(x) (R is some subset of encountered solutions) 16: Sb = Sb ∪ R b 17: xmax = arg min{f(x): x ∈ S} 18: if f(xmax) < f(xbest) then 19: xbest = xmax 20: end if 21: update elite set(E,R) 22: end for 23: end for 24: if f(xold) > f(xmax) then 25: nfail = 0 26: end if 27: Sb = E 28: end for 29: P = P ∪ N(xbest, dp) 30: E = E − P 31: if RESTART-criterion= TRUE then 32: E = ∅ 33: end if 34: Sb = E; b 35: xmax = arg min{f(x): x ∈ S} 36: end while 37: return xbest
Figure 2-1. GES method (general scheme)
28 The main cycle (lines 2-36) is repeated until some stopping criterion is satisfied. The algorithm execution can be terminated when the best known record for the given problem is improved, or when the running time exceeds some limiting value. If the set of
known solutions Sb is empty, then the initialization of the data set is performed in lines 3-7. The cycle in lines 9-28 is executed until there is no improvement in nfail∗ consecutive cycles. The main element of the GES method is the temperature cycle (lines 11-23). The
probabilities that guide the search are estimated using expression (2–4) in the beginning of each temperature stage (line 12). For each probability vector, ngen solutions are generated (lines 13-22). These solutions are used as initial solutions for the local search procedure
(line 15). The subset of encountered solutions R is used to update the set Sb (line 16). Some set of the solutions can be stored in memory, in order to provide a fast initialization of the algorithm’s memory structures (lines 27,34). Such set is referred to as an elite set in the algorithm pseudocode. Certain solutions can be excluded from this set to avoid searching the same areas multiple times. In lines 29 and 30, the solutions for which the Hamming distance to to xbest is less than parameter dp are excluded from the elite set. 2.2 Approximation Formula
The formula 2–4 for generation probalities is very sensitive whenever the set of know solutions is small. By Such situation is typical for the initial stage, since the set Sb is initially empty. In order to overcome this instability, the alternative approximation for formula 2–3 is usually used in computations.
29 By taking derivative of pj in 2–3, we obtain the following expression: Ã ! P P ∂p X x∈S1 exp(−µf(x)) x∈S1 f(x) exp(−µf(x)) j = f(x) exp(−µf(x)) · P j − P j = ∂µ ( exp(−µf(x)))2 exp(−µf(x)) x∈S x∈S x∈S ÃP P ! 1 f(x) exp(−µf(x)) x∈S f(x) exp(−µf(x)) x∈Sj = pj(µ) · P − P = x∈S exp(−µf(x)) x∈S1 exp(−µf(x)) ÃP P j ! x∈S1 f(x) exp(−µf(x)) x∈S0 f(x) exp(−µf(x)) j j xj =1 = pj(µ) · P + P − hfiµ = x∈S exp(−µf(x)) x∈S exp(−µf(x)) Ã P P ! x∈S0 exp(−µf(x)) x∈S0 f(x) exp(−µf(x)) xj =1 j j xj =1 = pj(µ) · pj(µ)hfiµ + P · P − hfiµ = exp(−µf(x)) 0 exp(−µf(x)) x∈S x∈Sj ¡ ¢ xj =1 xj =0 xj =1 = pj(µ) · pj(µ)hfiµ + (1 − pj(µ))hfiµ − hfiµ , and
∂pj ¡ ¢ = p (µ) (1 − p (µ)) hfixj =0 − hfixj =1 , (2–5) ∂µ j j µ µ
where the conditional expectations are given by: P x∈Su f(x) exp(−µf(x)) xj =u j hfiµ = E[f(ξ(µ))|xj = u] = P , u ∈ {0, 1}. (2–6) u exp(−µf(x)) x∈Sj
The solution of equation (2–5) gives us the following formula for pj(µ), which can be verified by simple substitution into (2–5):
1 pj(µ) = R (2–7) 1 + 1−pj (0) exp(− µ(hfixj =0 − hfixj =1)dt) pj (0) 0 t t The distribution properties discussed above are used in GES method to generate new solutions based on the partial information about the search space. Let S˜ be some subset
˜1 ˜ ˜0 of S (it is some subset of known solutions), Sj = {x | x ∈ S, xj = 1}, and Sj = {x | x ∈ ˜ S, xj = 0}. Let K be the number of temperature stages, and let µ0 < µ1 < . . . < µK be the temperature values with lower index corresponding to the number of the temperature stage. Additionally, let us define the following numbers for k = 0,...,K and j = 1, . . . n:
30 P x∈S˜u f(x) exp(−µkf(x)) P j ˜u , if Sj 6= ∅ u x∈S˜u exp(−µkf(x)) E = j u ∈ {0, 1} (2–8) kj 0, otherwise
Then probability pj(µ) can be approximated applying trapezoid rule to expression (2–7):
1 p˜j(µk) = P (2–9) 1−p˜j (0) 1 k−1 0 0 1 1 1 + exp{− (µi+1 − µi)(E + E − E − E )} p˜j (0) 2 i=0 ij i+1j ij i+1j
GES can either generate solutions according to the distribution defined by (2–9), or can apply some number of perturbations to the given solution which asymptotically lead to this distribution. We should also point out here that formula (2–9) is just one of the possible ways to estimate (2–7). Another variant of estimation probabilities (2–7) for JSP
is given in Chapter 2.
31 CHAPTER 3 UNCONSTRAINED BINARY QUADRATIC PROBLEM
3.1 Introduction
One of the well-known and most interesting classes of integer optimization problems is the maximization of the quadratic 0–1 function:
Xn Xn max f(x) = qijxixj, (3–1) x∈{0,1}n i=1 j=1
n×n where qij are elements of an n × n symmetric real matrix Q ∈ R . This problem is referred to as an unconstrained binary quadratic programming problem (UBQP). Since
2 xi = xi for all 0–1 variables, linear function of x can be moved into the quadratic part of the objective function in (3–1). Many fundamental problems in science, engineering, finance, medicine and other diverse areas can be formulated as quadratic binary programming problems. Quadratic functions with binary variables naturally arise in modeling selections and interactions. For example, consider a set of n objects {1, . . . , n}, each of which is selected or not. For each pair (i, j) of objects we associate a weight qij measuring the interaction between points i and j. Let xi = 1 if the object is selected, and xi = 0 otherwise. If the global interaction is the sum of all interactions between the selected points, then their global
Pn Pn interaction can be represented as a quadratic binary function i=1 j=1 qijxixj.A class of such problems has been studied in solid-state physics [18, 19]. Furthermore, applications of constrained and unconstrained versions of the problem (3–1) can be found
in numerous areas including but not limited to medicine [20], computer-aided design [19, 21, 22], scheduling [23], models of message management [24] and chemistry [25]. Many graph theoretical problems can be naturally represented in terms of quadratic binary optimization, including well-studied maximum clique and maximum independent set
problems [26–28].
32 Unconstrained quadratic binary programming problem belongs to the class of NP -hard combinatorial optimization problems. The simplest class of quadratic binary optimization problems, that is the product of two linear binary functions, is NP -hard [29].
There is only a limited number of classes known to be polynomially solvable [30–33]. It is also known that the problem of checking if an unconstrained binary quadratic problem has a unique solution is NP -hard [34]. Furthermore, quadratic binary programming problem remains NP -hard even if we know that the global optimum is unique [34]. One of the classical examples of an NP -hard problem [35], reformulated in terms of quadratic 0–1 optimization, is the aforementioned maximum clique problem [27]. Approximation of large cliques is also difficult, since as it is shown by H˚astad[36] that unless NP = ZPP no polynomial time algorithm can approximate the clique number within a factor of n1−² for any ² > 0. Khot improved this bound to n/2(log n)1−² [37]. Regarding approximability we should also mention an important result by Nesterov
[38], which is a generalization of idea by Goemans and Williams, who developed an approximation algorithm for the maximum cut problem [39]. Nesterov proved that boolean quadratic programming, max {q(x) = xT Qx | x ∈ {±1}n} can be approximated by semidefinite programming with accuracy 4/7. Extensions of this result can be found in Nesterov [40] and Ye [41].
Unfortunately, due to the intrinsic complexity of the problem (3–1), exact algorithms (see, for example, [26, 42, 43]) can handle instances only of size of several hundred variables for reasonably sparse matrices Q. For larger instances with dense matrices only heuristic approaches are applicable. Among various proposed heuristic methods for solving the unconstrained binary quadratic problem we should mention algorithms based on tabu search [12, 44, 45], evolutionary [46]. Some of the algorithmic codes, test problems generators and benchmark instances are publicly available and can be found at
[1, 2, 47, 48].
33 In this chapter we discuss a new heuristic for solving the unconstrained binary quadratic problem based on the global equilibrium search (GES) framework suggested in
[17] (in Russian). A brief description of GES (in English) is given in a survey [49]. Recent
application of GES for solving job shop scheduling problem (GESJSP2006) with excellent results is reported in [13]. The results of our computational experiments indicate that the developed GES-based heuristic algorithm is highly efficient (it outperforms the best available solver [12] in the literature) and can be used for solving hard instances (with several thousands of zero-one variables) of the unconstrained quadratic binary programming problem.
Another important and promising conclusion of the study is as follows. Tabu search is one of the most powerful heuristic methods successfully applied for solving a variety
of problems [15]. Furthermore, this approach may become even more efficient if it is used within some multi-start framework (see, for example, [12]). The results of our computational studies indicate that the combination of the GES method and tabu search can provide another very attractive alternative for solving large-scale combinatorial optimization problems.
3.2 Description of the Algorithm
The Global Equilibrium Search (GES) framework described in Chapter 1 was used to construct the algortihm for the UBQP (see Figure 2-1). The only solutions that are memorized by the algorithm are so called elite solutions (set E), which are used to form the initial set of known solutions (lines 27, 34, Figure 2-1). ˆ The set E contains Esize best solutions from the set S (line 21, Figure 2-1). In line 29-30 (Figure 2-1) we remove from the set E the best known solution and all
solutions within the Hamming distance of dp from it. The idea behind this prohibition is to force the algorithm to explore the solutions with different structure then those that have been already found.
The lines 29-31 (Figure 2-1) are used to restart the search process.
34 3.2.1 Search Procedures
Search methods (see Figure 2-1, line 15) used in our study are based on the 1-flip neighborhood. A solution y belong to 1-flip neighborhood of binary vector x if the
Hamming distance d(x, y) is equal to 1. In other words, if y ∈ N1(x) then yj = 1 − xj for some index j and yk = xk for k 6= j. In order to accelerate 1-flip move evaluations, the special data structure is maintained throughout the search process. Let gains(x) denote a vector in which the j-th element represents a gain in the cost function resulting from applying 1-flip operator to the variable xj. This can be calculated in linear time: Xn gainsj(x) = qjj(1 − 2xj) + 2 qij(1 − 2xj)xi. i=1,i6=j
Based on this neighborhood, the search method based on tabu search [15] was implemented
(see Figure 3-1). 3.2.2 Generation Procedure
The generation procedure is used to transform some given solution according to the GES probabilities introduced in Chapter 1. The pseudo-code of generation procedure is shown in Figure 3-2.
Given the vector x, the generation procedure applies the number of 1-flip move operators (lines 2-16). The uniformly distributed variable in the interval (0,1) is used to determine whether to apply perturbation to a given variable or not (lines 4,8). The procedure terminates, whenever the Hamming distance between x and a perturbed
solution becomes equal to the parameter distmax. 3.2.3 Computational Experiments
The GES algorithm described in this chapter was implemented in C++ language. All computational experiments were conducted using the personal computer Pentium 4 3.00
GHz with 1.00 Gb of RAM. For the parameters described above the following values were
used: the number of temperature stages K = 25, the generation parameter distmax = n/2
35 Require: x – initial solution, g(x)– vector of gains, nbad– stopping par., tabu– the tabu tenure Ensure: 1: xbest = x; n = |x|; M = {1, 2, . . . , n}; step = 0; impr =true; R = ∅ 2: while impr do 3: impr =false; ∆best = 0; ∆cur = 0; stepimpr = step; last used(j) = −∞ j ∈ M 4: repeat 5: repeat 6: ∆ = 0 7: if gj(x) ≤ 0 for ∀j then 8: R = R ∪ x 9: end if 10: Generate random permutation RP of the set M 11: for k = 1 to n do 12: j = RP [k] 13: if step − last used(j) > tabu OR f(x) + gj(x) > f(xbest) then 14: if gj(x) ≥ 0 then 15: ∆ = ∆ + gj(x); last used(j) = step; xj = 1 − xj 16: g(x) ←recalculate gains(x); step = step + 1 17: end if 18: end if 19: end for 20: ∆cur = ∆cur + ∆ 21: until ∆ > 0 22: if ∆cur ≥ ∆best then 23: if ∆cur > ∆best then 24: ∆best = ∆cur; stepimpr = step; impr =true 25: end if 26: xbest = x 27: end if 28: ∆ = −∞ 29: for k = 1 to n do 30: j = RP [k] 31: if step − last used(j) > tabu OR f(x) + gj(x) > f(xbest) then 32: if gj(x) ≥ ∆ then 33: ∆ = gj(x); indbest = j 34: end if 35: end if 36: end for
37: ∆cur = ∆cur + ∆; last used(indbest) = step; xindbest = 1 − xindbest 38: g(x) ←recalculate gains(x); step = step + 1 39: until step − stepimpr < nbad 40: x = xbest; g(x) ←recalculate gains(x) 41: end while 42: return R
Figure 3-1. Local Search TABU 36 Require: x – solution to apply random permutations,p ˜(µk)– probability vector corresponding to k-th temperature stage Ensure: 1: dist = 0; j = 1; 2: while (j ≤ n) do 3: if xj = 1 then 4: if p˜j(µk) ≤random[0,1] then 5: xj = 0; dist = dist + 1; 6: end if 7: else 8: if p˜j(µk) ≥random[0,1] then 9: xj = 1; dist=dist+1; 10: end if 11: end if 12: j = j + 1 13: if dist = distmax then 14: return x 15: end if 16: end while 17: return x
Figure 3-2. Generation procedure
for temperature stages 1-5 and distmax = n/5 for temperature stages 6-K, the number of solutions generated ngen = 27, the tabu tenure tabu = 21, prohibition parameter dp = 200, nfail∗ = 1. Initial probabilities at the beginning of each temperature cycle are all initially
−7 set to 0.5. For the temperature schedule the following values were used: µ0 = 0, µ1 = 10 ,
log 10−log µ1 µk = µk−1 ∗ 48 for k = 2,...K. The maximal size of elite set is set to Esize = n/2.
The parameter nbad (tabu search method) was set to n. Our computational experiments showed that in order to improve the performance of the algorithm the RESTART-criterion should be set to FALSE for the first two test sets (Beasley [1] and Palubeckis [2]) and
TRUE for the maximum independent set problem instances (Sloane [3]).
The best algorithms for the UBQP problem are based on tabu search based techniques [12, 15, 45]. Among them we have selected the implementation of Palubeckis [12] to provide a comparative experimental study of the GES performance. There are multiple reasons for this choice. First of all, to our best knowledge the MST2
37 implementation (MST stands for the multi-start tabu search) presented in [12] is currently the best heuristic for solving the UBQP problem. It shows excellent and robust performance on a various set of benchmark instances. The comparative study of MST2 with other well-known approaches including tabu search, simulated annealing and genetic local search is given in [12]. Furthermore, the source code for the algorithm is available online [2]. Therefore, it allowed us to organize the computational experiments, in which the same personal computer was used to run the algorithms. Hence, the computational times provided below can be easily used as one of the measures for comparison of the algorithms. We designed our experiments in the following manner. First, the MST2 algorithm of Palubeckis was being run 10 times on each problem from the benchmarks set. For each instance, we record the best solution found by MST2 in 10 runs (fbest), the average of the best solution value found (favr), the average time until the best solution was found
(tbest) and the average total running time ttotal (the maximum number of iteration was used as termination criterion for MST2). We also record how many times MST2 was able to obtain the solution value given by fbest for a given problem (# found). The similar experiment was conducted for the GES algorithm. The GES algorithm terminated whenever the solution of the same or better quality than fbest provided by MST2 was found (i.e. the best solution found by MST2 was used as target solution for the GES). The meaning of the data columns for GES is the same as for MST2, except for the column
’# found’, which contains the number of times the GES algorithm was able to find the solution of the same or better quality then the target solution provided by MST2 (fbest). Such design of the computational experiments allowed us to obtain quite meaningful data for the comparison of algorithmic performance. The first set of benchmark instances used in our study was originally introduced by Beasley [44]. The experiments were limited to the largest instances of this set, since the smaller ones are extremely easy and therefore do not provide a good basis for the
38 Table 3-1. Results for Beasley instances [1]
Problem Algorithm fbest favr # found tbest ttotal 2500-1 MST2 1515944 1515944 10 3.1 465.6 GES 1515944 1515944 10 2.6 2500-2 MST2 1471392 1471392 10 19.4 460.4 GES 1471392 1471392 10 47.40 2500-3 MST2 1414192 1414192 10 17.5 469.6 GES 1414192 1414192 10 8.71 2500-4 MST2 1507701 1507701 10 2.1 467 GES 1507701 1507701 10 1.5 2500-5 MST2 1491816 1491816 10 2.2 462 GES 1491816 1491816 10 2.77 2500-6 MST2 1469162 1469162 10 4.8 473 GES 1469162 1469162 10 5.46 2500-7 MST2 1479040 1479040 10 8.7 469 GES 1479040 1479040 10 24.5 2500-8 MST2 1484199 1484199 10 7.8 460 GES 1484199 1484199 10 2.6 2500-9 MST2 1482413 1482413 10 14.9 462 GES 1482413 1482413 10 7.5 2500-10 MST2 1483355 1483355 10 26.4 458.4 GES 1483355 1483355 10 24.7
comparison. We use the same notation as in [12] to refer to these instances: b2500-1, b2500-2, . . . , b2500-10 (the size of the problems is 2500 variables). The maximal number of iterations for MST2 (stopping criterion) was set to 100. The results of the experimental testings are presented in Table 1. Based on the data presented in Table 1, one can see that both techniques are able to obtain the solution of the same quality. The computational times does not differ too much. For some of the instances MST2 outperforms GES, for the others GES outperforms MST2. On average MST2 is performing slightly better, however the difference is not significant enough. These results asserts the need for more difficult instances in order to come up with more meaningful conclusions regarding the comparative effectiveness of these algorithms. For the second set of experiments, the problems of higher dimensionality and higher density were used. These problems were introduced by Palubeckis and used in his studies [12]. In our experiments, we limited testing to the largest problems of the set (p5000-1,
. . . , p5000-5, p6000-1, p6000-2, p6000-3, p7000-1, p7000-3). The maximal number of
iterations for MST2 (stopping criterion) was set to 1000. The computational results are
presented in Table 2.
39 Table 3-2. Results for Palubeckis’ instances [2]
Problem Algorithm fbest favr # found tbest ttotal p5000-1 MST2 8566041 8566497.1 2 2280.5 4567.7 GES 8567067 8566602 10 2269.8 p5000-2 MST2 10833518 10833518 10 710.6 7431.1 GES 10833518 10833518 10 442.26 p5000-3 MST2 10482962 10482962 10 1721.7 7610.3 GES 10482962 10482962 10 1271.3 p5000-4 MST2 12256474 12256474 10 2387.9 9486.9 GES 12256474 12256474 10 2081.6 p5000-5 MST2 12727442 12727352.3 9 2871.9 9729.1 GES 12727442 12727442 10 1345.5 p6000-1 MST2 11385007 11385007 10 1958.1 6443.3 GES 11385007 11385007 10 3141.7 p6000-2 MST2 14328191 14328050.5 5 6144.8 10507.4 GES 14328191 14328191 10 1950.3 p6000-3 MST2 16138892 16138848.6 8 2540.7 13605.6 GES 16138892 16138892 10 3549.9 p7000-1 MST2 14474315 14473796.4 2 4545.5 8567 GES 14474315 14474315 10 2231.1 p7000-3 MST2 20571136 20570091.2 1 6351.9 17985.6 GES 20571136 20571136 10 5641.1
Looking through the data presented in Table 2, one can see that the GES algorithm outperforms MST2 both in solution quality and computational times. For the problems p5000-1, p5000-5, p6000-2, p6000-3, p7000-1 and p7000-3, MST2 was not able to find the best solution in every run of the algorithm. The column ‘# found’ indicates that the multi-start strategy used in MST2 does not allow the algorithm to improve the solution during some of the runs. On contrary, GES showed the robust behavior on the whole set of instances. The differences in computational times are quite noticeable. Based on this data, we conclude that the GES framework in concert with a tabu search implementation provides a highly effective alternative for the existing UBQP algorithmic approaches. It is well known that the problem of finding the maximum independent set can be equivalently represented in terms of the UBQP problem [28]. For the third set of experiments we used algorithms MST2 and GES to solve maximum independent set problem on five graphs arising in coding theory (1zc1024, 1zc2048, 1zc4096, 1zc8192 and 1zc16384) [3] . The maximal number of iterations for MST2 (stopping criterion) was set to 1000. The results of computational experiments for the maximum independent set problems are given in Table 3-3.
40 Table 3-3. Results for the Maximum Independent Set instances arising in coding theory [3]
Problem Algorithm fbest favr # found tbest ttotal 1024 MST2 109 109 10 18.9 95.4 GES 109 109 10 4.2 2048 MST2 181 178 1 33.9 206.2 GES 182 181.3 10 45.1 4096 MST2 313 311.5 2 157.9 529.7 GES 316 313.9 10 4.0 8192 MST2 566 564.5 4 3.3 1519.9 GES 589 580.7 10 10.7 16384 MST2 1051 1049.5 4 10.3 5646.4 GES 1070 1065.9 10 39.8
The performance data in Table 3 reveals that the GES technique dominates MST2 algorithm for the set of the problems solved. For all instances, the GES algorithm had no problems of finding the solution of the same quality as found by MST2 (see ‘# found’). Furthermore, for all of the instances GES obtained better solutions with less computational cost.
3.3 Concluding Remarks
In this chapter we applied the GES framework for solving the classical unconstrained binary quadratic programming problem. The developed algorithm performs favorably compared to the best known heuristic [12] on a set of well-known publicly available benchmark instances [1–3]. The ideas introduced by the global equilibrium search method provide a very promising approach for construction of algorithms for solving discrete optimization problems.
41 CHAPTER 4 PERIODIC COMPLEMENTARY BINARY SEQUENCES
4.1 Introduction
In this chapter we establish a new formalism that allows one to use the powerful methods available in Combinatorial Optimization [50], in the study of sequences with Periodic Autocorrelation Function zero.
We begin with the definition of the Periodic Autocorrelation Function, PAF, from [51]:
Definition 2. Let A = {a1, a2, . . . , an} be a sequence of length n, where a1, a2, . . . , an are
real numbers. The periodic autocorrelation function, PA(s), of A is defined as: Xn PA(s) = aiai+s , s = 0, 1, . . . , n − 1 i=1 where we consider i+s modulo n (when i + s > n). The following lemma records a symmetry property of the elements of the PAF vector, the proof is easy and is left to the reader, also see [51]. Lemma 1.
PA(s) = PA(n − s), s = 1, 2, . . . , n − 1.
The importance of lemma 1 lies in the fact that for a sequence of length n one needs to £ ¤ n consider only the first 2 elements of the PAF vector, where [x] denotes the integer part of x. £ ¤ n Notation: In the rest of this chapter we denote m = 2 . 4.2 PAF and Quadratic Forms
In this section we view the elements of the PAF vector as Quadratic Forms and we associate certain matrices with them. These matrices are used to formulate problems about sequences with PAF zero, as binary feasibility problems.
T Definition 3. Let a = [a1, a2, . . . , an] be a column n × 1 vector, where a1, a2, . . . , an ∈
{−1, +1} and consider the elements of the PAF vector PA(1),...,PA(m). Define the
42 following m symmetric matrices (which are independent of the sequence a) 1 mjk = mkj = , when ajak ∈ PA(i), j, k ∈ {1, . . . , n} M = (m ), s.t. 2 , i = 1, . . . , m i jk 0, otherwise
The following lemma can be proved by straightforward computation.
Lemma 2. The matrices Mi can be used to write the PAF equations in a matrix form:
• for n odd: T a Mia = PA(i), i = 1, . . . , m.
• for n even: 1 aT M a = P (i), i = 1, . . . , m − 1 and aT M a = P (m). i A m 2 A
Example Let n = 8, a = [a1, . . . , a8]. Then we have that m = 4 and
1 aT M a = P (i), i = 1, 2, 3 and aT M a = P (4) i A 4 2 A
2 3 2 3 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 6 2 2 7 6 2 2 7 6 7 6 7 6 7 6 7 6 7 6 7 6 1 0 1 0 0 0 0 0 7 6 0 0 0 1 0 0 0 1 7 6 2 2 7 6 2 2 7 6 7 6 7 6 7 6 7 6 7 6 7 6 0 1 0 1 0 0 0 0 7 6 1 0 0 0 1 0 0 0 7 6 2 2 7 6 2 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 7 6 2 2 7 6 2 2 7 M = 6 7 ,M = 6 7 , 1 6 7 2 6 7 6 1 1 7 6 1 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 7 6 2 2 7 6 2 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 7 6 2 2 7 6 2 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 1 7 6 0 0 0 0 0 2 0 2 7 6 2 0 0 0 2 0 0 0 7 6 7 6 7 4 5 4 5 1 1 1 1 2 0 0 0 0 0 2 0 0 2 0 0 0 2 0 0 2 3 2 3 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 6 2 2 7 6 2 7 6 7 6 7 6 7 6 7 6 7 6 7 6 0 0 0 0 1 0 1 0 7 6 0 0 0 0 0 1 0 0 7 6 2 2 7 6 2 7 6 7 6 7 6 7 6 7 6 7 6 7 6 0 0 0 0 0 1 0 1 7 6 0 0 0 0 0 0 1 0 7 6 2 2 7 6 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 0 7 6 2 2 7 6 2 7 M = 6 7 ,M = 6 7 3 6 7 4 6 7 6 1 1 7 6 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 0 7 6 2 2 7 6 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 7 6 0 0 0 0 0 0 7 6 0 0 0 0 0 0 0 7 6 2 2 7 6 2 7 6 7 6 7 6 7 6 7 6 1 1 7 6 1 7 6 0 2 0 2 0 0 0 0 7 6 0 0 2 0 0 0 0 0 7 6 7 6 7 4 5 4 5 1 1 1 0 0 2 0 2 0 0 0 0 0 0 2 0 0 0 0
43 Figure 4-1. Graphical representations of the four symmetric matrices M1,M2,M3,M4
Problem I Now suppose that we are looking for two {−1, +1} sequences A and B of lengths n, such that
PA(i) + PB(i) = 0, i = 1, . . . , m.
Using Lemma 2 we can reformulate this problem as follows: Problem II Find two binary sequences a, b, (viewed as n × 1 column vectors) such that
T T a Mia + b Mib = 0, i = 1, . . . , m.
where a = [a1, . . . , an] and b = [b1, . . . , bn] and ai, bi ∈ {−1, +1}.
We are interested in ±1 values of the 2n variables ai, bi such that
PA(i) + PB(i) = 0, i = 1, . . . , m
or equivalently
T T a Mia + b Mib = 0, i = 1, . . . , m.
Example (continued) Let n = 8 and consider the sequences:
a := [1, 1, 1, 1, 1, 1, -1, -1]; b := [1, 1, -1, 1, -1, 1, -1, -1];
Then we have that PA(i) + PB(i) = 0, i = 1, 2, 3, 4. One can easily verify that we also have:
T T a Mia + b Mib = 0, i = 1, 2, 3, 4.
44 4.3 A Combinatorial Optimization Formalism for PCS
We now use lemma 2 to provide a Combinatorial Optimization formalism for the general problem of finding PCS(n, p). Definition 4. Let p be an integer such that p ≥ 2 and n be a positive integer. A collec- tion A1,...Ap of p sequences of length n each, with elements from {−1, +1} is called a PCS(n, p), if
PA1 (i) + ··· + PAp (i) = 0, i = 1, . . . , m.
The study of PCS(n, p) was discussed systematically in 1990 by B¨omerand Antweiler [52]. They gave necessary existence conditions and produced a diagram summarizing their results for the existence and nonexistence of PCS for all values 2 ≤ p ≤ 12 and 2 ≤ n ≤ 50. They also describe synthesis methods for PCS, using the concepts of mates, interleaving, matrices with orthogonal columns, perfect arrays and periodic products.
Via lemma 2, the search for PCS(n, p) can be expressed as a Combinatorial Optimization
problem:
Find a set of binary sequences a1, a2, ..., ap, (viewed as n × 1 column vectors) such that
T T a1 Mia1 + ··· + ap Miap = 0, for i = 1, . . . , m.
The n × n matrices M1,...,Mm that appear in the Combinatorial Optimization formalism of the PCS(n, p) problem, satisfy a simple additive property, given in the following
lemma.
Lemma 3. 1 M + ··· + M = M 1 m 2
where M is an n × n matrix with elements mij = 1 − δij where δij denotes the usual Kronecker’s delta.
Proof
45 We use the notations of definition 3. The terms PA(1),...,PA(m) are made up from
n(n−1) 2 different monomials of the form ajak (with j 6= k). The n monomials of the form
ajak (with j = k) i.e. the squares, do not appear in the terms PA(1),...,PA(m). Example (continued) Let n = 8 and consider the sequences of Example 2, that form a
PCS(8, 2). The relevant matrices M1,M2,M3,M4, given in Example 1, satisfy the additive
property: 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 M1 + M2 + M3 + M4 = 2 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0
4.4 Applications of PCS and Related Sequences
We give some references to works describing applications of PCS. We do not aim to provide a comprehensive, or by all means complete, treatment of the subject, as this is not
the purpose of the present work. We are merely interesting in giving a flavor of the many different application areas involved.
As noted in [52], PCS are used to construct sequences with desirable properties for radar applications, as described in [53]. Again as noted in [52], PCS intervene in coded aperture imaging [54] and higher-dimensional signal processing applications such as
time-frequency-coding [55] or spatial correlation [56].
The book [57] contains a chapter on applications of combinatorial designs in general, in authentication codes, threshold schemes and group testing algorithms.
46 The book [58] is a rich source of information on applications of sequences with low autocorrelation function properties in general, in signal design for communications, radar, and cryptography applications.
Some categories of PCS, e.g. PCS(n, 2), are used as first rows of circulant matrices to construct Hadamard matrices from two circulant submatrices, see [59] and references therein. These Hadamard matrices are then used for Coding Theory purposes, i.e. to construct binary linear codes with desirable properties. 4.5 Optimization Algorithm
The formulation described in the previous section enabled us to use the algorithms
for the Unconstrained Binary Quadratic Optimization Problem from [60] with slight
modifications. Firstly, the substitution of ai with 2xi − 1, where xi ∈ {0, 1}, transformed the problem to 0–1 domain:
T T a Mia = 0 =⇒ x Qix + n = 0, i = 1, . . . m,
Qi = (qjk),Mi = (mkj)
qjj = 4, j = 1, . . . n 1 q = −2, if m = jk jk 2
qjk = 0, if mjk = 0
T Let di(x) = x Qix + n denote an error corresponding to the ith equation (positive, negative or zero). The sum of the absolute errors for each equation is used as the cost function to minimize:
Xm Xm T f(x) = abs(x Qix + n) = abs(di(x)) i=1 i=1
Search methods used in [60] are based on the 1-flip neighborhood. A solution b
belongs to 1-flip neighborhood of binary vector a if bj = 1 − aj for some index j and
bk = ak for k 6= j. In order to accelerate 1-flip move evaluations, the special data structure is maintained throughout the search process. Let gains(x) denote a vector in which the
47 j-th element represents a gain in the cost function resulting from applying 1-flip operator
to the variable xj:
Xm Xn gainsj(x) = (abs[di(x) + qjj(1 − 2xj) + 2 qij(1 − 2xj)xi] − abs[di(x)]) i=1 i=1,i6=j
The negative elements of the vector gains(x) indicate the variables that can be inverted in order to improve the current solution x.
The tabu search [15] was implemented based on 1-flip neighbourhood (see [60]
for implementation details). At each iteration, the tabu algorithm explores 1-flip neighbourhood of the current solution. The best solution from the neighbourhood is set to be a current solution and the procedure is repeated. A prohibition rules are used
to escape from locally optimal solutions. If the variable xj was inverted during the move,
then it is is prohibited to invert xj for a fixed number of iterations, referred to as a tabu tenure. A random binary sequence was used to initialize the tabu search. The algorithm is restarted after every 10000 iterations with a new random initial solution. The algorithm is stopped whenever the solution x is obtained, such that f(x) = 0. The tabu tenure parameter of the algorithm was set to 21 iterations. 4.6 Results
In this section we give a number of new results, namely solutions to all open problems on page 319 of the second edition of the Handbook of Combinatorial Designs [61].
We use the standard notation, + stands for +1 and − stands for −1. 4.6.1 PCS(50, 2)
A pair of PCS(50, 2) has first been found in using refined pruning methods. However, finding PCS(50, 2) remains a tough algorithmic challenge for conventional methods and so we tested our new formalism with it. We found many PCS(50, 2) and
below we give two such pairs.
[-++-++-++++-+---+-+-+-++++-++---++--+-+++++--+-+++]
48 [-+---+++-++++--+++++-++------+--+-+----+++-+--+++-]
[+-+-+-++-+-+-+++----++--+++--++-++++--+-+++------]
[--++----+-+--+--+-+-++--+++++--+-----++-++--+-----] 4.6.2 Consequences
The new PCS that we found in this chapter, imply a certain number of consequences, stated in the next theorem:
Theorem 4. There is a PCS(50, 2), see appendix, and hence, PCS(50, 2k) exist for all
k ≥ 1. There is a PCS(40, 3), see appendix, and hence, PCS(40, 3k) exist for all k ≥ 1. There is a PCS(44, 3), see appendix, and hence, PCS(44, 3k) exist for all k ≥ 1. There is a PCS(48, 3), see appendix, and hence, PCS(48, 3k) exist for all k ≥ 1. There is a PCS(40, 5), see appendix, and hence, PCS(40, 5k) exist for all k ≥ 1. There is a PCS(44, 5), see appendix, and hence, PCS(44, 5k) exist for all k ≥ 1. There is a PCS(48, 5), see appendix, and hence, PCS(48, 5k) exist for all k ≥ 1. Proof
It is well-known (see [52]) that the existence of PCS(n, p) implies the existence of PCS(n, pk) for all positive integers k ≥ 1. The previous theorem has an important corollary, that (based on our results) settles six more open cases in the table of open cases listed on page 319 of the second edition of the Handbook of Combinatorial Designs [61].
Corollary 3. There exist PCS(44, 6) and PCS(50, 6). There exist PCS(40, 9), PCS(44, 9), PCS(48, 9). There exist PCS(50, 10).
4.7 Conclusion
In view of our new results (as included in the appendix) and their consequences (as stated
in the previous section), it is clear that we have solved all open cases for PCS(n, p) stated
49 in Table 8.17 on page 319 of the second edition of the Handbook of Combinatorial Designs [61]. Therefore, the program started by B¨omerand M. Antweiler in 1990 (see [52]) can now be declared completed.
We present the new PCS(n, p) that we found using our new technique. In conjunction with corollary 1, this settles all open cases in the updated B¨omerand Antweiler diagram stated on page 319 of [61]. PCS(40,3)
[--++++----+++-++++-++-----+---+++-+++-+-]
[-+-+-++-++-++-+++-++-++++----+-++-+--+--] [+++-++--+-++---++---++++++-+-+-++++-+++-]
PCS(44,3)
[--++++-++++--+++--+-++---+-+-+---+--++-+----] [-+-+-++-----+--++---+-+++---++-++-+-+----+--] [+-++--++++++++-+++--+-+++--+--+-++-++++-----]
PCS(48,3)
[+++-+++---+-++++++++-+--+-+-++---+---+--+-++--+-] [++-++-----+--+--+-+--+++-+---+++++----+----+---+] [--+---+---++++-++-+--+++----++-----++--+--+-+-+-]
PCS(40,5)
[++++++-+--+--++-+-++++++-+++-+---++-+--+]
50 [-+++-----++--+------++---+-+-++-++-+-+++] [--+++--+-+--++-+++++--++++---+++-++-----] [--++---++--+----+-+-+--+-++++---++-----+]
[++-+-+-+--++++++--+----++-+-+-++-+--++-+]
PCS(44,5)
[-+--++-+++-++++++---++---+--+-++--++--+-+--+] [++--++-++-++----+-++++-+------+++-----++---+] [--+-+-+-+-+++++--++-+-+----+-----+++--++----] [---+-+----++--+--++-++-+----+++++-+--+-+----]
[+-+---+--+++--+-+---+-+++---+------+--++-+--]
PCS(48,5)
[-++---+-+---++------+-+-++-++----+--+--++---++++] [+-++-+++-++---++-++-+-----++++++-+++--+-+---++-+] [-+-+-+----+-++-+-+++++++---+-++++-++-++++-++-++-] [-+-+-++++--+--+++-+-++--+----++--++-++-----+-+-+] [+----++--+++--+++-+-+++++----++----+++------+-]
PCS(38,6)
[--++++------++--++-+-+-++-+++-++--++--] [--+------+-+-++--+--++-----++-+---+-+] [-++-+-++++--++----+---+--++++++++++++-]
[-+++++--++++-+----+-++--+-+---++--+-+-]
51 [-+++-+-+--++-++-+--+----++-+++-+-+----] [-+-++++---+++++-+-+--+-+++--+++---+--+]
PCS(42,6)
[---++-+-+---++----++-+++++-+-++--+--++---+] [----+-++-++-+-+++-+--+++-+++--+-----+++-+-] [-+-+-++-+++-+-++-+++--++-----+-+++-+----++] [+--+-+---+++----+---+++-+---++-++-----++--] [-++-++-+++-++-+-++++-+--++++-++++++-+-+---] [-+--+--+++---+++++-+++------+-++---++]
PCS(46,6)
[+++++-+-+---+----+++--+-++++--+--++-+-++++++++] [+-+-++-+-+++---++--++-+++-----++-+-++--+---+-+] [++-+-+-+----+++----+--+---+---+++-+-++--+--+-+] [+++--+-+---++-++-+-+++---+--+--+---++------++-] [--+++++-++--++-+----+-+-++++--++-+++++++--+-++]
[----++++-++-+-+--++---+++------+++++-+-++-+++-]
PCS(40,7)
[---++++++-----+--+--+++++-+-++-++--+++-+]
[-+---++-+-+-+-++-++--+++--++-+-++++--+--]
52 [-+++-+---++++--+-++++--+-+--+++--+----++] [++------++++++-+++++++++--+-+---+-++++--] [+-++--+-+++--++-+++-+--+++++-+++---+++--]
[+++-+++-++++-----+---+-++-+-++++--++-+++] [+----+--+-+-+-+--+--+---+++---+--+-++-+-]
PCS(44,7)
[-++++--+--++++--+-++++-++--+-+++--+---+-+-+-] [+-++-+-++-++--++---++-+-+----++--+++-++++-++]
[--+---+-++++++-+---+--++-+-+++-+---++-+-++-+] [--+---+--++++-+---+---++++--+-+-++-+--+---++] [--++++++++--++-+-++++---+++++--+---+-+----++] [-+------+-+++-++++-+-+------++-----+-----+] [+-+++-++++---++-----++-+--+++++----++-+-++--]
PCS(48,7)
[-+++++-++-+++--+--+-+++++--++----+---+--+--+-+--]
[-+-+-++++--++-----++-++-----+++++++--+++-+++-+++] [-++----+-++--+-+++----+-++------++---++--+++--+] [+-+-+--+-++----+---+------+-++++--+-++-+---++---] [++--+++-+-+--+---++-+-+-++--+--+----++++-++++-+-] [++++++--+-+---+-+++-+++-++-+------++++--+-++--]
[+-+--+++-+---++++++++--+-+-+++-++-++-+++---++-+-]
53 PCS(38,10)
[--++---+-+++++++++-+--+-+++--+-+-+----]
[+-+-+++---++++---+--++++--++-++++-+--+] [+++--++-+-----++--+++--+--+--+-+-+---+] [+-++++---++-+++-++++++--+---++-+--+--+] [-+---++-+++++++----+-+-+---++--+-+++-+] [--+--+++-+-+++---+++-----+-+--+------] [-+-+-++--+-++++--+++-+--+++-+++----+--] [------+++--++-+-+------+-+--+--+--++-] [++-++--+-+-++---+-+++--++-+--+-++++-++]
[+--++-+---+--+++-+-++-+--+--++++++----]
PCS(42,10)
[+-+-+-+--++----+++--+-----+++-++++--++--+-] [++--+++++--+++-+--+++-+++-+++--+-+------+-] [---+-+-++-+++-+++++++++-+-++-+-+---+-+-++-] [-++-++--+--+++-+-+-+--+--+-+--+-----++-++-] [-++---+++-++--+-+--+--+-+---++++-++++----+]
[-++--++++-+++++---+++++++--+++++-+--+--+--] [--+++---+++-++-+---+-++----++++++---+--+--] [+--++++-+----+-+++--+--++-+--+-++++++++---] [+---+---++-+-+--+-----++----+-----++-++++-] [---++--++--+-+-----+-+-+---+---++++--+--+-]
PCS(44,10)
54 [+++-+++----+-++-+-+--++-+-++-+-+++++---+--++] [--++--++---+--+-+---+++++-++-+++-++--+-----+]
[+-----+--++------++----++--++---+-----+-+++] [--+-++----++++-+----+-----++-+-+-++----+---+] [---+++-++-+------+---+++++--++---+++++--] [-+-+-++-++-+--+----+++----++---+-+-+--++-+-+] [+---+-+-----++-+++---++-+---++++-+++-+-++-+-] [++--+++++--+--+++-+--+--++---++-++---+-+---+]
[-++------+--++-+-++++++-+++-++-++--++-++++-] [-+----++-+-+-+-+--+-++-++++--+--+++++-+-++-+]
PCS(46,10)
[---+++++++--+----+-+-+++-+-++-+---+++--+++-+-+] [+-+-+--+-+-+++-+++++-+-++-++--+-+-+-+-+-++--+-] [+-+++++++---+----+---++-++--+-++-++--+++--+-++] [--+--+++-----+-----+++-++--+-++-+--+---+++---+] [--++++--++-+++++--++++-++---+--+--+---++--++++] [+--+--+++-++---++++---+++++++---+--+---+-++-++]
[-++++++-+-+++--++-++-+-++-++++---+--+-----+--+] [------++++------+++-----+++-+--+-+--+--++--] [-++---++-+-+++-+------+--+-+----++++--+++++-++] [-+---+--+-+-++-+-+-+---+---+--+++--+----++++--]
PCS(40,11)
55 [--+---+-+---+-++-++++-++++-+++-+++++-++-] [++--++----+++--+-++++-+++----+++-+-++-++] [---+----+--++--++-++-+++-++-+-++----+--+]
[-+++-+-++-+---+++++++---++++++----+++---] [-++++-++--+-+--+--+-+-+--++--+-+--+-----] [--++-+-++--+-++-+-+++++----+-+----++++++] [++--+++---+++-++++++++-++-+-+--++-+---+-] [++---+++++-+++-+----++-+++-+-+-++-+-+-+-] [+--+---+++--+------+---++-++-++-----+-+] [++--+--++++++-+-++----+++-++-++---+--+++] [++-----+--+++--+++-+-+++-+-+-+---+--+++-]
PCS(44,11)
[+-+-++--++++-++--+-+---+-++-+-++-+-++-+-+-++] [++-++---+--+++------+-+++--+-+++------++-++] [++------+++--+--++-+-+++-+-+-+++-+-+-++-+-] [--+--+-+-----+-+----+-----+++-++++---+-+++--] [-++++---+------+-++-++-+--+---++---++++-++--] [++-++-+----+------+-+++---+++++-+++----+--+-]
[+-+---++-+++-+--++-++--++++-++++++-+--++-+++] [---+--+-+-+--+++++--+--+++++++--+++++-++++++] [+-+-+---++----+++-++--+++--++-++---++-++--++] [-+++--+-+++++--++-----+---+---++-++-+-+++++-] [-+-++-----+-+-+++-+--+---+-+---++-++-++--+++]
PCS(48,11)
56 [-++-+-++++++-----+-+-+++++--+++-+++--++++-+--+--] [--+------+++-----+--++---+--+--+----+-++-++-+-+-]
[-----+---+++-----+++-++-+-++++++-+-++--+--++-++-] [-+-+----+-++++++-++-++-+--++--+--+-+----+-+++++-] [+++-++---+-+-++-+-+-+++-+++-++++-+-++--+++----++] [-+--+-+++++---+++--+---+-+-+-+++-+++-+--+--++-+-] [++++++-+-++++--+++------++--++---+++--+--+--+-+-] [----++-----+--++-++++---+++++++--+++--++-++--+++]
[+-+++--+++-++-+--+-+----+++++---+-+--+------+---] [--+-+-+---++--+---+-+---++-+-++++---+-+-+++++-+-]
[--+--++-+-++-+--+++++++-++---++-+++--++++-+++--+]
57 CHAPTER 5 WEIGHTED MAX-SAT PROBLEM
5.1 Introduction
Let X = {x1, x2, . . . , xn} be a set of Boolean variables, i.e., xi can take value either 0 (false) or 1 (true). A propositional formula Φ is said to be in conjunctive normal form
(CNF) if it is a conjunction of clauses C = {C1,C2,...,Cm}, where each clause Ci is a disjunction of literals: ^m |_Ci| Φ = `ij , (5–1) i=1 j=1
where |Ci| is the number of literals in a clause Ci, and `ij is a literal, i.e., a boolean
variable xk, or its negationx ¯k, 1 ≤ k ≤ n. A clause is satisfied if at least one of its literals is true. In the MAXIMUM SATISFIABILITY (MAX-SAT) problem we need to find an assignment of values to the variables that satisfies as many clauses as possible. A
natural generalization of the above problem is to define a positive weight wi for each
clause Ci and search for an assignment, which maximizes the total weight of the satisfied clauses. The MAX-SAT as well as its weighted version remain NP -hard even if each clause has at most two literals (MAX-2SAT problem) [35].
Among various heuristic approaches for solving the MAX-SAT problem we should mention algorithms based on reactive tabu search [62], simulated annealing [63], GRASP [64, 65], iterated local search [66] and guided local search [67]. Detailed surveys on MAX-
SAT, related applications and solution approaches can be found in [68]. Some of the algorithmic codes and benchmark instances are publicly available and can be found at [69–72]. In this note we consider a heuristic approach for solving weighted MAX-SAT based on the global equilibrium search (GES) framework suggested in [17, 49]. We investigate
performance of three variants of the proposed method and compare them with other
existing algorithms on publicly available benchmarks instances. The computational results
58 revealed strong and robust performance of the approach asserting that the techniques similar to the simulated annealing provide a competitive solving tool for weighted MAX- SAT. Finally, we believe that GES-based methods are promising for solving a variety of
discrete optimization problems, which is also confirmed by our work in application of GES for the unconstrained quadratic 0–1 programming problem [60].
5.2 Algorithm for the Weighted MAX-SAT
The algorithm presented in this chapter is based on the GES method described in Chapter 1. The framework of the algortihm is similar to the pseudocode presented in
Figure 2-1. However, our implementation for the weighted MAX-SAT problem is slightly different from the general framework. These differences are due to problem specifics and can be summarized in a following list:
1. The set of elite solution is not used, E = ∅ throughout the algorithm stages.
2. The set of R of solutions that are used to update the memory of the algorithm (Figure 2-1, line 15) consists of the best solution found by the search procedure.
3. The set P of prohibited solution is not used, P = ∅ throughout the algorithm stages.
4. RESTART-criterion=TRUE (Figure 2-1, line 31). In other words, the algorithm deletes all the information after the unsuccesful temperature cycle (complete restart).
5.2.1 Generation Procedure
Given x, the generation procedure (see Figure 5-1) scans through the components of x in a random sequence and the values of the solution components are changed based on the
k k p˜j (i.e. the j-th componentof solution x is set to 1 with probabilityp ˜j , k -is the number of the temperature stage at which generation procedure is applied). The perturbations cycle stops whenever the Hamming distance between x and perturbed solution becomes equal to
the parameter distmax. 5.2.2 Local Search for the Weighted MAX-SAT
Three versions of search method (Figure 2-1, line 15) were implemented and tested.
All of them are based on a 1-flip move operator, which changes the value of a single
59 Require: x – solution to apply random permutations,p ˜k– probability vector corresponding to k-th temperature stage Ensure: 1: dist = 0; j = 1; 2: while (j ≤ n) do 3: if xj = 1 then k 4: if p˜j ≤random[0,1] then 5: xj = 0; dist = dist + 1; 6: end if 7: else k 8: if p˜j ≥random[0,1] then 9: xj = 1; dist=dist+1; 10: end if 11: end if 12: j = j + 1 13: if dist = distmax then 14: return x 15: end if 16: end while 17: return x
Figure 5-1. Generation procedure
variable xi to its negation ¬xi, i.e., from xi to 1 − xi (in terms of 0–1 variables), for some i ∈ 1, n. Furthermore, in order to accelerate move evaluations, the special data structure is maintained throughout the search process. Let gains(x) denote a vector in which the ith
element represents a gain in the total weight resulting from applying 1-flip operator to the
variable xi. The vector gains(x) directly allows us to identify the improving moves. After
applying 1-flip move to variable xk, one can recalculate the vector gains(x) by updating
the entries corresponding to variables which belong to the same clauses as xk. The simplest search method used in our studies is referred to as “1-opt” local search. This procedure applies a number of the 1-flip moves until the solution is being improved. The “k-opt” local search allows algorithm to accept non-improving moves. The 1-flip moves are applied until they lead to better solutions. However, the 1-flip moves are
restricted to the variables from the set M. When there is no improving move available, the
best possible move is applied. Initially, the set M contains all variables, but whenever a
60 Require: x – solution, gains(x)– the vector of gains Ensure: 1: repeat 2: Generate random permutation RP of the set {1,. . . ,n} 3: ∆ = 0 4: for k = 1 to n do 5: j → RP [k] 6: if gj(x) ≥ 0 then 7: ∆ = ∆ + gj(x) 8: xj = 1 − xj 9: g(x) ←recalculate gains(x) 10: end if 11: end for 12: until ∆ > 0 13: return x
Figure 5-2. Local Search 1-opt non-improving move is applied to some variable, it is removed from the set M. If the set M becomes empty, the search is restarted from the best solution found during the local search. The local search returns the best solution found if there was no improvement in the last nbad iterations. The third search method investigated in this note is based on the tabu method [15]. method used in our algorithm is presented in Fig. 5-4. 5.3 Computational Experiments
5.3.1 Benchmarks
• jnh [70] (“easy” instances). The number of variables is n = 100. The number of clauses ranges from m = 800 to m = 900. These instances were used for computational experiments in [65–67, 73].
• rndw1000b [48] (“harder” instances). The number of variables and clauses is n = 1000 and m = 11050, respectively. This set was utilized in [66, 73]
• rndw1000c (“harder” instances). We constructed this additional sets of test instances in order to confirm the results obtained on rndw1000b. Test instances were generated from rndw1000b randomly modifying clause weights uniformly between 1 and 1000 (i.e., we followed the same idea, which was applied for generation of jnh instances, see [70]).
61 Require: x – initial solution, g(x)– vector of gains, nbad – # of moves without improvement Ensure: 1: xbest = x; ∆best = 0; ∆cur = 0; step = 0; stepimpr = 0; impr =true 2: while impr do 3: M = {1, 2, . . . , n}; 4: impr =false 5: repeat 6: repeat 7: ∆ = 0 8: Generate random permutation RP of the set Moves 9: for k = 1 to |M| do 10: j = RP [k] 11: if gj(x) ≥ 0 then 12: ∆ = ∆ + gj(x) 13: xj = 1 − xj; g(x) ←recalculate gains(x) 14: step = step + 1 15: end if 16: end for 17: ∆cur = ∆cur + ∆ 18: until ∆ > 0 19: if ∆cur ≥ ∆best then 20: if ∆cur > ∆best then 21: ∆best = ∆cur; stepimpr = step; impr =true; 22: end if 23: xbest = x; 24: end if 25: ∆ = −∞ 26: for k = 1 to |M| do 27: j = RP [k] 28: if gj(x) ≥ ∆ then 29: ∆ = gj(x); indbest = j 30: end if 31: end for
32: ∆cur = ∆cur + ∆; xindbest = 1 − xindbest 33: g(x) ←recalculate gains(x); step = step + 1 34: M = M − {indbest} 35: if step − stepimpr > nbad then 36: break 37: end if 38: until |M| > 0 39: x ← xbest; ∆cur ← ∆best; g(x) ← recalculate gains(x) 40: end while 41: return xbest
Figure 5-3. Local Search k-opt
62 Require: x – initial solution, g(x)– vector of gains, nbad – # of moves without improvement, tabu– the tabu tenure Ensure: 1: xbest = x; step = 0; impr =true 2: while impr do 3: impr =false; ∆best = 0; ∆cur = 0; stepimpr = step 4: M = {1, 2, . . . , n}; last used(j) = −∞, j ∈ M 5: repeat 6: repeat 7: ∆ = 0 8: Generate random permutation RP of the set M 9: for k = 1 to |M| do 10: j = RP [k] 11: if step − last used(j) > tabu OR f(x) + gj(x) > f(xbest) then 12: if gj(x) ≥ 0 then 13: ∆ = ∆ + gj(x); last used(j) = step; xj = 1 − xj 14: g(x) ←recalculate gains(x); step = step + 1 15: end if 16: end if 17: end for 18: ∆cur = ∆cur + ∆ 19: until ∆ > 0 20: if ∆cur ≥ ∆best then 21: if ∆cur > ∆best then 22: ∆best = ∆cur; stepimpr = step; impr =true 23: end if 24: xbest = x 25: end if 26: ∆ = −∞ 27: for k = 1 to |M| do 28: j = RP [k] 29: if step − last used(j) > tabu OR f(x) + gj(x) > f(xbest) then 30: if gj(x) ≥ ∆ then 31: ∆ = gj(x); indbest = j 32: end if 33: end if 34: end for
35: ∆cur = ∆cur + ∆; last used(indbest) = step; xindbest = 1 − xindbest 36: g(x) ←recalculate gains(x); step = step + 1 37: until step − stepimpr < nbad 38: x = xbest; g(x) ←recalculate gains(x) 39: end while 40: return xbest
Figure 5-4. Local Search TABU
63 Table 5-1. Results for jnh benchmarks Problem sa ils tabu ges 1-opt ges k-opt ges tabu gls1 gls2 jnh04 35.09 6.57 67.83 10.10 2.24 0.70 1.72 0.84 jnh05 1.24 0.29 41.63 0.29 0.57 0.10 1.11 0.10 jnh06 0.07 0.12 0.44 0.13 0.09 0.06 0.09 0.13 jnh08 2.06 3.64 1.28 0.33 0.53 0.13 0.02 0.04 jnh09 5.34 37.73 4.68 2.75 1.14 0.12 2.00 0.77 jnh10 8.95 4.17 30.57 0.58 0.49 0.12 0.04 0.04 jnh11 21.63 18.55 27.25 4.28 3.19 0.60 0.48 0.57 jnh13 0.51 0.26 0.76 0.17 0.29 0.03 0.03 0.02 jnh14 0.54 1.39 4.12 0.64 0.90 0.20 0.03 0.02 jnh15 1.87 0.65 27.50 0.81 0.41 0.08 0.07 0.11 jnh16 47.74 0.12 25.80 5.94 2.93 0.36 3.21 0.57 jnh18 5.65 7.74 13.22 0.89 0.51 0.08 1.14 1.01 jnh19 3.21 73.31 28.34 27.70 20.43 2.69 0.07 0.18 jnh202 32.22 34.33 63.62 4.34 1.86 0.36 0.04 0.05 jnh203 16.02 18.15 2.20 0.48 0.65 0.78 0.04 0.07 jnh208 1.35 2.98 0.99 0.38 0.45 0.23 0.13 0.07 jnh211 0.32 0.50 0.74 0.76 0.46 0.04 0.02 0.05 jnh214 2.70 1.44 6.42 0.24 0.34 0.18 0.08 0.04 jnh215 11.27 0.69 5.04 1.30 0.48 0.17 0.05 0.05 jnh216 0.50 1.45 10.70 0.44 0.38 0.10 0.30 0.27 jnh219 1.99 34.26 13.98 8.15 2.58 0.14 0.12 0.14 jnh302 0.07 0.23 1.23 0.23 0.18 0.11 0.13 0.19 jnh303 2.39 0.70 7.81 0.81 1.08 0.13 0.35 0.21 jnh304 0.71 1.75 7.87 1.52 0.19 0.03 0.08 0.10 jnh305 1.90 3.88 9.08 3.67 1.91 0.25 5.43 1.18 jnh306 0.27 0.31 1.32 0.29 0.21 0.04 0.03 0.04 jnh307 0.32 0.79 3.10 0.26 0.47 0.10 0.22 0.28 jnh308 48.85 66.23 36.63 8.49 2.58 0.19 0.24 0.22 jnh309 0.17 2.03 0.21 0.20 0.16 0.05 0.14 0.10 jnh310 0.42 1.88 209.53 1.26 2.04 0.07 0.29 0.38 Total time(sec): 255.38 326.16 653.89 87.44 49.73 8.21 17.68 7.81
Table 5-2. Results for rndw1000b benchmarks
Problem ges tabu gls1 gls2 rndw1000 meant meanf bestf meant meanf bestf meant meanf bestf b01 420 5552786 5553057 2381 5551629 5552657 2533 5551695 5552065 b02 319 5519999 5520245 2412 5519335 5519829 2526 5519337 5519605 b03 484 5579425 5579673 2300 5578475 5579159 2518 5578744 5578208 b04 1258 5504266.9 5504348 2322 5502842 5503341 2531 5503249 5504246 b05 624 5527403 5527445 2392 5525988 5527357 2528 5526290 5527059 b06 333 5523689 5523832 2329 5522259 5523423 2538 5523143 5523559 b07 247 5513115 5513393 2364 5511917 5512232 2525 5512292 5512983 b08 829 5491782 5491999 2313 5490155 5490566 2523 5490915 5491696 b09 451 5543211 5543353 2498 5542028 5543114 2557 5542167 5542606 b10 1270 5554784 5554855 2276 5553519 5554709 2543 5553444 5553745
Table 5-3. Results for rndw1000c benchmarks
Problem ges tabu gls1 gls2 rndw1000 meant meanf bestf meant meanf bestf meant meanf bestf c01 393 5520297 5520841 1962 5519435.5 5520066 2041 5519233 5519983 c02 834 5497278 5497582 1946 5495597 5496645 2018 5496246 5497066 c03 4117 5541418 5541452 1954 5539629.3 5540355 2028 5540140 5541399 c04 491 5522087 5522372 1958 5521264.2 5521925 2032 5521756 5521947 c05 593 5501340 5501664 1961 5500468.1 5501213 2040 5500578 5501249 c06 543 5502583 5502903 1959 5501467.5 5502483 2040 5501556 5502016 c07 409 5505078 5505222 1955 5503706.8 5504510 2038 5504234 5504984 c08 676 5533540 5533641 1958 5532340.6 5533259 2038 5532846 5533450 c09 482 5544435 5544702 1958 5542906.7 5544191 2038 5543638 5544265 c10 422 5524889 5525283 1963 5523939.1 5524726 2039 5524047 5524505
64 5.3.2 Algorithms and Software
The GES algorithm was implemented in C++. All computational experiments were conducted using PC with Pentium 4 3.00 GHz and 1.00 Gb of RAM. The parameters of
the algorithm were the same for all computational experiments: K = 50, distmax = n/2, ngen = 100, tabu = n/10, nfail∗ = 1, nbad = 12n. Initial probabilities at the beginning of each temperature cycle are all initially set to 0.5. For the temperature schedule the
−6 log µ50−log µ1 following values were used: µ0 = 0, µ1 = 3 ∗ 10 , µk = µk−1 ∗ 48 for k = 2,... 50,
µ50 = 10. In order to compare the proposed algorithm with other techniques discussed in the literature, we used the software packages available online [48, 72]. The website of Yagiura [48] provides the codes for a set of algorithms for the weighted MAX-SAT problem. Among those, the tabu search algorithm (tabu), simulated annealing algorithm (sa) and
iterated local search (ils) dominate the rest of the algorithms given at [48] in terms of their computational efficiency. Another technique used in our studies is guided local search (GLS). In [67, 72, 74] the authors provide an implementation of guided local search for the weighted MAX-SAT
problem. The authors provide two versions of GLS referred to as gls1 and gls2 (here the
same notation is used as in [72, 74]). The demo version software of their implementation is available at [72]. 5.3.3 Results and Conclusions
Results of computational experiments on “easy” instances are given in Table 1. For these problems all of the algorithms were able to find optimal solutions. The data presented in Table 1 represents the average computational times till the optimal solution found for 10 independent runs of each algorithm. In terms of CPU time, gls1, gls2 and GES based algorithms outperform the rest of the techniques. The best average
performance is demonstrated by gls2 algorithm. However, it is rather obvious that jnh
test instances are not computationally difficult enough to draw conclusive comparisons
65 between these three methods. Therefore, further testing on harder instances (rndw1000b and rndw1000c) was performed and limited only to this group of the algorithms. The algorithms gls1 and gls2 were tested with termination criteria parameter
’maximal number of repairs’ set to 66000. The best solutions obtained by gls1 and gls2 for each instance were used as target solutions for ges tabu algorithm, i.e., ges tabu terminated as soon as the solution of the same or better quality than the target solution was obtained. We present the data for the average solution quality (meanf), average solving time (meant), the best solution found (bestf). These values are calculated based on 10 independent runs of each algorithm. The results are presented in Tables 2 and 3.
The performance data in Tables 2 and 3 indicates that, overall, on “harder” instances ges tabu performs robustly and dominates both gls1 and gls2. For every instance and every run of ges tabu, the algorithm was able to find the solution of the same or better quality as the best solutions found by gls1 and gls2 in 10 runs. Besides finding better solutions in terms of quality, ges tabu was spending less CPU running time to obtain them (except rndw1000c03).
66 CHAPTER 6 JOB SHOP SCHEDULING PROBLEM
6.1 Introduction
The minimum makespan problem of job-shop (JSP) consists in scheduling the set of jobs J on some set of machines M with the objective to minimize completion time needed for processing all jobs, subject to constraints that: (i) each job has a predefined processing order through all machines (precedence constraints), and (ii) each machine can process at most one job at a time (resource constraints). If a machine starts processing any job it must finish it without any interruption (non-preemptive). This problem is known to be NP-hard [75], but even within this class of problems it is considered as one of the most challenging. Exact methods were successfully applied to problems of small dimensions, but for job shop problems with more than 15 machines and 15 jobs they are not able to find a high-quality solution with reasonable computational effort. For large dimensions there is a need for good approximate algorithms. The local search based methods were successfully applied to such problems: tabu method [76],[77], greedy randomized adaptive search [78], and simulated annealing [79]. The detailed review of the methods for the JSP can be found in [80] and [81]. An approximate algorithm for the job-shop scheduling problem based on the Global Equilibrium Search (GES) method [17] is proposed in this chapter. The main idea of this approach is to use the history of the search for guiding it into the areas with potentially good solutions. At first, the minimum makespan problem of job-shop scheduling is formalized in terms of a mathematical model, and then the proposed heuristic algorithm based on the GES method for this problem is described. Finally, the results of the computational experiments on the well-known benchmark instances are shown, and our approach is compared with some of the best-performing algorithms for job-shop scheduling.
67 6.2 Mathematical Model and Notations
For each problem of job-shop scheduling we have a set M = {M1,M2,...,Mm} of machines and a set O = {σ0, σ1, . . . , σN+1} of operations, where σ0 and σN+1 denote fictive ”start” and ”finish” operations. Each operation σ ∈ O has two properties: a machine
M(σ) ∈ M that must process it and its processing time p(σ)(p(σ0) = p(σN+1) = 0). Additionally, let A denote the set of pairs of operations which are tied by precedence
constraints (i), and Ek denote the set of all pairs of operations which require machine Mk for their processing (k = 1, . . . , m).
Further, let s(σ) denote the start time of the operation σ, and let s(σ0) = 0. The problem is to find a starting time for each operation σ ∈ O (schedule) such that
max[s(σ) + p(σ)] (6–1) σ∈O is minimized subject to
s(σ) ≥ 0, σ ∈ O (6–2)
s(ω) − s(σ) ≥ p(σ), (σ, ω) ∈ A (6–3)
s(ω) − s(σ) ≥ p(σ) ∨ s(σ) − s(ω) ≥ p(ω),
(ω, σ) ∈ Ek, k = 1, . . . , m. (6–4)
The value given in (6–1) is often referenced to as the makespan of the schedule represented by starting times of operations from O. This problem can be described using the disjunctive graph model of Roy and
Sussmann [82]. Let G = (O, A, E) denote the disjunctive graph, where O is a set of nodes
(a node for each operation), A is a set of directed arcs ((σ0, σ) ∈ A and (σ, σN+1) ∈ A for Sm all σ ∈ O), and E = Ek is a set of undirected arcs. The weight of each node is given k=1 by the processing time of the corresponding operation. Let S (selection) denote a set of directed arcs which is obtained from E by choosing the direction for each of its arcs. Then
68 let GS = (O,A ∪ S) be a directed graph that corresponds to the selection S. If GS is an acyclic graph, then it defines a set of feasible schedules for the problem (1)-(4), and there
is an obvious one-to-one correspondence between GS and the best schedule from this set:
s(σ) in the best schedule is equal to the longest path from σ0 to σ in DS. Furthermore,
the length of the longest path in GS is equal to the makespan of this schedule. Thus we can reformulate the problem (1)-(4) as follows. We should find a selection S that
minimizes the length of the longest path in the corresponding directed acyclic graph GS. The following notations are useful for the further discussion. Let JP (σ)(JS(σ)) denote an operation for which (JP (σ), σ) ∈ A ((σ, JS(σ)) ∈ A). Assuming S to be a
selection with corresponding acyclic directed graph DS, let MP (σ)(MS(σ)) denote an operation for which (MP (σ), σ) ∈ S ((σ, MS(σ)) ∈ S). Let q(σ) be a length of the longest
path from σ to σN+1, and s(σ) be a length of the longest path from σ0 to σ (start time).
The longest path in DS is referenced to as a critical path, and the operations which belong to this path are called critical operations. We can decompose any critical path into a set of
critical blocks: Critical operations σ (σ 6= σ0) and ω (ω 6= σN+1 and s(σ) < s(ω)) belong to the same critical block if and only if there is a path of length l between them with all arcs from S and s(σ) + p(σ) + l = s(ω). 6.3 Algorithm Description
The global equilibrium search method [17] has some common features with the simulated annealing method. It has been applied to many discrete optimization problems, and very promising results were obtained [17]. The main idea of the GES method is to collect information about the solution space for its further use in the next stages of the search. The search is organized as a series of temperature cycles. During each stage of the temperature cycle some set of solutions is randomly generated. On the early stages
of each temperature cycle the algorithm generates absolutely random solutions, but on the further stages the solutions are generated in such a way that they are more and more
likely to have some common features with the best solutions found in the previous stages
69 procedure GES for JSP initialize algorithm parameters(iter, maxiter, ngen, maxnfail) while (iter < maxiter) do Construct random feasible solution x bestx ← x F ← bestx {F – set of already known solutions} nfail ← 0 {number of full cycles without improvement} while (nfail < maxnfail) do k ← 0 {k – number of current temperature} while (k < K) do calculate generation probabilities(prk,F ) cur generation ← 0 while (cur generation < ngen) do x ←generate solution(x, prk) x ← improve solution(x) if (f(x) = f(bestx)) then cur generation ← cur generation + 1 continue end if if (f(x) < f(bestx)) then bestx ← x nfail ← 0 end if F ← F ∪ x cur generation ← cur generation + 1 end while k ← k + 1 end while nfail ← nfail + 1 F ← ∅ ∪ bestx end while end while return the best solution found end GES for JSP
Figure 6-1. GES for JSP
(on the last stage the algorithm just generates the best solution found during the search). Such organization allows us to alternate between diversification and intensification of the search in a very effective manner. The general scheme of our implementation of the GES algorithm is outlined in Figure 6-1.
70 The main cycle of the algorithm is performed while some stopping criterion is not satisfied. The algorithm in Figure 1 runs until maxiter iterations have been performed. For the generation of feasible solutions we use a set F of known solutions. Since it is impossible to keep all of the solutions in the memory of the computer, we use reduced information about the solutions. In our implementation we store two values for each variable: the value of the cost function of the best solution in which the variable is equal to zero and the best value of the cost function when the variable is equal to one. It appears that for the job shop scheduling problem this simple data allows us to construct a quite effective algorithm based on the GES technique. For simplicity of the description of the algorithm we will use the notation F , as if we keep all the solutions in the memory. In order to generate random feasible solutions, we introduce, for each variable, the probability of being equal to one or zero. These probabilities (pr) are uniquely defined by the set F and the current temperature. There are K temperature stages, and for each stage we have a predefined temperature value. The temperature increases as the algorithm proceeds from stage to stage. At the beginning of each stage the probabilities for generation of solutions are calculated. The details of the generation procedure (calculate generation probabilities) are described in the next section. At every stage we generate ngen random feasible solutions, and the set F is updated with new solutions. After that we change the current temperature, and the new stage begins with recalculation of generation probabilities. In our implementation after the temperature cycle is finished, we ”forget” all solutions found during the cycle except the best solution by setting F = {bestx}. Furthermore, if the algorithm runs for maxnfail cycles without an improvement of the best solution then we set F = ∅; in other words, the search process restarts. There are several basic elements which have to be defined more properly. The main part of this framework is the procedure that generates random solutions. It almost always appears that randomly generated solutions can be improved by applying some local search
71 procedure. In our implementation, we use local search together with a heuristic procedure which conducts an extensive search for the improvements in the neighborhood of the local minima. Finally, the previous search results must be somehow used to encourage the
generation of high-quality solutions. In the next subsections, the implementation of these elements is described in detail.
6.3.1 Encoding and Generation Procedure
For each pair of operations which require the same machine for processing, we introduce a probability that one operation is processed before the other one. These
probabilities are used in the generation procedure to determine the orderings of the
operations. Let pr denote a vector which is composed of all such probabilities, and let
prσ,ω be the element of pr which corresponds to the probability of σ being processed before
ω (prω,σ = 1 − prσ,ω). We can encode any feasible schedule as a binary vector. For each ordered pair of operations processed by the same machine, we introduce the binary variable which is equal to one if the first operation is processed earlier than the second, and equal to zero otherwise. Let x denote a binary vector which is composed of all such variables, and
let x(σ, ω) denote the element in x which corresponds to the operations σ, ω ∈ O; thus x(σ, ω) = 1 means that in the solution represented by x, operation σ is processed earlier than ω. Let X denote a set of all binary vectors x which represent feasible solutions. All these vectors must encode different processing orderings; f(x) is equal to the makespan of the solution that corresponds to x. Let S be a selection representing the initial solution, then the full set of moves N1(S) can be defined as a set of pairs (σ, ω) such that: σ, ω are critical operations, (σ, ω) ∈ S,
s(σ) + p(σ) = s(ω). Let S0 denote a selection which is obtained from S by applying move
(σ, ω) ∈ N1(S). Then graph DS0 can be easily derived from DS by: 1. Removing arcs (σ, ω),(MP (σ), σ),(ω, MS(ω);
72 function generate solution(pr,x){pr – probabilities, x – initial solution} nswaps ← random integer from [1,N] i ← 0 tabuSet ← ∅ {set of tabu moves} while (i ≤ nswaps) do C ← set of moves (σ, ω) for the neighborhood N1(x) while (|C|= 6 0) do (σ, ω) ∈ C {random move selection} C ← C − (σ, ω) if (σ, ω) ∈/ tabuSet then rand ← random value from [0,1] if rand > prσ,ω then Apply move (σ, ω) to x tabuSet ← tabuSet ∪ (ω, σ) C ← ∅ end if end if end while i ← i + 1 end while return transformed solution x
Figure 6-2. Procedure that generates solution
2. Adding arcs (ω, σ),(MP (σ), ω),(σ, MS(ω). A very useful property of such transformation is that it can never lead to a cyclic graph DS0 [79]. Empirical analysis reveals that it is much more efficient to apply a series of random moves within the N5-neighborhood to escape from the local minima than to construct
a totally new solution. In [83] it is shown that the average number of moves that are required to escape from a local minimum is usually very small for most job shop
scheduling problems. The procedure that generates solutions is outlined in Figure 6-2. We have already introduced the probabilities for generation of the initial solutions. Now we shall describe how we use the solution history to update these probabilities. The temperature cycle for the GES algorithm is determined by a set of K + 1 temperatures µ0, . . . , µK , such that µ0 < µ1 < . . . < µK . Let F denote a set of solutions
73 that we know for a given problem (F ⊆ X), and let F q(σ, ω) = {x ∈ F : x(σ, ω) = q}, where q = 0, 1 and σ, ω ∈ O. In our implementation of the global equilibrium search method the probabilities for solution generation for a given temperature µk are:
0 k prσ,ω prσ,ω = 0 0 prσ,ω + (1 − prσ,ω) exp[−µk( min f(x) − min f(x))] x∈F 0(σ,ω) x∈F 1(σ,ω)
This simplification allows us to reduce both computational time and memory usage for our algorithm. It appears that for the job shop scheduling problem, it does not affect the overall quality of algorithm performance.
6.3.2 Local Search
In contrast to most recent local search based methods for the JSP, our local search is based on the N4 move operator proposed by Grabowski et al. [84] (we use the neighborhood notation introduced in [85]). The N4-neighborhood consists of solutions obtained by moving a critical operation either to the beginning or to the end of its critical block. In many applications the priority has been given to the highly restricted N5 move operator. The solutions from the N5-neighborhood are obtained by changing the processing order of two consecutive critical operations such that at least one of them is either the first or the last operation of its critical block. The main drawback of such a restriction is that the resulting search space becomes disconnected, i.e. the existence of the path, with respect to the N5, from any arbitrary solution to the optimum is not guaranteed. On the contrary, the N4 move operator induces the connected search space. Additionally, as the N5 move operator is just a restriction of the N4 move operator, it is clear that the latter provides a more thorough search. Nowicki and Smutnicki proposed an effective method of the makespan calculation for the N5 move operator [4], which significantly decreases the overall time spent on move evaluations. Based on their approach, we use the similar accelerator for the N4 move
74 operator. The details of such acceleration are simple, but quite cumbersome, thus we refer the reader to [13] for the details. As a result of such acceleration, the move evaluations are performed up to 3 times faster.
In order to provide an even more thorough search, we introduce a procedure, which has some common features with the iterated local search technique [86], [83]. Whenever an operation is moved either to the beginning or the end of the block, the reason that such move does not improve the solution is because of a constraint coming from the job predecessor of some operation which belonged to the same critical block as the moved operation.
Therefore, after identifying such an operation, we push its job predecessors back in the processing order to remove this specific constraint. After the transformation is finished, the local search starts from the obtained solution (see Figure 6-3). This procedure was discussed in [13] and is used in our studies as an enhancement of the local search. The investigation of its impact on the algorithm performance was provided in [13]. Whenever two solutions have the same makespan value, we consider the total processing time of their critical operations. The solution with smaller total processing time is considered an improving solution. This rule allows us to take into account cases where we encounter multiple critical paths (Figure 6-3: line 27).
75 Require: x – a local minimum Let M = N4(x) (Grabowski’s neigborhood) while M 6= ∅ do choose randomly y ∈ M M = M − y let B be a set of operations σ, ω for which y(σ, ω) 6= x(σ, ω) find an operation σ ∈ B such that end(JP (σ)) = head(σ) if σ 6= ∅ then ω = JP (σ) while σ 6= ω do if head(JS(ω)) 6= end(ω) then ω = JS(ω) else if JP (ω) 6= ∅ and end(JP (ω)) = head(ω) then ω = JP (ω) else if MP (ω) 6= ∅ then Switch MP (ω) and ω in y if makespan of the resulting solution is worse then break end if else break end if end while end if y=localSearchN4(y) {Starting in y} if f(y) < f(x) or f(y) = f(x) and y is an improving solution then RETURN y {Improvement!} end if end while RETURN x {No improvement}
Figure 6-3. Improvement procedure
6.3.3 Acceleration of Move Cost Estimation
Nowicki and Smutnicki (2002) proposed an efficient scheme for acceleration of the local search. We have modified their approach for the N4-neighborhood. Let FS =
(FS(σ0),...,FS(σN+1) denote a list of operations from O, which is a topological order of nodes from graph DS. Hence, this list must satisfy the following conditions: for any pair
76 of operations σ, ω ∈ O such that (σ, ω) ∈ A ∪ S, it is required that fS(σ) < fS(ω), where fS(σ) and fS(ω) are the positions occupied by σ and ω in FS. Let us consider a case when we apply the move (σ, ω) ∈ N4(S) to S, and let σ be the
first operation of critical block. If the resulting selection S0 leads to an acyclic graph DS0, then it is obvious that for every operation δ ∈ O, such that fS(δ) < fS(σ), the value s(δ) will not change: s(S, δ) = s(S0, δ), where s(S, δ) and s(S0, δ) denote a length of the longest path from σ0 to δ in graphs DS and DS0 accordingly; similarly, for every operation δ ∈ O, such that fS(δ) > fS(ω), the value q(δ) will not change. In Figure 6-4 the procedure for a quick move estimation is outlined, which uses this property. This procedure returns the
lower bound of the longest path in DS0. Here s(S, δ) denotes the length of the longest path from σ0 to δ in the graph DS = (O,A ∪ S); q(S, δ) denotes the length of the longest path
−1 from δ to σN+1 in the graph DS = (O,A ∪ S), fS (i) denotes the operation such that −1 fS(fS (i)) = i. This procedure can be further improved so that it can return an exact
value of s(S0, σN+1), as it is done in [87]. Only slight modifications for this procedure are necessary when σ is the last operation of its critical block. 6.4 Computational Results
The GES algorithm described in this chapter was implemented in C language. All computational experiments were conducted using the personal computer Pentium 2.8 GHz with 512 Mb of RAM. The parameters of the algorithm were the same for all computational experiments: maxnfail = 3,K = 40, ngen = 100. Initial probabilities at the beginning of each temperature cycle are all initially set to 0.5. The choice of temperatures was done in such a manner that algorithm converges to the best solution found at the end of the
6 ln(µK )−ln(µ1) temperature cycle. So we set µ0 = 0, µ1 = 4 · 10 , and µk = µk−1 exp( K−2 ), (k = 2,...,K − 1). Thus there was no tuning of the algorithm for any particular instance.
The proposed algorithm has been tested on the benchmark problems taken from
OR-library [1]. We used several well-known problem classes from the literature:
77 function estimate move(S, FS, (σ, ω)){σ – first operation in critical block} est ← 0 If S0 (resulting selection) is a cyclic selection return ∞ s(S0, δ) = s(S, δ), for all δ such that fS(δ) < fS(σ) q(S0, δ) = q(S, δ), for all δ such that fS(δ) > fS(ω) q(S0, ω) = q(S0,MS(ω)) + p(MS(ω)){not final value} s(S0, ω) = max[s(S,JP (ω)) + p(JP (ω)), s(S, MP (σ)) + p(MP (σ))] s(S0, σ) = max[s(S, JP (σ)) + p(JP (σ)), s(S0, ω) + p(ω)] bi ← fS(ω) − 1 ; fi ← fS(σ) + 1 while bi 6= fS(σ) do −1 −1 δ ← fS (bi); γ ← fS (fi) q(S0, δ) = max[q(S0,JS(δ)) + p(JS(δ)), q(S0,MS(δ)) + p(S0,MS(δ))] s(S0, γ) = max[s(S0,JP (γ)) + p(JP (γ)), s(S0,MP (γ)) + p(MP (γ))] if (bi ≤ fi) then est ← max[est, s(S0, δ) + p(δ) + q(S0, δ), s(S0, γ) + p(γ) + q(S0, γ)] end if if (est > s(S, σN+1) then return est end if bi ← bi − 1 fi ← fi + 1 end while q(S0, σ) = max[q(S0,JS(σ)) + p(JS(σ)), q(S0,MS(σ)) + p(S0,MS(σ))] q(S0, ω) = max[q(S0,JS(ω)) + p(JS(ω)), q(S0, σ) + p(σ)] est ← max[est, s(S0, σ) + p(σ) + q(S0, σ), s(S0, ω) + p(ω) + q(S0, ω)] return est
Figure 6-4. Estimation procedure
1) 10 problems denoted as (ORB1-ORB10) due to Applegate and Cook [88]; 2) 40 problems denoted as (LA01-LA40) due to Laurence [89];
3) 80 problems denoted by (TA1-TA80) due to Taillard [77]. Optimal solutions are known for 47 problems from this class. 4) 80 problems denoted by (DMU1-DMU80) due to Demircol et al. [90]. The computational results are given in the tables below. For the problems with an unknown optimal solution, we provide its lower bound (LB) and upper bound (UB). We compare our results with those obtained by Balas and Vazacopoulos (1995a)(BV),
78 Pezella and Merelli (2000) (TSSB). We present the original computational times for these approaches; thus, TSSB was tested on Pentium 133Mhz and BV on Sun Sparc 330. Additionally, we have implemented and tested a multi-start version of TSAB algorithm, which was proposed by Nowicki and Smutnicki (1996); we refer to it as RTSAB. This algorithm initially starts with a random solution. After each run of TSAB, the best solution found during that run is used to generate a new initial solution for the next run of TSAB. For such generation we use the function outlined in Figure 6-2, where all
1 components of vector pr are set equal to 2 . In our implementation of TSAB algorithm we use the acceleration proposed in [87]. Each run of TSAB uses the following parameters: maxt = 8, maxl = 5, maxc = 2, maxδ = 100, and maxiter = 250000, see [76] for details. We have also investigated the behavior of our algorithm when the generation
1 probabilities are all fixed and are equal to 2 for all iterations. Hence, in this case the algorithm doesn’t use any of GES features. We refer to this algorithm as RIMP; its testing results provide a good illustration of the performance of the simple combination of the local search and the improvement heuristic described above and allow to reveal the enhancement provided by GES strategy. In comparative tests we use a notation RI, which is a percentage by which the best found solution (with makespan CMax) is above the best-known upper bound:
CMax − UB RI = 100% UB
All results for other approaches were taken from the literature and some results were taken from Taillard’s home page (http://www.eivd.ch/ina /collaborateurs/etd/default.htm). The results for problems La01-La40 and Orb01-Orb10 are shown in Tables 6-1,6-2. All of these instances were solved optimally by our approach. The comparison with algorithm TSSB [91] is given in Table 6-5.
79 TA benchmarks are one of the hardest instances for job shop scheduling. Table 6-3 shows the results for these problems. Despite the fact that there was a lot of time consuming experiments on solving these problems with many different algorithms, our approach was able to find several new upper bounds (TA11,TA15,TA19,TA20,TA32,TA46). In general the proposed algorithm is performing quite well on all sets of problems comparatively with shifting bottleneck procedure [92] and tabu search algorithm [91]. The computational results for RTSAB reveal that GES outperforms it on the prevailing majority of instances. Comparison with RIMP shows that GES is performing significantly better. For easy problems, the use of GES methodology allows us to decrease solving time. On the other hand, this strategy allows us to obtain a better solution for hard instances.
The average results for all algorithms are given in Table 6-6.
The DMU class of problems consists of 80 instances. Instances DMU41-80 are considered as particulary hard [90], but computational experiments showed that our approach allows us to obtain high-quality solutions even for these fragile problems and for many of them the best upper bounds were improved. We tested each problem in such a way that the algorithm stopped if either the best known upper bound was improved or overall running time exceeded 30000 seconds. The computational experiments for these problems are given in Table 6-7.
6.5 Concluding Remarks
In this chapter, a new heuristic approach for the job shop scheduling problem based on the global equilibrium search method has been proposed. Computational experiments on a wide range of benchmark problems were conducted, and for many of them the algorithm was able to find optimal or near optimal solutions. For all experiments, the algorithm’s parameters were fixed; so there was no tuning for each problem. In spite of numerous, time-excessive computational experiments, which were conducted in the last decades, our approach was able to improve many of the known upper bounds. GES method has been tested on many combinatorial problems, and in this chapter we show
80 that this approach is quite good for one of the most challenging NP-hard problems. Our modification of GES algorithm is a very powerful and competitive tool for job shop scheduling problems comparing to other best-performing algorithms. The ideas introduced by the global equilibrium search method provide a very promising approach for the construction of algorithms for discrete optimization problems.
81 6.6 Benchmark Problems
We use the following notations:
Opt − the optimal value of the makespan
LB − the best lower bound for the optimal makespan
UB − the best upper bound for the optimal makespan, if we found the new upper bound then the old upper bound is presented in brackets
|J|x|M| − problem size (number of jobs x number of machines)
GES1 − best solution by GES algorithm run for 5000 seconds
GES2 − best solution by GES algorithm run for 30000 seconds
TSSB − best solution by TSSB [91]
Time (sec) − GES computing time in seconds
Tav − average computing time
RTSAB − best solution by RTSAB algorithm run for 5000 seconds
RIMPR − best solution by RIMPR algorithm run for 5000 seconds
BV − best solution among those provided [93]
ARI − average RI for the best solution
Table 6-1. Computational results by GES for ORB1-ORB10 Prob |J| |M| LB UB GES1 Time (sec) orb01 10 10 1059 1059 1059 11 orb02 10 10 888 888 888 1 orb03 10 10 1005 1005 1005 6 orb04 10 10 1005 1005 1005 50 orb05 10 10 887 887 887 4 orb06 10 10 1010 1010 1010 2 orb07 10 10 397 397 397 2 orb08 10 10 899 899 899 3 orb09 10 10 934 934 934 14 orb10 10 10 944 944 944 1
82 Table 6-2. Computational results for LA01-LA40 Prob |J|x|M| Opt GES1 GES2 TSSB la01 10x5 666 666 666 666 la02 10x5 655 655 655 655 la03 10x5 597 597 597 597 la04 10x5 590 590 590 590 la05 10x5 593 593 593 593 la06 15x5 926 926 926 926 la07 15x5 890 890 890 890 la08 15x5 863 863 863 863 la09 15x5 951 951 951 951 la10 15x5 958 958 958 958 la11 20x5 1222 1222 1222 1222 la12 20x5 1039 1039 1039 1039 la13 20x5 1150 1150 1150 1150 la14 20x5 1292 1292 1292 1292 la15 20x5 1207 1207 1207 1207 la16 10x10 945 945 945 945 la17 10x10 784 784 784 784 la18 10x10 848 848 848 848 la19 10x10 842 842 842 842 la20 10x10 902 902 902 902 la21 15x10 1046 1046 1046 1046 la22 15x10 927 927 927 927 la23 15x10 1032 1032 1032 1032 la24 15x10 935 935 935 938 la25 15x10 977 977 977 979 la26 20x10 1218 1218 1218 1218 la27 20x10 1235 1235 1235 1235 la28 20x10 1216 1216 1216 1216 la29 20x10 1152 1153 1152 1168 la30 20x10 1355 1355 1355 1355 la31 30x10 1784 1784 1784 1784 la32 30x10 1850 1850 1850 1850 la33 30x10 1719 1719 1719 1719 la34 30x10 1721 1721 1721 1721 la35 30x10 1888 1888 1888 1888 la36 15x15 1268 1268 1268 1268 la37 15x15 1397 1397 1397 1411 la38 15x15 1196 1196 1196 1201 la39 15x15 1233 1233 1233 1240 la40 15x15 1222 1226 1222 1233
83 Table 6-3. Results for TA1-TA80 Prob |J|x|M| LB UB GES2 GES1 RTSAB RIMP TSSB BV ta01 15x15 1231 1231 1231 1231 1231 1231 1241 1231 ta02 15x15 1244 1244 1244 1244 1244 1244 1244 1244 ta03 15x15 1218 1218 1218 1218 1220 1218 1222 1218 ta04 15x15 1175 1175 1175 1175 1175 1175 1175 1181 ta05 15x15 1224 1224 1224 1224 1229 1224 1229 1233 ta06 15x15 1238 1238 1238 1238 1238 1238 1245 1243 ta07 15x15 1227 1227 1228 1228 1228 1228 1228 1228 ta08 15x15 1217 1217 1217 1217 1217 1217 1220 1217 ta09 15x15 1274 1274 1274 1274 1280 1274 1291 1274 ta10 15x15 1241 1241 1241 1241 1241 1241 1250 1241 ta11 20x15 1323 1357 1357 1357 1367 1365 1371 1392 (1358) ta12 20x15 1351 1367 1367 1375 1377 1377 1379 1367 ta13 20x15 1282 1342 1344 1344 1350 1350 1362 1350 ta14 20x15 1345 1345 1345 1345 1345 1345 1345 1345 ta15 20x15 1304 1339 1339 1339 1347 1345 1360 1353 (1340) ta16 20x15 1302 1360 1360 1360 1361 1365 1370 1369 ta17 20x15 1462 1462 1469 1473 1476 1473 1481 1478 ta18 20x15 1369 1396 1401 1401 1408 1411 1426 1396 ta19 20x15 1297 1332 1332 1332 1338 1342 1351 1341 (1335) ta20 20x15 1318 1348 1348 1352 1355 1357 1366 1359 (1351) ta21 20x20 1539 1644 1647 1647 1648 1652 1659 1659 ta22 20x20 1511 1600 1602 1602 1607 1606 1623 1603 ta23 20x20 1472 1557 1558 1558 1560 1563 1573 1558 ta24 20x20 1602 1647 1653 1653 1654 1656 1659 1659 ta25 20x20 1504 1595 1596 1596 1597 1598 1606 1615 ta26 20x20 1539 1645 1647 1647 1654 1655 1666 1659 ta27 20x20 1616 1680 1685 1685 1691 1687 1697 1689 ta28 20x20 1591 1614 1614 1616 1619 1619 1622 1615 ta29 20x20 1514 1625 1625 1625 1627 1625 1635 1629 ta30 20x20 1472 1584 1584 1584 1585 1585 1614 1604 ta31 30x15 1764 1764 1764 1766 1764 1766 1771 1766 ta32 30x15 1774 1793 1793 1801 1818 1816 1840 1803 (1796) ta33 30x15 1778 1793 1799 1802 1805 1817 1833 1796 ta34 30x15 1828 1829 1832 1832 1832 1835 1846 1832 ta35 30x15 2007 2007 2007 2007 2007 2007 2007 2007 ta36 30x15 1819 1819 1819 1819 1821 1824 1825 1823 ta37 30x15 1771 1778 1779 1779 1789 1793 1813 1784 ta38 30x15 1673 1673 1673 1673 1678 1682 1697 1681
84 Table 6-4. Results for TA1-TA80 (continued) Prob |J|x|M| LB UB GES2 GES1 RTSAB RIMP TSSB BV ta39 30x15 1795 1795 1795 1795 1806 1798 1815 1798 ta40 30x15 1631 1674 1680 1683 1685 1695 1725 1686 ta41 30x20 1859 2014 2022 2029 2035 2047 2045 2026 ta42 30x20 1867 1956 1956 1957 1968 1969 1979 1967 ta43 30x20 1809 1859 1870 1875 1875 1887 1898 1881 ta44 30x20 1927 1984 1991 1992 2000 2006 2036 2004 ta45 30x20 1997 2000 2004 2004 2013 2016 2021 2008 ta46 30x20 1940 2011 2011 2011 2024 2036 2047 2040 (2016) ta47 30x20 1789 1903 1903 1911 1922 1931 1938 1921 ta48 30x20 1912 1952 1962 1975 1966 1978 1996 1982 ta49 30x20 1915 1968 1969 1974 1980 1988 2013 1994 ta50 30x20 1807 1926 1931 1940 1942 1945 1975 1967 ta51 50x15 2760 2760 2760 2760 2760 2760 2760 2760 ta52 50x15 2756 2756 2756 2756 2756 2756 2756 2756 ta53 50x15 2717 2717 2717 2717 2717 2717 2717 2717 ta54 50x15 2839 2839 2839 2839 2839 2839 2839 2839 ta55 50x15 2679 2679 2679 2679 2679 2679 2684 2679 ta56 50x15 2781 2781 2781 2781 2781 2781 2781 2781 ta57 50x15 2943 2943 2943 2943 2943 2943 2943 2943 ta58 50x15 2885 2885 2885 2885 2885 2885 2885 2885 ta59 50x15 2655 2655 2655 2655 2655 2655 2655 2655 ta60 50x15 2723 2723 2723 2723 2723 2723 2723 2723 ta61 50x20 2868 2868 2868 2868 2868 2868 2868 2868 ta62 50x20 2869 2869 2872 2872 2877 2875 2942 2900 ta63 50x20 2755 2755 2755 2755 2755 2755 2755 2755 ta64 50x20 2702 2702 2702 2702 2702 2702 2702 2702 ta65 50x20 2725 2725 2725 2725 2734 2725 2725 2725 ta66 50x20 2845 2845 2845 2845 2881 2845 2845 2845 ta67 50x20 2825 2825 2825 2825 2825 2825 2865 2826 ta68 50x20 2784 2784 2784 2784 2784 2784 2784 2784 ta69 50x20 3071 3071 3071 3071 3071 3071 3071 3071 ta70 50x20 2995 2995 2995 2995 2995 2995 2995 2995 ta71 100x20 5464 5464 5464 5464 5464 5464 5464 5464 ta72 100x20 5181 5181 5181 5181 5181 5181 5181 5181 ta73 100x20 5568 5568 5568 5568 5568 5568 5568 5568 ta74 100x20 5339 5339 5339 5339 5339 5339 5339 5339 ta75 100x20 5392 5392 5392 5392 5392 5392 5392 5392 ta76 100x20 5342 5342 5342 5342 5342 5342 5342 5342 ta77 100x20 5436 5436 5436 5436 5436 5436 5436 5436 ta78 100x20 5394 5394 5394 5394 5394 5394 5394 5394 ta79 100x20 5358 5358 5358 5358 5358 5358 5358 5358 ta80 100x20 5183 5183 5183 5183 5183 5183 5183 5183
85 Table 6-5. Comparison with TSSB Problem |J|x|M| GES1 TSSB ARI Tav ARI Tav la01-la05 10x5 0.00 - 0.00 9,8 la06-la10 15x5 0.00 - 0.00 - la11-la15 20x5 0.00 - 0.00 - la16-la20 10x10 0.00 - 0.00 61,5 la21-la25 15x10 0.00 17.2 0.11 115 la26-la30 20x10 0.02 15.4 0.28 105 la31-la35 30x10 0.00 - 0.00 - la36-la40 15x15 0.07 109.2 0.58 141
Table 6-6. Comparison with other algorithms Prob GES1 RTSAB RIMP TSSB BV ARI Tav ARI Tav ARI Tav ARI Tav ARI Tav ta01-ta10 0.01 84.5 0.11 566.5 0.01 190.0 0.45 2175 0.17 1498 ta11-ta20 0.21 1154.6 0.55 1612.5 0.60 2044.3 1.19 2526 0.75 4559 ta21-ta30 0.13 1668.8 0.31 2241.7 0.34 1561.6 1.01 34910 0.61 6850 ta31-ta40 0.19 1460 0.46 1171.4 0.62 2052.2 1.41 14133 0.30 8491 ta41-ta50 0.49 2111.4 0.78 3229.9 1.18 2205.6 1.92 11512 1.11 16018 ta51-ta60 0.00 6.4 0.00 238.8 0.00 7.5 0.02 421 0.00 196 ta61-ta70 0.01 185.2 0.19 1155.2 0.02 188.6 0.40 6342 0.11 2689 ta71-ta80 0.00 38.1 0.00 452.5 0.00 38.2 0.00 231 0.00 851
86 Table 6-7. Results for DMU1-DMU80 Prob n m LB UB GES2 DMU1 20 15 2501 2563 2566 DMU2 20 15 2651 2706 2706 DMU3 20 15 2731 2731 2731 DMU4 20 15 2601 2669 2669 DMU5 20 15 2749 2749 2749 DMU6 20 20 2834 3250(3252) 3250 DMU7 20 20 2677 3053(3063) 3053 DMU8 20 20 2901 3197(3199) 3197 DMU9 20 20 2739 3092 3092 DMU10 20 20 2716 2984(2985) 2984 DMU11 30 15 3395 3453(3457) 3453 DMU12 30 15 3481 3518(3519) 3518 DMU13 30 15 3681 3697(3698) 3697 DMU14 30 15 3394 3394 3394 DMU15 30 15 3332 3343 3343 DMU16 30 20 3726 3781(3787) 3781 DMU17 30 20 3697 3848(3854) 3848 DMU18 30 20 3844 3849(3852) 3849 DMU19 30 20 3650 3807(3814) 3807 DMU20 30 20 3604 3739(3740) 3739 DMU21 40 15 4380 4380 4380 DMU22 40 15 4725 4725 4725 DMU23 40 15 4668 4668 4668 DMU24 40 15 4648 4648 4648 DMU25 40 15 4164 4164 4164 DMU26 40 20 4647 4667(4670) 4667 DMU27 40 20 4848 4848 4848 DMU28 40 20 4692 4692 4692 DMU29 40 20 4691 4691 4691 DMU30 40 20 4732 4732 4732 DMU31 50 15 5640 5640 5640 DMU32 50 15 5927 5927 5927 DMU33 50 15 5728 5728 5728 DMU34 50 15 5385 5385 5385 DMU35 50 15 5635 5635 5635 DMU36 50 20 5621 5621 5621 DMU37 50 20 5851 5851 5851 DMU38 50 20 5713 5713 5713 DMU39 50 20 5747 5747 5747 DMU40 50 20 5577 5577 5577 DMU41 20 15 2839 3267(3275) 3267 DMU42 20 15 3066 3401(3416) 3401 DMU43 20 15 3121 3443(3455) 3443
87 Table 6-8. Results for DMU1-DMU80 (continued) Problem n m LB UB GES2 DMU44 20 15 3112 3489(3501) 3489 DMU45 20 15 2930 3273 3273 DMU46 20 20 3425 4099(4101) 4099 DMU47 20 20 3353 3972(3973) 3972 DMU48 20 20 3317 3810(3834) 3810 DMU49 20 20 3369 3754(3765) 3754 DMU50 20 20 3379 3768(3772) 3768 DMU51 30 15 3839 4247(4252) 4247 DMU52 30 15 4012 4380(4383) 4380 DMU53 30 15 4108 4450(4454) 4450 DMU54 30 15 4165 4424(4425) 4424 DMU55 30 15 4099 4331(4332) 4331 DMU56 30 20 4366 5049 5051 DMU57 30 20 4182 4779(4781) 4779 DMU58 30 20 4214 4829(4834) 4829 DMU59 30 20 4199 4694(4696) 4694 DMU60 30 20 4259 4888(4890) 4888 DMU61 40 15 4886 5293(5294) 5293 DMU62 40 15 5004 5342 5354 DMU63 40 15 5049 5437 5439 DMU64 40 15 5130 5367 5388 DMU65 40 15 5072 5269(5271) 5269 DMU66 40 20 5357 5902(5910) 5902 DMU67 40 20 5484 6012(6016) 6012 DMU68 40 20 5423 5934(5936) 5934 DMU69 40 20 5419 5891 6002 DMU70 40 20 5492 6072(6076) 6072 DMU71 50 15 6050 6302 6333 DMU72 50 15 6223 6571 6589 DMU73 50 15 5935 6283 6291 DMU74 50 15 6015 6376 6376 DMU75 50 15 6010 6380(6384) 6380 DMU76 50 20 6329 6974(6975) 6974 DMU77 50 20 6399 6930 7006 DMU78 50 20 6508 6962 6988 DMU79 50 20 6593 7158(7164) 7158 DMU80 50 20 6435 6824 6843
88 REFERENCES
[1] J. E. Beasley, OR-library webpage, http://people.brunel.ac.uk/~mastjjb/jeb/ info.html (last accessed on june 2008).
[2] G. Palubeckis, Unconstrained binary quadratic optimization, http://www.soften. ktu.lt/~gintaras/binqopt.html (last accessed on october 2008). [3] N. Sloane, Challenge problems: Independent sets in graphs, http://www.research. att.com/~njas/doc/graphs.html (last accessed on october 2008). [4] E. Nowicki, C. Smutnicki, Some New Ideas in TS for Job Shop Scheduling, Vol. 30 ,Part II of Operations Research/Computer Science Interfaces Series, Springer, 2005, pp. 165–190. [5] M. Luby, A. Sinclair, D. Zuckerman, Optimal speedup of las vegas algorithms, Inf. Process. Lett. 47 (4) (1993) 173–180. [6] M. Luby, M. Luby, W. Ertel, Optimal parallelization of las vegas algorithms, in: STACS ’94: Proceedings of the 11th Annual Symposium on Theoretical Aspects of Computer Science, Springer-Verlag, London, UK, 1994, pp. 463–474.
[7] C. P. Gomes, B. Selman, H. Kautz, Boosting combinatorial search through randomization, in: Proceedings of the 15th National Conference on Artificial Intelligence, AAAI Press, Madison, Wisconsin, 1998, pp. 431–437. [8] I. V. Sergienko, V. P. Shilo, V. A. Roshchin, Restart technology for solving discrete optimization problems, Cybernetics and Sys. Anal. 36 (5) (2000) 659–666.
[9] H. Chen, C. P. Gomes, B. Selman, Formal models of heavy-tailed behavior in combinatorial search, in: CP ’01: Proceedings of the 7th International Conference on Principles and Practice of Constraint Programming, Springer-Verlag, London, UK, 2001, pp. 408–421. [10] A. P. A. van Moorsel, K. Wolter, Analysis and algorithms for restart, in: QEST ’04: Proceedings of the The Quantitative Evaluation of Systems, First International Conference, IEEE Computer Society, Washington, DC, USA, 2004, pp. 195–204. [11] I. V. Sergienko, V. P. Shilo, V. A. Roshchin, Optimization parallelizing for discrete programming problems, Cybernetics and Sys. Anal. 40 (2) (2004) 184–189.
[12] G. Palubeckis, Multistart tabu search strategies for the unconstrained binary quadratic optimization problem, Journal Annals of Operations Research 131 (1-4) (2004) 259–282. [13] P. Pardalos, O. Shylo, An algorithm for the job shop scheduling problem based on global equilibrium search techniques, Computational Management Science 3 (4) (2006) 331–348.
89 [14] M. M. D’apuzzo, A. Migdalas, P. M. Pardalos, G. Toraldo, Parallel computing in global optimization, in: E. Kontoghiorghes (Ed.), Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, 2006, pp. 225–258. [15] F. Glover, M. Laguna, Tabu search, IEEE Trans. Inform. Theory 36 (6) (1990) 1487–1494.
[16] S. Kirkpatricks, C. D. Gelatt, M. P. Vecchi, Optimization by simulated annealing, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1987, pp. 606–615. [17] V. Shylo, A global equilibrium search method, Kybernetika i Systemnuiy Analys (in russian) 1 (1999) 74–80.
[18] F. Barahona, On the computational complexity of spin glass models, Journal of Physics A: Mathematical and General 15 (1982) 3241–3253. [19] F. Barahona, M. Gr¨otschel, M. J¨uger,G. Reinelt, An application of combinatorial optimization to statistical physics and circuit layout design, Operations Research 36 (1988) 493–513.
[20] L. Iasemidis, P. Pardalos, J. Sackellares, D. Shiau, Quadratic binary programming and dynamical system approach to determine the predictability of epileptic seizures, Journal of Combinatorial Optimization 5 (2001) 9–26. [21] M. J¨unger,A. Martin, G. Reinelt, R. Weismantel, Quadratic 0/1 optimization and a decomposition approach for the placement of electronic circuits, Mathematical Programming 63 (1994) 257–279. [22] J.Krarup, P. Pruzan, Computer aided layout design, Mathematical Programming Study 9 (1978) 75–94.
[23] B. Alidaee, G. Kochenberger, A. Ahmadian, 0-1 quadratic programming approach for the optimal solution of two scheduling problems, International Journal of Systems Science 25 (1994) 401–408. [24] G.Gallo, P. Hammer, B.Simeone, Quadratic knapsack problems, Mathematical Programming 12 (1980) 132–149. [25] J. Klepeis, C. Floudas, D. Morikis, C. Tsokos, J. Lambris, Design of peptide analogues with improved activity using a novel de novo protein design approach, Industrial & Engineering Chemistry Research 43 (14) (2004) 3817–3826. [26] P. M. Pardalos, G. P. Rodgers, Computational aspects of a branch and bound algorithm for quadratic zero-one programming, Computing 45 (2) (1990) 131–144.
[27] P. M. Pardalos, G. P. Rodgers, A branch and bound algorithm for the maximum clique problem, Computers OR (1992) 363–375.
90 [28] P. Pardalos, J. Xue, The maximum clique problem, Journal of Global Optimization 4 (1994) 301–328.
[29] P. Hammer, P. Hansen, P. Pardalos, J. D. Rader, Maximizing the product of two linear functions in 0–1 variables, Optimization 51 (2002) 511–537. [30] K. Allemand, K. Fukuda, T. Liebling, E. Steiner, A polynomial case of unconstrained zero-one quadratic optimization, Mathematical Programming (Ser. A, 91) (2001) 49–52. [31] F. Barahona, A solvable case of quadratic 0-1 programming, Discrete Applied Mathematics 13 (1986) 23–26.
[32] P. Pardalos, S. Jha, Graph separation techniques for quadratic zero-one programming, Computers and Mathematics with Applications 21 (1991) 107–113. [33] J. Picard, H. Ratliff, Minimum cuts and related problems, Networks 5 (1975) 357–370.
[34] P. Pardalos, S. Jha, Complexity of uniqueness and local search in quadratic 0–1 programming, Operations Research Letters 11 (1992) 119–123. [35] M. R. Garey, D. S. Johnson, Computers and Intractability; A Guide to the Theory of NP-Completeness, W. H. Freeman & Co., New York, NY, USA, 1990.
[36] J. Hastad, Clique is hard to approximate within n1-, in: FOCS ’96: Proceedings of the 37th Annual Symposium on Foundations of Computer Science, IEEE Computer Society, Washington, DC, USA, 1996, p. 627. [37] S. Khot, Improved inapproximability results for maxclique, chromatic number and approximate graph coloring, in: FOCS ’01: Proceedings of the 42nd IEEE symposium on Foundations of Computer Science, IEEE Computer Society, Washington, DC, USA, 2001, p. 600. [38] Y. Nesterov, Quality of semidefinite relaxation for nonconvex, CORE Discussion Paper #9719, Belgium (March 1997). [39] M. X. Goemans, D. P. Williamson, Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming, J. ACM 42 (6) (1995) 1115–1145.
[40] Y. Nesterov, Semidefinite relaxation and nonconvex quadratic optimization, Optimization Methods and Software 9 (1998) 141–160. [41] Y. Ye, Approximating global quadratic optimization with convex quadratic constraints, J. of Global Optimization 15 (1) (1999) 1–17.
[42] A. Billionnet, A. Sutter, Minimization of a quadratic pseudo-boolean function, European Journal of Operational Research 78 (1994) 106–115.
91 [43] C. Helmberg, F. Rendl, Solving quadratic (0, 1)-problems by semidefinite programs and cutting planes, Mathematical Programming 82 (1998) 291–315.
[44] J. Beasley, Heuristic algorithms for the unconstrained binary quadratic programming problem, Working paper, The Management School, Imperial College, London, England.
[45] F. Glover, G. Kochenberger, B. Alidaee, Adaptive memory tabu search for binary quadratic programs, Management Science 44 (1998) 336–345. [46] A. Lodi, K. Allemand, T. Liebling, An evolutionary heuristic for quadratic 0-1 programming, European Journal of Operational Research 119 (1999) 662–670.
[47] P. M. Pardalos, Construction of test problems in quadratic bivalent programming, ACM Trans. Math. Softw. 17 (1) (1991) 74–87. [48] Colorado, Hearin center for enterprise science, benchmarks for unconstrained binary quadratic problems, http://hces.bus.olemiss.edu/tools.html (last accessed on october 2008).
[49] I. Sergienko, V. Shylo, Problems of discrete optimization: Challenges and main approaches to solve them, Cybernetics and Systems Analysis 42 (2006) 465–482. [50] C. A. Floudas, P. M. Pardalos (Eds.), Encyclopedia of Optimization, Second Edition, Springer, 2009. [51] C. Koukouvinos, Sequences with zero autocorrelation, in: C. J. Colbourn, J. H. Dinitz (Eds.), The CRC Handbook of Combinatorial Designs, Part VI, CRC Press, 1996, Ch. 42. [52] L. B¨omer,M. Antweiler, Periodic complementary binary sequences, IEEE Trans. Inform. Theory 36 (6) (1990) 1487–1494.
[53] G. Weathers, E. M. Holiday, Group-complementary array coding for radar clutter rejection, IEEE Transaction on Aerospace and Electronic Systems 19. [54] E. Fenimore, T. Cannon, Coded aperture imaging with uniformly redundant arrays, Appl. Optics 17 (1978) 337–347.
[55] S. Golomb, H. Taylor, Two-dimensional synchronization patterns for minimum ambiguity, IEEE Trans. Inform. Theory 28 (1982) 600–604. [56] J. Hersheya, R. Yarlagadda, Two-dimensional synchronisation, Electron. Lett. 19 (1983) 801–803.
[57] D. R. Stinson, Combinatorial Designs: Constructions and Analysis, SpringerVerlag, 2003. [58] S. W. Golomb, G. Gong., Signal design for good correlation. For wireless communication, cryptography, and radar., University Press, Cambridge, 2005.
92 [59] I. S. Kotsireas, C. Koukouvinos, Hadamard ideals and hadamard matrices from two circulant submatrices, J. Combin. Math. Combin. Comput. 61 (2007) 97–110.
[60] P. M. Pardalos, O. A. Prokopyev, O. V. Shylo, V. P. Shylo, Global equilibrium search applied to the unconstrained binary quadratic optimization problem, Optimization Methods and Software 23 (2008) 129–140.
[61] C. J. Colbourn, J. H. Dinitz, Handbook of Combinatorial Designs, Second Edition (Discrete Mathematics and Its Applications), Chapman & Hall/CRC, 2006. [62] R. Battiti, M. Protasi, Reactive search, a history-sensitive heuristic for max-sat, J. Exp. Algorithmics 2 (1997) 2.
[63] W. Spears, Simulated annealing for hard satisfiability problems: Second dimacs implementation challenge, in: D. Johnson, M. Trick (Eds.), DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26, AMS, 1996, pp. 533–555.
[64] P. Festa, P. M. Pardalos, L. S. Pitsoulis, M. G. C. Resende, Grasp with path relinking for the weighted maxsat problem, J. Exp. Algorithmics 11 (2006) 2.4.
[65] M. G. C. Resende, Thomas, A. Feo, A grasp for satisfiability, in: Cliques, Coloring, and Satisfiability: The Second DIMACS Implementation Challenge, volume 26 of DIMACS Series on Discrete Mathematics and Theoretical Computer Science, American Mathematical Society, 1996, pp. 499–520.
[66] M. Yagiura, T. Ibaraki, Efficient 2 and 3-flip neighborhood search algorithms for the max sat: Experimental evaluation, Journal of Heuristics 7 (5) (2001) 423–442.
[67] P. Mills, E. Tsang, Guided local search for solving sat and weighted max-sat problems, J. Autom. Reason. 24 (1-2) (2000) 205–223. [68] P. Hansen, B. Jaumard, Algorithms for the maximum satisfiability problem, Computing 44 (4) (1990) 279–303.
[69] M. Resende, Algorithm source code distribution, http://research.att.com/~mgcr/ src (last accessed on october 2008).
[70] M. Resende, Test problem distribution, http://research.att.com/~mgcr/data (last accessed on october 2008).
[71] M. Yagiura, Benchmark instances and program codes for max sat, http://www.al. cm.is.nagoya-u.ac.jp/~yagiura/msat.html (last accessed on october 2008). [72] P. Mills, Constraint programming and optimization laboratory, university of essex, gls solver, algorithm: Extended gls, http://cswww.essex.ac.uk/CSP/glsdemo.html (last accessed on october 2008).
93 [73] M. Yagiura, T. Ibaraki, Analyses on the 2 and 3-flip neighborhoods for the max sat, Journal of Combinatorial Optimization 3 (1999) 95–114.
[74] P. Mills, E. Tsang, Guided local search for solving sat and weighted max-sat problems, J. Autom. Reason. 24 (1-2) (2000) 205–223. [75] E. L. Lawler, J.-K. Lenstra, K. A. Rinnooy, Recent Developments in Deterministic Sequencing and Scheduling, Reidel, Dordrecht, The Netherlands, 1982, pp. 35–73.
[76] E. Nowicki, C. Smutnicki, A fast tabu search algorithm for the job-shop problem, Management Science 42 (6) (1996) 797–813.
[77] E. Taillard, Benchmarks for basic scheduling problems, European Journal of Operational Research 64 (1993) 278–285. [78] S. Binato, W. Hery, D. Loewenstern, M. Resende, A GRASP for job shop scheduling, Essays and surveys on metaheuristics, Kluwer Academic Publishers, 2001, pp. 59–79. [79] P. Van Laarhoven, E. Aarts, J. Lenstra, Job shop scheduling by simulated annealing, Operations Research 40 (1992) 113–125.
[80] A. Jain, S. Meeran, Deterministic job shop scheduling: Past, present and future, European Journal of Operational Research 113 (1999) 390–434.
[81] C.-Y. Lee, M. Pinedo, Optimization and heuristic scheduling, Handbook of Applied Optimization, P.M. Pardalos and M.G.C. Resende (Eds), Oxford University Press, 2002, pp. 569–584. [82] B. Roy, B. Sussman, Les problemes d’ordonnancement avec contraintes disjonctives, Note DS9 bis, SEMA, Paris (in French) (1964). [83] J. Watson, A. Howe, L. Whitley, An analysis of iterated local search for job-shop scheduling, Fifth Metaheuristics International Conference (MIC 2003).
[84] J. Grabowski, E. Nowicki, C. Smutnicki, Block algorithm for scheduling of operations in job-shop system, (polish), Przeglad Statystyczny 35 (1988) 67–80. [85] J. Blazewicz, W. Domschke, E. Pesch, The job-shop scheduling problem: Conventional and new solution techniques, European Journal of Operational Research 93(1) (1996) 1–33. [86] H. Lourenco, O. Martin, T. Sttzle, Iterated local search, Handbook of Metaheuristics, In F. Glover and G. Kochenberger (Eds), Kluwer Academic Publishers, 2003.
[87] E. Nowicki, C. Smutnicki, Some new tools to solve the job shop problem, Technical Report 60/2002, Institute of Engineering Cybernetics, Technical University of Wroclaw, Wroclaw, Poland (2002). [88] D. Applegate, W. Cook, A computational study of the job-shop scheduling instance, ORSA Journal on Computing 3 (1991) 149–156.
94 [89] S. Lawrence, Resource constrained project scheduling: an experimental investigation of heuristic scheduling techniques (supplement), Graduate School of Industrial Administration, Carnegie-Mellon University, Pittsburgh, Pennsylvania (1984). [90] E. Demirkkol, S. Mehta, R. Uzsloy, Benchmarks for job shop scheduling problems, European Journal of Operational Research 109 (1997) 137–141.
[91] F. Pezzella, E. Merelli, A tabu search method guided by shifting bottleneck for the job shop scheduling problem, European Journal of Operational Research 120 (2000) 297–310. [92] E. Balas, A. Vazacopoulos, Guided local search with shifting bottleneck for job shop scheduling, MSSRR 609, GSIA, Carnegie Mellon University, Pittsburgh.
[93] E. Balas, A. Vazacopoulos, Guided local search with shifting bottleneck for job shop scheduling, Management Science 44 (1995) 262–275.
95 BIOGRAPHICAL SKETCH Oleg Shylo was born on September 03, 1981, in Kiev, Ukraine. He received his bachelors and specialists degrees in applied mathematics from the National Technical
University of Ukraine (“KPI”) in Kiev, Ukraine, in 2003 and 2004, respectively. In January 2005, he began his doctoral studies in the Industrial and Systems Engineering Department at the University of Florida. He earned his Ph.D. in August 2009.
96