From: AAAI-00 Proceedings. Copyright © 2000, AAAI (www.aaai.org). All rights reserved.
Depth-First Branch-and-Bound versus Local Search: A Case Study
Weixiong Zhang Information Sciences Institute and Computer Science Department University of Southern California 4676 Admiralty Way, Marina del Rey, CA 90292 Email: [email protected]
Abstract 1983). DFBnB explores a state space in a depth-first order, and finds many suboptimal solutions with in- Depth-first branch-and-bound (DFBnB) is a complete creasingly better qualities. These solutions are approx- algorithm that is typically used to find optimal solu- imations to the optimal solution if DFBnB is termi- tions of difficult combinatorial optimization problems. It can also be adapted to an approximation algorithm nated prematurely. Furthermore, DFBnB is also an and run as an anytime algorithm, which are the sub- anytime algorithm. An anytime algorithm (Dean & jects of this paper. We compare DFBnB against the Boddy 1988) can provide a solution at any time dur- Kanellakis-Papadimitriou local search algorithm, the ing its execution, and is able to improve the qual- best known approximation algorithm, on the asymmet- ity of the current best solution with more computa- ric Traveling Salesman Problem (ATSP), an important tion. DFBnB finds better suboptimal solutions with NP-hard problem. Our experimental results show that more computation and eventually reaches the opti- DFBnB significantly outperforms the local search on mal solution. In contrast to the importance of any- large ATSP and various ATSP structures, finding bet- time problem solving and the effort of developing ter solutions faster than the local search; and the qual- new anytime algorithms (Hoebel & Zilberstein 1997; ity of approximate solutions from a prematurely ter- minated DFBnB, called truncated DFBnB, is several Horvitz & Zilberstein 1996), DFBnB has not been stud- times better than that from the local search. ied as an approximation or anytime algorithm so far. We study DFBnB as an approximation and any- time algorithm in this paper. We compare it Introduction and Overview against the Kanellakis-Papadimitriou local search algo- Depth-first branch-and-bound (DFBnB) (Balas & Toth rithm on the asymmetric Traveling Salesman Problem 1985; Papadimitriou & Steiglitz 1982) and local (ATSP) (Kanellakis & Papadimitriou 1980). This local search (Lin & Kernighan 1973; Kanellakis & Papadim- search algorithm is an adaptation and extension of the itriou 1980; Johnson 1990; Johnson & McGeoch 1997) well-known Lin-Kernighan local search algorithm (Lin are the two most applied search methods for solving & Kernighan 1973), and the only local search algo- combinatorial optimization problems, such as planning rithm for the ATSP which we found in the literature. and scheduling. DFBnB is usually the algorithm for We choose the ATSP due to the following two reasons. finding optimal solutions of large problems, due to the First, the ATSP is an important problem in the NP- virtue of its linear-space requirement. Local search, hard class (Papadimitriou & Steiglitz 1982) and has on the other hand, is a method for high-quality ap- many practical applications. Many difficult combina- proximate solutions, and has been shown to be effec- torial optimization problems, such as vehicle routing, tive and efficient on many combinatorial optimization workshop scheduling and computer wiring, can be for- problems, such as the symmetric Traveling Salesman mulated and solved as the ATSP (Lawler et al. 1985). Problem (TSP) (Lin & Kernighan 1973; Johnson 1990; Second, despite its importance, little work has been Johnson & McGeoch 1997). done on the ATSP, which is disproportional to that on Besides that DFBnB is an efficient complete algo- the symmetric TSP (see (Johnson & McGeoch 1997) rithm for optimal solutions, it can be used as an ap- for an excellent survey and references cited). It will be proximation algorithm1, as suggested in (Ibaraki et al. very helpful if information regarding which algorithm should be used for a particular type of ATSP is avail- Copyright c 2000, American Association for Artificial In- able to guide algorithm selection in practice. telligence (www.aaai.org). All rights reserved. 1We use the term approximation algorithm loosely to re- The paper is structured as follows. We discuss the fer to an algorithm that is able to find a suboptimal solution. ATSP, DFBnB, truncated DFBnB and the Kanellakis- Such an algorithm does not provide a quality guarantee un- Papadimitriou local search algorithm in Section 2. In der the definition of -approximation algorithm (Papadim- Section 3, we describe various ATSP structures used itriou 1994). in our experiments. In Section 4, we investigate initial tour-construction heuristics. In Section 5, we compare selects a recently generated node n to examine next. If DFBnB with the local search algorithm. We discuss the AP solution of n is a complete tour, meaning that n the features of DFBnB and the weakness of the local is a leaf node in the search tree, and its cost is less than search algorithm on the ATSP in Section 6. Finally, we the current upper bound α, α is revised to the cost of conclude and discuss future work in Section 7. n.Ifn’s AP solution is not a complete tour and its cost is greater than or equal to α, n is pruned, because node The Problem and the algorithms costs are non-decreasing along a path from the root so that no descendent of n will have a cost smaller than { ··· } Given n cities, 1, 2, ,n , and a matrix (ci,j) that de- n’s cost. Otherwise, n is expanded, generating all its fines the costs of pairs of cities, the Traveling Salesman child nodes. To find an optimal goal node quickly, the Problem (TSP) is to find a minimum-cost tour that vis- children n should be searched in an increasing order of its each city once and returns to the starting city. When their costs. This is called node ordering. We use node the cost matrix is asymmetric, i.e., the cost from city i ordering in this study. to city j is not necessarily equal to the cost from j to DFBnB can be used as an anytime algorithm. Dur- i, the problem is the asymmetric TSP (ATSP). ing the depth-first exploration of a state space, DF- BnB may encounter many leaf nodes and improve the DFBnB and truncated DFBnB best solution at hand continuously. Furthermore, DF- Branch-and-bound (BnB) (Balas & Toth 1985; Pa- BnB can be stopped at any time during its execution. padimitriou & Steiglitz 1982) solves an ATSP as a This is an extension to the M-Cut strategy suggested state-space search and uses the assignment problem as in (Ibaraki et al. 1983), which terminates BnB with a a lower-bound cost function. The assignment problem fixed amount of computation. We call DFBnB with an (AP) (Martello & Toth 1987; Papadimitriou & Stei- early termination truncated DFBnB. Among all possi- glitz 1982) is to assign to each city i another city j, ble stopping points, of particular interest is where the with ci,j as the cost of the assignment, such that the first leaf node is reached. When no initial tour is used, total cost of all assignments is minimized. The AP is this simple, special truncated DFBnB is a greedy search a relaxation of the ATSP, since the assignments need in a search space, which always chooses to explore next not form a complete tour. Therefore, the AP cost is the minimum-cost child node of the current state until a lower bound on the ATSP tour cost. If the AP so- it reaches a leaf node. When a high-quality initial tour lution happens to be a complete tour, it is also the is employed, DFBnB may not necessarily encounter a solution to the ATSP. The BnB search takes the origi- leaf node before the total allocated computation is ex- nal ATSP as the root of the state space and repeats the hausted. Without confusion, we call DFBnB that ter- following two steps. First, solve the AP of the current minates when it reaches a leaf node or consumes all problem. If the AP solution is not a complete tour, allowed computation truncated DFBnB in the rest of decompose it into subproblems by subtour elimination. this paper. Specifically, select a subtour from the AP solution, and generate subproblems by excluding some edges from Kanellakis-Papadimitriou local search the assignments, so as to eliminate the subtour. There Local search is based on a fundamental concept called are many subtour-elimination heuristics (Balas & Toth neighborhood structure. If two TSP tours differ by λ 1985), and we use the Carpaneto-Toth scheme in our ex- edges, and one can be changed to the other by swap- periments (Carpaneto & Toth 1980), which generates ping the different edges, one tour is a λ-change neighbor no duplicate subproblem. Next, select as the current of the other. A neighborhood structure is established problem a new subproblem that has been generated but by defining the legitimate changes. Within a neighbor- not yet expanded. This process continues until there is hood, a tour is a local optimum if it is the best among no unexpanded problem, or until all unexpanded prob- its neighbors. Given a neighborhood structure, a local lems have costs greater than or equal to the cost of the search moves from a tour to a neighboring tour that has best complete tour found so far. Note that a subprob- a smaller cost until a local optimum is reached. lem is more constrained than its parent problem, there- The Kanellakis-Papadimitriou local search algorithm fore the AP cost to the subproblem must be as much as for the ATSP (Kanellakis & Papadimitriou 1980) fol- that to the parent. This means that the AP cost func- lows the Lin-Kernighan local search algorithm for the tion is monotonically nondecreasing with search depth. symmetric TSP (Lin & Kernighan 1973), and uses pri- The AP to the root node can be computed in O(n3) mary changes, which change an odd number of edges time; and the APs to non-root nodes can be computed in a tour. Figure 1 shows a primary 3-change. Primary in O(n2) time (Martello & Toth 1987). changes are found by the following sequential process. Depth-first branch-and-bound (DFBnB) is a special To construct a primary change of a tour π, we first re- BnB that explores nodes or subproblems in a depth- move an arc x1 of π, resulting in a directed path (Figure first order. DFBnB uses an upper bound α on the opti- 2(a)). We then add a new arc y1, which determines an mal cost, whose initial value can be infinity or the cost arc x2 that needs to be deleted and creates a cycle C1 of a tour generated by a polynomial-time heuristic (cf. (Figure 2(b)). We can immediately break the cycle C1 Section 4). Starting at the root node, DFBnB always by adding a new arc y2 as shown in Figure 2(c). We x 3 x3 x4 C1 y2 y1 y y 3 2 x1 x1 x2 x1y1 x2 x1 x3
y x 1 x 1 2 2 (a) 2 (b) 2 (c) x2
Fig. 1: Primary 3-chage. Fig. 2: Sequential search of primary changes. Fig. 3: Quad change.
end up with a directed path, as we started with in Fig- random matrices as in (1), and then used a closure pro- ure 2(a). In general, we call two subsequent pairs of cedure to enforce the triangle inequality. (3) Matrices deleted and added arcs,
ci,j ≤ i × j (sec.) TDFBnB 0.023 0.127 0.365 0.740 1.267 2.141 3.148 4.606 5.981 7.901 win LS 10.60 4.20 2.20 1.20 1.75 1.40 1.00 0.00 2.00 1.00 (%) TDFBnB 83.00 94.20 95.80 97.40 97.00 98.20 99.00 100.00 98.00 99.00 error LS 0.035 0.013 0.008 0.006 0.004 0.003 0.003 0.003 0.002 0.002 no-wait (%) TDFBnB 0.030 0.011 0.007 0.005 0.003 0.003 0.002 0.002 0.002 0.001 flowshop time LS 0.063 0.468 1.797 4.812 9.881 19.631 33.353 49.026 70.553 98.802 scheduling (sec.) TDFBnB 0.029 0.177 0.543 1.213 2.222 3.891 5.748 8.552 2.249 5.781 win LS 11.80 9.60 7.60 7.20 8.20 7.40 6.00 7.00 6.00 7.00 (%) TDFBnB 34.80 40.40 45.80 50.20 47.00 52.60 48.00 51.00 49.00 55.00
Table 1: Truncated DFBnB vs. local search. forms the local search on the ATSP with random cost instances, and 99.916% for 1,000-city instances. A matrices. Similar results have also been observed on good lower-bound cost function can usually give rise other problem sizes and structures. DFBnB typically to a strong branch-and-bound algorithm by providing finds better tours earlier than the local search. This a good node ordering. This has also been observed is partially because the AP can be computed in O(n2) on number partitioning using branch-and-bound with time on a non-root node of the search tree (Martello Karmarkar-Karp heuristics (Korf 1998). & Toth 1987), rather than in O(n3) time on the root node. Based on our experiments, the AP on a non-root The second factor that leads to the superb perfor- node can be computed in roughly one tenth, one twen- mance of DFBnB is that the state space under BnB tieth, and one seventeenth of the CPU time required subtour elimination with the decomposition rules of for the AP of the initial ATSP with random cost ma- (Carpaneto & Toth 1980) is a shallow but bushy tree trices, matrices with triangle inequality, and matrices without duplicate nodes. Our experiments revealed with ci,j ≤ i × j, respectively. Thanks to the superior that with random cost matrices, the depth of an ATSP performance of truncated DFBnB, DFBnB can obtain a state-space tree is typically less than n/20, and its av- high-quality tour very early in its execution, which can erage branching factor of the nodes is around n/5. For further help to restrict the search to the areas where a random 500-city ATSP for instance, the tree depth better solutions can be found. is less than 25 and the branching factor is around 100. Since leaf nodes are shallow, they can be reached easily, Discussions so as to give rise to solutions quickly. The superior performance of DFBnB and truncated The poor performance of the Kanellakis- DFBnB on the ATSP is primarily due to two factors. Papadimitriou algorithm for the ATSP indicates The first is the assignment problem (AP) lower-bound that its local-search neighborhood structure may be cost function. As observed in previous research (Balas restricted. Comparing to the symmetric TSP, there is & Toth 1985) and in our own experiments, this cost no primary change with an even number of edges in the function gives a superb estimation on the actual ATSP ATSP. Thus, finding a better primary change of large tour cost, approaching the ATSP cost from below as steps is difficult. Furthermore, the local search searches problem size increasing. In our experiments on the ran- for increasingly better primary changes, which may dom ATSP with the elements of cost matrices indepen- substantially curtail the effectiveness of the algorithm. dently and uniformly chosen from {0, 1, 2, ···, 224 − 1}, In addition, to find favorable pairs of steps, edges need the cost of the AP is 99.090% of the ATSP cost on to be sorted, further preventing the local search to average for 100-city instances, 99.816% for 500-city quickly improve the initial tour, as shown in Figure 4. TDF- TDF- TDF- TDF- TDF- TDF- problem AP patch LS LS BnB1 BnB1 BnB1 LS BnB2 BnB2 BnB2 LS (millisec.) (millisec.) win win (millisec.) win win xbr17 0 40 39 0 39 1 39 1 ft53 5931 7932 7364 0 7917 8 x 7372 9 x ft70 37978 39401 38947 2 38855 2 x 38855 2 x ftv170 2631 2793 2783 5 2776 14 x 2767 153 x ftv33 1185 1455 1390 0 1443 0 x 1329 3 x ftv35 1381 1493 1493 1 1490 2 x 1487 3 x ftv38 1438 1550 1550 0 1547 2 x 1546 5 x ftv44 1521 1737 1699 0 1663 1 x 1613 4 x ftv47 1652 1787 1784 0 1787 8 x 1787 8 x ftv55 1435 1680 1649 0 1675 1 x 1624 8 x ftv64 1721 1970 1851 1 1853 0 x 1839 20 x ftv70 1766 2016 1990 1 1999 2 x 1954 21 x kro124p 33978 40183 36787 5 40180 18 x 37241 43 x p43 43 5629 5625 1 5623 1 x 5623 1 x rbg323 1326 1326 1326 71 1326 32 1326 32 rbg358 1163 1163 1163 79 1163 33 1163 33 rbg403 2465 2465 2465 302 2465 164 2465 164 rbg443 2720 2720 2720 382 2720 183 2720 183 ry48p 12517 15430 14792 1 15285 3 x 14735 5 x SUM 6 8 11 3
Table 2: Truncated DFBnB vs. local search on problems from TSP library.
Conclusions and Future Work dom walk, to strengthen a local search algorithm. We would like to conclude the paper by pointing out Depth-first branch-and-bound (DFBnB) is not only a that one of the major results of the paper, i.e., a sys- general technique for optimally solving difficult NP- tematic search may significantly outperform a nonsys- complete combinatorial optimization problems, but can tematic search, is not an isolated observation on a par- also be adapted to efficient anytime and approximation ticular problem. Similar observations have been made algorithms. In this paper, we studied DFBnB and trun- on number partitioning (Korf 1998) and random coding cated DFBnB, a DFBnB with an early termination, on networks (Dechter & Kask ; Kask & Dechter 1999), on the asymmetric Traveling Salesman Problem (ATSP) which DFBnB significantly outperforms the best known of various structures and sizes. Specifically, we ex- stochastic search. Our future research will concentrate perimentally compared DFBnB and truncated DFBnB on characterizing the common features of these differ- against the Kanellakis-Papadimitriou local search algo- ent application domains and identifying the conditions rithm, the best known approximation algorithm for the under which a systematic search is able to outperform ATSP. Our experimental results showed that DFBnB a nonsystematic search. outperforms the local search algorithm, finding better ATSP tours significantly earlier than the local search, Acknowledgments on large ATSP and various problem structures. The contribution of this work is twofold. First, to the This work was funded by NSF Grant #IRI-9619554. specific problem of the ATSP, it provides a thorough comparison of DFBnB and the local search, showing References that DFBnB and truncated DFBnB are the choices of Balas, E., and Toth, P. 1985. Branch and bound algorithms for the problem in practice. Second, beyond methods. In The Traveling Salesman Problem. Essex, the specific problem of the ATSP, this work shows that England: John Wiley & Sons. 361–401. DFBnB, a systematic approach, is also well suited for approximate and anytime problem solving. The results Carpaneto, G., and Toth, P. 1980. Some new branch- of this paper demonstrated that DFBnB can compete ing and bounding criteria for the asymmetric traveling with and outperform a local search algorithm for finding salesman problem. Management Science 26:736–743. both approximation and anytime solutions. Dean, T., and Boddy, M. 1988. An analysis of time- The poor performance of the Kanellakis-Papadim- dependent planning. In Procceedings of the 7th Na- itriou algorithm suggests two possible future research tional Conference on Artificial Intelligence (AAAI- for the ATSP. One is to define more effective local- 88), 49–54. search neighborhood structures, and the other is to de- Dechter, R., and Kask, K. Personal communications. velop strategies for escaping local minima, such as ran- 1999-2000. Frieze, A.; Galbiati, G.; and Maffioli, F. 1982. On Reddi, S. S., and Ramamoorthy, C. V. 1972. On the the worst-case performance of some algorithms for flowshop sequencing problem with no wait in process. the asymmetric traveling salesman problem. Network Operational Research Quarterly 23:323–331. 12:23–39. ftp://ftp.zib.de/pub/packages/mp- Hoebel, L., and Zilberstein, S., eds. 1997. Proceedings testdata/tsp/tsplib/tsplib.html. of the AAAI Workshop on Building Resource-Bounded Zhang, W., and Korf, R. E. 1995. Performance of Reasoning Systems. Providence, RI: AAAI. linear-space search algorithms. Artificial Intelligence Horvitz, E., and Zilberstein, S., eds. 1996. Proceedings 79:241–292. of the AAAI Fall Symposium on Flexible Computation Zhang, W., and Korf, R. E. 1996. A study of complex- in Intelligent Systems: Results, Issues and Opportuni- ity transitions on the asymmetric Traveling Salesman ties. Cambridge, MA: AAAI. Problem. Artificial Intelligence 81:223–239. Ibaraki, T.; Muro, S.; Murakami, T.; and Hasegawa, T. 1983. Using branch-and-bound algorithms to ob- tain suboptimal solutions. Zeitchrift f¨ur Operations Research 27:177–202. Johnson, D. S., and McGeoch, L. A. 1997. The Travel- ing Salesman Problem: A case study. In Local Search in Combinatorial Optimization. John Wiley & Sons. 215–310. Johnson, D. S. 1990. Local optimization and the traveling salesman problem. In Proc. of the 17th In- tern. Colloquium on Automata, Languages and Pro- gramming, 446–461. Kanellakis, P. C., and Papadimitriou, C. H. 1980. Lo- cal search for the asymmetric traveling salesman prob- lem. Operations Research 28:1086–1099. Karp, R. M. 1979. A patching algorithm for the non- symmetric Traveling-Salesman Problem. SIAM Jour- nal on Computing 8:561–573. Kask, K., and Dechter, R. 1999. Stochastic local search for bayesian networks. In Proc. Intern. Work- shop on AI and Statistics. Korf, R. E. 1998. A complete anytime algorithm for number partitioning. Artificial Intelligence 105:133– 155. Lawler, E. L.; Lenstra, J. K.; Kan, A. H. G. R.; and Shmoys, D. B. 1985. The Traveling Salesman Problem. Essex, England: John Wiley & Sons. Lin, S., and Kernighan, B. W. 1973. An effective heuristic algorithm for the traveling salesman problem. Operations Research 21:498–516. Martello, S., and Toth, P. 1987. Linear assignment problems. Annals of Discrete Math. 31:259–282. Miller, D. L., and Pekny, J. F. 1991. Exact solution of large asymmetric traveling salesman problems. Sci- ence 251:754–761. Papadimitriou, C. H., and Kanellakis, P. C. 1980. Flowshop scheduling with limited temporary storage. Journal of ACM 27:533–549. Papadimitriou, C. H., and Steiglitz, K. 1982. Com- binatorial Optimization: Algorithms and Complexity. Englewood Cliffs, NJ: Prentice-Hall. Papadimitriou, C. H. 1994. Computational Complex- ity. Reading, MA: Addison-Wesley.