Interval Methods for Global Optimization by Belaid Moa B.Eng., Ecole Hassania des Travaux Publics, Morocco, 1998 M.Eng., Ecole Nationale Superieure´ d’Electrotechnique, d’Electronique, d’Informatique, d’Hydraulique et des Tel´ ecommunications,` France, 2000 A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the Department of Computer Science ­c Belaid Moa, 2007 University of Victoria All rights reserved. This dissertation may not be reproduced in whole or in part by photocopy or other means, without the permission of the author. Interval Methods for Global Optimization by Belaid Moa B.Eng., Ecole Hassania des Travaux Publics, Morocco, 1998 M.Eng., Ecole Nationale Superieure´ d’Electrotechnique, d’Electronique, d’Informatique, d’Hydraulique et des Tel´ ecommunications,` France, 2000 Supervisory Committee Dr. M.H. van Emden, Supervisor (Department of Computer Science) Dr. W.W. Wadge, Co-supervisor (Department of Computer Science) Dr. J. Ellis, Department Member (Department of Computer Science) Dr. F. Gebali, Outside Member (Department of Electrical and Computer Engineering) Dr. M. Ibnkahla, External Examiner (Department of Electrical Engineering, Queen’s University) Supervisory Committee Dr. M.H. van Emden, Supervisor (Department of Computer Science) Dr. W.W. Wadge, Co-supervisor (Department of Computer Science) Dr. J. Ellis, Department Member (Department of Computer Science) Dr. F. Gebali, Outside Member (Department of Electrical and Computer Engineering) Dr. M. Ibnkahla, External Examiner (Department of Electrical Engineering, Queen’s University) ABSTRACT We propose interval arithmetic and interval constraint algorithms for global optimization. Both of these compute lower and upper bounds of a function over a box, and return a lower and an upper bound for the global minimum. In interval arithmetic methods, the bounds are computed using interval arithmetic evaluations. Interval constraint methods instead use domain reduction operators and consistency algorithms. The usual interval arithmetic algorithms for global optimization suffer from at least one of the following drawbacks: ¯ Mixing the fathoming problem, in which we ask for the global minimum only, with the localization problem, in which we ask for the set of points at which the global minimum occurs. ¯ ¯ Not handling the inner and outer approximations for -minimizer, which is the set of iv points at which the objective function is within ¯ of the global minimum. ¯ Nothing is said about the quality for their results in actual computation. The prop- erties of the algorithms are stated only in the limit for infinite running time, infinite memory, and infinite precision of the floating-point number system. To handle these drawbacks, we propose interval arithmetic algorithms for fathoming prob- lems and for localization problems. For these algorithms we state properties that can be verified in actual executions of the algorithms. Moreover, the algorithms proposed return the best results that can be computed with given expressions for the objective function and the conditions, and a given hardware. Interval constraint methods combine interval arithmetic and constraint processing tech- niques, namely consistency algorithms, to obtain tighter bounds for the objective function over a box. The basic building block of interval constraint methods is the generic propa- gation algorithm. This explains our efforts to improve the generic propagation algorithm as much as possible. All our algorithms, namely dual, clustered, deterministic, and selec- tive propagation algorithms, are developed as an attempt to improve the efficiency of the generic propagation algorithm. The relational box-consistency algorithm is another key algorithm in interval constraints. This algorithm keeps squashing the left and right bounds of the intervals of the variables until no further narrowing is possible. A drawback of this way of squashing is that as we proceed further, the process of squashing becomes slow. Another drawback is that, for some cases, the actual narrowing occurs late. To address these problems, we propose the following algorithms: 1. Dynamic Box-Consistency algorithm: instead of pruning the left and then the right bound of each domain, we alternate the pruning between all the domains. 2. Adaptive Box-Consistency algorithm: the idea behind this algorithm is to get rid of the boxes as soon as possible: start with small boxes and extend them or shrink them depending on the pruning outcome. This adaptive behavior makes this algorithm v very suitable for quick squashing. Since the efficiency of interval constraint optimization methods depends heavily on the sharpness of the upper bound for the global minimum, we must make some effort to find the appropriate point or box to use for computing the upper bound, and not to randomly pick one as is commonly done. So, we introduce interval constraints with exploration. These methods use non-interval methods as an exploratory step in solving a global opti- mization problem. The results of the exploration are then used to guide interval constraint algorithms, and thus improve their efficiency. Table of Contents Supervisory Committee ii ABSTRACT iii Table of Contents vi List of Tables xi List of Figures xii List of Abbreviations xvi Notations xvii Glossary xix Acknowledgement xxii Dedication xxiii 1 Introduction 1 1.1 Motivation . 1 1.2 The problem statement . 2 1.3 Methods for solving global optimization problems . 4 1.4 Layout of the dissertation . 6 Table of Contents vii 2 Interval Arithmetic 10 2.1 From floating-point numbers to interval arithmetic . 10 2.2 Advantages of interval arithmetic . 11 2.3 Basics of interval arithmetic . 12 2.3.1 Intervals . 12 2.3.2 Interval extensions and Inclusion functions . 14 2.4 Solving inequalities via interval arithmetic . 16 2.5 Functional box consistency . 18 3 Unconstrained global optimization via interval arithmetic 21 3.1 Motivation . 21 3.2 The fathoming problem . 24 3.2.1 The first fathoming problem . 25 3.2.2 The second fathoming problem . 27 3.3 The localization problem . 34 3.3.1 The outer approximation . 34 3.3.2 The inner approximation . 37 4 Constrained global optimization via interval arithmetic 40 4.1 Overview . 41 4.2 The feasibility problem . 43 4.2.1 The outer approximation . 44 4.2.2 The inner approximation . 45 4.3 The fathoming problem . 49 4.4 The localization problem . 53 5 Interval constraints 59 5.1 Constraint satisfaction problems . 59 5.1.1 Constraints . 60 Table of Contents viii 5.1.2 Constraint propagation . 62 5.2 Interval constraints and interval constraint solver . 63 6 Dual, clustered, deterministic, and selective propagation algorithms 70 6.1 Introduction . 71 6.2 Relations and Recursive Functions on Power Sets . 72 6.3 Dual Propagation Algorithms . 74 6.4 Clustered Propagation Algorithms . 77 6.5 Deterministic propagation algorithms . 79 6.6 Selective propagation algorithms . 80 6.7 Implementation . 80 7 Dynamic and adaptive box-consistency algorithms 85 7.1 Box consistency . 85 7.2 Dynamic box consistency . 86 7.3 Adaptive box consistency . 88 7.4 Other improvements . 91 8 Global optimization via interval constraints 95 8.1 Overview . 95 8.1.1 Computing a lower bound for ÑÒ .................. 96 8.1.2 Computing an upper bound for ÑÒ ................. 96 8.1.3 Branch-and-bound in interval constraints . 97 8.2 Exploration and interval constraints for global optimization . 98 8.2.1 The fundamental rule of P-NP . 98 8.2.2 Motivation of exploration . 98 8.2.3 Interval constraints with exploration . 99 8.2.4 Effect of exploration on the upper bound of ÑÒ ..........101 8.2.5 Effect of exploration on the lower bound of ÑÒ ..........101 Table of Contents ix 8.2.6 Effect of exploration on splitting the search space . 102 8.2.7 Propagating or squashing? . 104 8.3 Interval constraints as a verification tool . 107 8.4 Interval constraint solver with exploration . 108 8.4.1 Effect of exploration on splitting and propagation . 109 8.4.2 Effect of exploration on squashing algorithms . 109 8.4.3 Does a box contain a solution to ? ................110 8.4.3.1 involves inequalities only . 111 8.4.3.2 involves equalities also . 114 9 Experimental results 117 9.1 Description of BelSyste . 117 9.1.1 BelSyste architecture . 117 9.1.2 BelSyste components . 119 9.2 Experimental results . 121 9.2.1 An application using squashing and propagation . 121 9.2.2 Box consistency . 123 9.2.3 Box satisfaction . 125 9.2.4 Box unsatisfaction . 126 9.2.5 Largest box in a convex set . 126 9.2.6 Booth function . 131 9.2.7 Rosenbrock banana function . 133 9.2.8 Six-hump camel function . 134 9.2.9 Discrete examples . 137 9.2.10 An inequality on integers . 137 9.2.11 Integral points on elliptic curves . 138 9.2.11.1 Fermat’s last theorem . 139 9.2.11.2 Taxicab number . 139 Table of Contents x 9.2.12 BelSyste as a verification tool . 140 10 Conclusion 143 10.1 Contributions . 143 10.1.1 Interval arithmetic . 143 10.1.2 Interval constraints . 143 10.1.2.1 Propagation . 143 10.1.2.2 Relational box-consistency . 144 10.1.2.3 Interval constraints with exploration . 144 10.1.3 Implementation . 145 10.2 Future work . 145 10.2.1 Dependency effect . 146 10.2.2 Parametric CSPs . 146 10.2.3 Criteria for choosing a consistency algorithm . 147 10.2.4 Confidence quantification . 147 10.2.5 Disjunctive constraints . 148 10.2.6 Iterative widening with exploration . 148 Bibliography 149 Appendix A Source Code 154 A.1 Box consistency . 154 A.2 Adaptive box consistency . 155 A.3 Dynamic box consistency . 155 A.4 Box satisfaction algorithms . 156 A.5 Default explorer and explorer with propagation . 159 A.6 Interval constraint global optimization solver with exploration . 159 List of Tables Table 9.1 Propagating and squashing in an interval constraint solver.