NUMERICAL ALGEBRA, doi:10.3934/naco.2012.2.69 CONTROL AND OPTIMIZATION Volume 2, Number 1, March 2012 pp. 69–90 UNIVARIATE GEOMETRIC LIPSCHITZ GLOBAL OPTIMIZATION ALGORITHMS Dmitri E. Kvasov and Yaroslav D. Sergeyev1 DEIS, University of Calabria Via P. Bucci, Cubo 42C, 87036 – Rende (CS), Italy Software Department, N.I. Lobachevsky State University Gagarin Av. 23, 603950 – Nizhni Novgorod, Russia (Communicated by David Gao) Abstract. In this survey, univariate global optimization problems are consid- ered where the objective function or its first derivative can be multiextremal black-box costly functions satisfying the Lipschitz condition over an interval. Such problems are frequently encountered in practice. A number of geometric methods based on constructing auxiliary functions with the usage of different estimates of the Lipschitz constants are described in the paper. 1. Introduction. Decision-making problems stated as problems of optimization of an objective function subject to a set of constrains arise in various fields of human activity such as engineering design, economic models, biology studies, etc. Optimization problems characterized by the functions with several local optima (typically, their number is unknown and can be very high) have a great importance for practical applications. These problems are usually referred to as multiextremal, or global optimization, ones. Both the objective function and constraints can be black-box and hard to evaluate functions with unknown analytical representations. Such a type of functions is frequently met in real-life applications, but the problems related to them often cannot be solved by traditional optimization techniques (see, e.g., [13, 20, 22, 26, 27, 35, 50, 68, 81, 91] and the references given therein) usually making strong suppositions (convexity, differentiability, etc.) that cannot be used with multiextremal problems. This explains the growing interest of researchers in developing numerical global optimization methods able to tackle this difficult class of problems (see, e.g., [4, 21, 37, 42, 48, 51, 73, 82, 85, 86, 96, 111, 113, 123]). A priori assumptions on the objective function serve as mathematical tools for obtaining estimates of the global solution related to a finite number of function evaluations (trials) and, therefore, play a key role in the construction of any efficient global search algorithm. Competitive global optimization methods are, as a rule, 2000 Mathematics Subject Classification. Primary: 65K05, 90C26; Secondary: 90C56. Key words and phrases. Global optimization, black-box function, Lipschitz condition, geomet- ric approach. This work was supported by the grants 1960.2012.9 and MK-3473.2010.1 awarded by the Pres- ident of the Russian Federation for supporting the leading research groups and young researchers, respectively, as well as by the grant 11-01-00682-a awarded by the Russian Foundation for Funda- mental Research. 1Corresponding author 69 70 DMITRIE.KVASOVANDYAROSLAVD.SERGEYEV sequential (see, e.g., [116]), i.e., the choice of new trials depends on the information obtained by such an algorithm at previous iterations. Since each function trial is a resource-consuming operation, it is desirable to obtain a required approximation of the problem solution in a relatively few iterations. One of the natural and valid from both the theoretical and the applied points of view suppositions on the global optimization problem is that the objective function and eventual constraints have bounded slopes. In other words, any limited change in the object parameters gives rise to some limited changes in the characteristics of the objective behavior. This can be justified by the fact that in technical systems the energy of change is always limited (see the related discussion in [113, 115]). One of the most popular and simple mathematical formulations of this property is the Lipschitz continuity condition, which assumes that the difference (in the sense of a chosen norm) of any two function values is majorized by the difference of the corresponding function arguments, multiplied by a positive factor L. In this case, the function is said to be Lipschitz and the corresponding factor L is said to be Lipschitz constant. The problem with either the Lipschitz objective functions or the objective functions having multiextremal Lipschitz first derivatives is said to be Lipschitz global optimization (LGO) problem. The Lipschitz continuity assumption, being quite realistic for many practical problems (see, e.g., [52, 53, 83, 85, 86, 96, 113, 123]), is also a suitable tool for devel- oping, studying and applying the so-called geometric LGO methods, i.e., sequential methods that use in their work auxiliary functions to estimate the objective function behavior over the search region. Together with other techniques for solving LGO problems (see, e.g., [12, 38, 41, 52, 53, 57, 73, 80, 113, 118, 123, 125]), the geometric idea has proved to be very fruitful and many algorithms based on constructing and improving auxiliary functions built by using the Lipschitz constant estimates have been proposed. Many of these methods can be studied within a general framework (as branch-and-bound scheme [53, 54, 85] or divide-the-best approach [96, 98, 105]) making them even more attractive for both theoretical and applied research. Dif- ferent geometric LGO algorithms will be considered in this paper. In order to give an insight to the class of geometric LGO methods, in what follows we shall pay our attention to one-dimensional problems. In global optimization, these problems play a very important role both in the theory and practice and, therefore, were intensively studied in the last decades (see, e.g., [21, 33, 38, 52, 85, 96, 113, 115, 116, 123]). In fact, on the one hand, theoretical analysis of one-dimensional problems is quite useful since mathematical approaches developed to solve them very often can be generalized to the multidimensional case by numerous schemes (see, e.g., [24, 52, 53, 56, 61, 64, 72, 82, 84, 85, 96, 106, 113, 123]). On the other hand, there exists a large number of real-life applications where it is necessary to solve these problems (see, e.g., [82, 85, 86, 90, 96, 113, 123]). Electrical engineering and electronics are among the fields where the usage of efficient one-dimensional global optimization methods are often required (see, e.g., [18, 33, 90, 93, 113]). Let us consider, for example, the following common problem in electronic mea- surements and electrical engineering. There exists a device whose behavior depends on a characteristic f(x), x ∈ [a,b], where the function f(x) may be, for instance, an electrical signal obtained by a complex computer aided simulation over a time interval [a,b] (see the function graph in thick line in Fig. 1). The function f(x) is often multiextremal and Lipschitz (it can be also differentiable with the Lipschitz first derivative). The device works correctly while f(x) > 0. Of course, at the initial UNIVARIATE GEOMETRIC LIPSCHITZ GLOBAL OPTIMIZATION 71 Figure 1. The problem of finding the minimal root of equa- tion f(x) = 0 with multiextremal non-differentiable left part arising in electrical engineering moment x = a we have f(a) > 0. It is necessary to describe the performance of the device over the time interval [a,b] either determining the point x∗ such that f(x∗)=0, f(x) > 0, x ∈ [a, x∗), x∗ ∈ (a,b], (1) or demonstrating that x∗ satisfying (1) does not exist in [a,b] (in this case the device works correctly for the whole time period; thus, an information about the global minimum of f(x) could be useful in practice to measure the device reliability). This problem is equivalent to the problem of finding the minimal root (the first root from the left) of the equation f(x) = 0, x ∈ [a,b], in the presence of certain initial conditions and can be reformulated as a global optimization problem. There is a simple approach to solve this problem based on a grid technique. It produces a dense mesh starting from the left margin of the interval and going on by a small step till the signal becomes less than zero. For an acquired signal, the determination of the first zero crossing point by this technique is rather slow especially if the search accuracy is high. Since the objective function f(x) is multiextremal (see Fig. 1) the problem is even more difficult because many roots can exist in [a,b] and, therefore, classical root finding techniques can be inappropriate. The rest of the paper is organized as follows. In Section 2, the Lipschitz global optimization problem is formally stated (for both non-differentiable and differen- tiable objective functions) and an overview of geometric ideas to its solving is given. A number of geometric LGO methods are described in Section 3 (in the case of Lip- schitz non-differentiable functions) and in Section 4 (in the case of differentiable functions with the Lipschitz first derivatives). It should be noted that to expose the principal ideas of some known geometric ap- proaches to solving the stated problem, box-constrained LGO problems will be con- sidered in the paper. This special case stays at the basis of the global optimization methods managing general multiextremal constraints. For example, such a promis- ing global optimization approach as the index scheme (see, e.g., [9, 97, 107, 112, 113]) reduces the general constrained problem to a (discontinuous) box-constrained one having a special nice structure. 2. Lipschitz global optimization problem. Formally, a box-constrained one- dimensional Lipschitz global optimization problem can be stated as follows (for 72 DMITRIE.KVASOVANDYAROSLAVD.SERGEYEV the sake of certainty, we shall consider the minimization problem). Given a small positive constant ε, it is required to find an ε-approximation of the global minimum point (global minimizer) x∗ of a multiextremal, black-box (and, often, hard to evaluate) objective function f(x) over a closed interval [a,b]: f ∗ = f(x∗) = min f(x), x ∈ [a,b].