How to Solve a Semi-infinite Optimization Problem Oliver Stein∗ March 27, 2012 Abstract After an introduction to main ideas of semi-infinite optimization, this article surveys recent developments in theory and numerical meth- ods for standard and generalized semi-infinite optimization problems. Particular attention is paid to connections with mathematical pro- grams with complementarity constraints, lower level Wolfe duality, semi-smooth approaches, as well as branch and bound techniques in adaptive convexification procedures. A section on recent genericity re- sults includes a discussion of the symmetry effect in generalized semi- infinite optimization. Keywords: Semi-infinite optimization, design centering, robust optimiza- tion, mathematical program with complementarity constraints, Wolfe dual- ity, semi-smooth equation, adaptive convexification, genericity, symmetry. AMS subject classifications: 90C34, 90C30, 49M37, 65K10, 00-02 ∗Institute of Operations Research, Karlsruhe Institute of Technology (KIT), Germany, [email protected] 1 1 Introduction This article reviews recent developments in theory, applications and numer- ical methods of so-called semi-infinite optimization problems, where finitely many variables are subject to infinitely many inequality constraints. In a general form, these problems can be stated as GSIP : minimize f(x) subject to x 2 M with the set of feasible points M = fx 2 Rnj g(x; y) ≤ 0 for all y 2 Y (x)g (1) and a set-valued mapping Y : Rn ⇒ Rm describing the index set of inequality constraints. The defining functions f and g are assumed to be real-valued and at least continuous on their respective domains. Standard assumptions on the set-valued mapping Y are somewhat technical and will be stated in Section 3 below. Clearly, the main numerical challenge in semi-infinite optimization is to find ways to treat the infinitely many constraints. While in the case of an x−dependent index set Y (x) one speaks of a general- ized semi-infinite problem (thus, the acronym GSIP), the important subclass formed by problems with constant index set Y ⊂ Rm is termed standard semi-infinite optimization (SIP). As basic references we mention [34] for an introduction to semi-infinite op- timization, [36, 74] for numerical methods in SIP, and the monographs [23] for linear semi-infinite optimization as well as [70] for algorithmic aspects. The monograph [85] contains a detailed study of generalized semi-infinite optimization. The most recent comprehensive reviews on theory, applications and numerical methods of standard and generalized semi-infinite optimization are [58] and [31], respectively. The reader is referred to these articles for descriptions of the state of art in semi-infinite optimization around the year 2007. The present paper, on the other hand, will focus on important developments of this very active field during the past five years. As internet resources we recommend the semi-infinite programming bibliography [59] with several hundreds of references, and the NEOS semi-infinite programming directory [22]. Furthermore, since the most recent results on stability theory for linear semi-infinite problems are excellently surveyed in [57], we will not discuss them in the present paper. 2 This article is structured as follows. In Section 2 we will briefly discuss some classical as well as recent applications of semi-infinite optimization. Sec- tion 3 introduces basic theoretical concepts on which both the derivation of optimality conditions and the design of numerical methods rely. Section 4 explains how the bilevel structure of semi-infinite optimization can be em- ployed numerically, where besides a well-known lifting approach leading to mathematical programs with complementarity constraints, we also present recent results on a lifting idea resulting in nondegenerate smooth problems. Section 5 describes a feasible point algorithm for standard SIPs which is based on recent developments in global optimization. Important genericity results in generalized semi-infinite optimization, which were obtained only in the past few years, are reviewed in Section 6, before we close the article with some final remarks in Section 7. 2 Applications Among the numerous applications of semi-infinite optimization, in this sec- tion we focus on design centering and on robust optimization. We emphasize, however, that semi-infinite optimization historically emerged as a smooth re- formulation of the intrinsically nonsmooth Chebyshev approximation prob- lem (cf., e.g., [31, 36, 85]). Further applications include the optimal layout of an assembly line ([47, 102]), time minimal control ([51, 54, 102]), disjunc- tive optimization ([85]) and, more recently, robust portfolio optimization ([13, 105, 106]), the identification of regression models as well as the dynam- ics of networks in the presence of uncertainty ([55, 103]). We also remark that semi-definite optimization ([100, 109]) can be interpreted as a special case of SIP. This approach is elaborated in [18, 101]. 2.1 Design centering A design centering problem consists in maximizing some measure f(x), for example the volume, of a parametrized body Y (x) while it is inscribed into a container set G(x): DC : max f(x) s.t. Y (x) ⊂ G(x): x2Rn Design centering problems have been studied extensively, see for example [25, 38, 68, 69, 87] and the references therein. They are also related to the so-called set containment problem from [61]. 3 In applications the set G(x) often is independent of x and has a complicated structure, while Y (x) possesses a simpler geometry. For example, in the so- called maneuverability problem of a robot from [24] the authors determine lower bounds for the volume of a complicated container set G by inscribing an ellipsoid Y (x) into G whose volume can be calculated. This approach actually gave rise to one of the first formulations of a generalized semi-infinite optimization problem in [35]. An obvious application of design centering is the cutting stock problem. The problem of cutting a gem of maximal volume with prescribed shape features from a raw gem is treated in [68] and, with the bilevel algorithm for semi- infinite optimization from [93] (cf. also Sec. 4.1), in [108]. For a gentle description of this industrial application see [56]. The connection with semi-infinite optimization becomes apparent when we assume that the container set G(x) is described by a functional constraint, G(x) = fy 2 Rmj g(x; y) ≤ 0g: (2) Then the inclusion constraint Y (x) ⊂ G(x) of DC is equivalent to the gen- eralized semi-infinite constraint g(x; y) ≤ 0 for all y 2 Y (x); so that the design centering problem becomes a GSIP. Note that an (easier to treat) standard SIP appears, if the body Y is independent of x, while the container G(x) is allowed to vary with x. Many design centering problems can actually be reformulated as standard SIPs, if the x−dependent transfor- mations of Y (x) are, for example, translations, rotations, or scaling, whose inverse transformations can as well be imposed on the container set. We point out that any GSIP with a given constraint function g can be interpreted as a design centering problem by defining G(x) as in (2). Thus, generalized semi-infinite optimization and design centering are equivalent problem classes. 2.2 Robust optimization Robustness questions arise when an optimization problem is subject to un- certain data. If, for example, an inequality constraint function g(x; y) de- pends on some uncertain parameter vector y from a so-called uncertainty set Y ⊂ Rm, then the `robust' or `pessimistic' way to deal with this constraint 4 is to use its worst case reformulation g(x; y) ≤ 0 for all y 2 Y; which is clearly of semi-infinite type. A point x which is feasible for this semi- infinite constraint satisfies g(x; y) ≤ 0, no matter what the actual parameter y 2 Y turns out to be. This approach is also known as the `principle of guaranteed results' (cf. [21]). When the uncertainty set Y also depends on the decision variable x, we arrive at a generalized semi-infinite constraint. For example, uncertainties concerning small displacements of an aircraft may be modeled as being dependent on its speed. For an example from portfolio analysis see [93]. In [7] it is shown that under special structural assumptions the semi-infinite problem appearing in robust optimization can be reformulated as a semi- definite problem and then be solved with polynomial time algorithms ([5]). Under similarly special assumptions a saddle point approach for robust pro- grams is given in [99]. We will discuss these assumption in some more detail in the subsequent Section 3. The current state of the art in robust optimiza- tion is surveyed in [8]. Again, any GSIP can be interpreted as a robust optimization problem, so that also these two problem classes are equivalent. The reason for rather separate bodies of literature on semi-infinite optimization on the one hand, and robust optimization on the other hand, lies in the fact that solution methods for robust optimization are mainly motivated from a complexity point of view, whereas methods in semi-infinite optimization try to deal with the numerical difficulties which arise if the stronger assumptions of robust optimization are relaxed. 3 The lower level problem The key to the theoretical as well as algorithmic treatment of semi-infinite optimization problems lies in their bilevel structure. In fact, it is not hard to see that an alternative description of the feasible set in (1) is given by M = fx 2 Rnj '(x) ≤ 0g (3) with the function '(x) := sup g(x; y): y2Y (x) 5 The common convention `sup; = −∞" is consistent here, as an empty index set Y (x) corresponds, loosely speaking, to `the absence of restrictions' at x and, hence, to the feasibility of x. In applications (cf. Sec. 2) often finitely many semi-infinite constraints gi(x; y) ≤ 0, y 2 Yi(x), i 2 I, describe the feasible set M of GSIP, along with finitely many equality constraints, so that with 'i(x) = supy2Yi(x) gi(x; y) one obtains feasible sets of the form n M = fx 2 R j 'i(x) ≤ 0; i 2 I; hj(x) = 0; j 2 Jg: In order to avoid technicalities, however, in this article we focus on the basic case of a single semi-infinite constraint and refer the interested reader to [85] for more general formulations.