ICCAS2004 August 25-27, The Shangri-La Hotel, Bangkok, THAILAND A Study for Global Optimization Using Dynamic Encoding Algorithm for Searches Nam Geun Kim, Jong-Wook Kim and Sang Woo Kim Division of Electrical and Computer Engineering, POSTECH, Pohang, Korea (Tel: +82-54-279-5018; Fax: +82-54-279-2903; Email:{kimng, kjwooks, swkim}@postech.ac.kr) Abstract: This paper analyzes properties of the recently developed nonlinear optimization method, Dynamic Encoding Algorithm for Searches (DEAS) [1]. DEAS locates local minima with binary strings (or binary matrices for multi-dimensional problems) by iterating the two oper- ators; bisectional search (BSS) and unidirectional search (UDS). BSS increases binary strings by one digit (i.e., zero or one), while UDS performs increment or decrement to binary strings with no change of string length. Owing to these search routines, DEAS retains the opti- mization capability that combines the special features of several conventional optimization methods. In this paper, a special feature of BSS and UDS in DEAS is analyzed. In addition, a effective global search strategy is established by using information of DEAS. Effectiveness of the proposed global search strategy is validated through the well-known benchmark functions. Keywords: DEAS, optimization, global search. 1. Introduction real-valued difference is equidistant under the same string length. Many practical engineering problem can be formulated as optimiza- In DEAS, these familiar characteristics of a binary string is applied tion problems using an objective function whose domain, D, repre- to the determination of a search direction and step length, we can sents the space of feasible solutions and whose range represents the get the gradient information and varying rate of step length easily relative utility of each solution. Solving the optimization problem is without another calculation process. GA is comparable to DEAS in to find the solution in D for which the objective function obtains its that it requires no derivative information of a cost function and uses smallest value, the global minimum. Global optimization thus aims the same decoding function, but differs by its concatenated binary at determining not just a local minimum but the smallest local min- structure. However, since GA simulates the natural selection and imum in the region D. Global optimization is an inherently difficult evolution on a digital computer, the basic philosophies of GA and problem since no general criterion exists for determining whether DEAS are quite different. Generally, most algorithms do not have the global optimum has been reached. Therefor, most global op- well defined stopping conditions. In practice, some methods are timization algorithms are required infinitely many function evalua- halted after a fixed number of iterations, or when the step size be- tions [2]. comes smaller than a given threshold. In DEAS, we can make rea- sonably stopping condition that stops the search when the length of In general, global optimization can be categorized into two differ- increasing binary strings reaches the desired grid achieving an accu- ent approaches; deterministic and stochastic methods, as they were racy on D. In order to address general global optimization problems, termed in [3] and [4]. Deterministic methods can guarantee absolute however, it is necessary to develop an appropriate global optimiza- success, but only by making certain restrictive assumptions, the cost tion scheme for DEAS whose performance is verified by benchmark function should be sufficiently smooth with twice differentiability, tests. In this paper, a multistart approach is embedded into DEAS which is quite a rigorous condition to real-world problems. Stochas- with various revisit check functions for efficient global search. tic methods evaluate the objective function at randomly generated The paper is organized as follows: Section 2 describes the local points. The convergence results for these methods are not absolute. and global search strategies of the proposed method with illustrative However, the probability of success approaches one as the sample examples. Section 3 gives the optimization results of the benchmark size tends to infinity under very mild assumptions about the objec- and parameter identification. Finally, section 4 gives conclusions. tive function. Not using the gradient information, these algorithms are more straightforward to apply but less efficient than determinis- tic methods in terms of their running time and solution qualities. 2. Dynamic Encoding Algorithm for Searches In this section, an optimization problem is defined and The ba- To make a more effective global optimization strategy, the two ba- sic search strategies of DEAS are introduced. In addition, several sic approaches are combined even if the basic philosophies behind global search strategy are proposed. the two approaches are quite different. In the sequel, a nonlin- 2.1. Optimization Problem ear optimization algorithm named dynamic encoding algorithm for Given a objective function F (x) as a minimum problem with- searches (DEAS) is developed in the framework of discrete struc- ∗ out constraints, the aim of global optimization is to find x = tures. DEAS is originated from the need of the compromise be- ∗ ∗ ∗ T [x1 x2 ··· xn] such that tween deterministic and stochastic search methods. In other words, an iterative algorithm which directs a current minimum to suc- x∗ = arg min{ F (x) | x ∈ Rn}. x cessively approach to the optimum without derivative information. DEAS employs the peculiar property of a binary string that if a bi- and the associated extreme value F ∗ = F (x∗) of the objective nary digit, 0 or 1, is concatenated to a binary string as a new least function F (x), in this case the minimum. The expression x ∈ Rn significant bit (LSB), the decoded real value of the new string is means that the variables are allowed to take all real values; x can slightly decreased for 0 or increased for 1. In addition, if a binary thus be represented by any point in an n-dimensional Euclidean string undergoes increment addition or decrement subtraction, the space Rn. 857 For a local minimum the following relationship holds: F (x∗) ≤ F(x∗) n ∗ ∗ 2 n for 0 ≤x − x = (xi − xi ) ≤ ε, x ∈ R i=1 This means that in the neighborhood of x∗ defined by the size of ε there is no vector x for which F (x) is smaller than F (x∗). In DEAS, we account for the fact that DEAS procedure can only produce approximate answers. Hence we consider the problem solved if for some ε>0 we find a solution in the level set Ld, where d = f(x)+ε and Ld = { x | f(x) ≤ d}. This is said that a solution x ∈ Ld is ε-accurate. 2.2. Basic Behavior of DEAS The basic behavior of DEAS is described by two terminologies: bisectional search (BSS) and unidirectional search (UDS). 2.2.1 Bisectional Search If a binary digit, 0 or 1, is appended to an existing binary string as a least significant bit (LSB), the decoded real number of a new binary string decreases for 0, and increases for 1 compared with that of the original binary string. This property is utilized in search algorithm, Fig. 1. One, two, and three dimensional diagrams of neighborhood DEAS, to generate the neighborhood, the set of solution that can be points generated by BSS reached from the current solution in a single iteration. The name ‘bisectional search’ comes from this dichotomous search aspect. end if From the viewpoint of local search techniques, the BSS can be com- d ← d +1 prehended as the selection of the best neighborhood matrices among i ← i +1 n the 2 neighborhood ones. In Fig. 1, the neighborhood strings gen- end while erated by the BSS are depicted with points. Among those neigh- Let an i-th dimensional m-bit-long binary string, borhood matrices, the best one whose cost is lowest is saved as an optimal matrix. Bi,m = bm ···bj ···b1,bj ∈{0, 1},j=1, ··· ,m, The pseudocode for an n-dimensional problem of BSS is stated as be termed a parent string. If 0 is added to the LSB of Bi,m, the follows,where BSS is undertaken for an n × m binary matrix. Note l new string is termed a ‘left-hand child string’, Bi,m+1. On the that n equals the number of search parameters, and bit number of other hand, if 1 is added, it is termed a ‘right-hand child string’, string, m, can vary from 1 to any finite number. r Bi,m+1. Bisectional Search ( Bn×m) In the process of DEAS, the mechanism of decoding from the binary m n Initialization: vector space {0, 1} to restricted continuous space R on finite i Set a direction vector as an all-zero binary string; intervals [ui,vi] for each variable x ∈ R is required. According i m d = 0n×1. to the standard binary decoding function fd : {0, 1} → [ui,vi], Set Jmin ← temporary variable, M 1 and i ← 1. where [5], the real value of the parent string is n while i ≤ 2 do m vi − ui j−1 d xi = fd(Bi,m)=ui + bj 2 , (1) Add as a least significant column; 2m − 1 j=1 Bn×(m+1) =[Bn×m d]. Decode a temporary matrix into a real vector as and the child strings are respectively f d m Bn× m → Xn×1. ( +1) l l vi − ui j x = fd(B )=ui + bj 2 Evaluate the cost; i+1 i,m+1 m+1 − 2 1 j J = f(X). =1 m J<J 2 +1 − 2 if min then = xi, m+1 − Save the current best optima; 2 1 m ∗ vi − ui vi − ui Bn× m ← Bn×(m+1), r r j ( +1) xi = fd(Bi,m )=ui + bj 2 + ← +1 +1 2m+1 − 1 2m+1 − 1 dopt d, j=1 ← m Xmin X, 1 2 +1 − 2 J ← J.