National University of Kaohsiung Repository System:Item 310360000Q/10527

全文

(1)國立高雄大學統計學研究所碩士論文. Surrogate-Assisted Trajectory Based Methodology for Optimization Problems in Computer Experiments 電腦實驗最佳化問題中的替代曲面軌跡法. 研究生：陳怡如撰指導教授：陳瑞彬博士. 中華民國九十八年七月.

(2) 謝辭寫到謝辭就表示即將要畢業，兩年的時間真的一下子就過了，想到剛進研究所的時候，還記憶猶新，轉眼間就要畢業了。從一開始我以為繼續讀研究所只是把大學所學的在延伸下去，但是讀了之後，我才知道我當時所學的實在太有限了，所以在第一學期的時候，讀的非常的痛苦，常常在想我到底有沒有能力可以繼續讀下去，但是之後找到了讀書的方法，也就可以順利的讀完研究所的課程。在這兩年裡，我最感謝的是我的指導教授陳瑞彬老師，剛開始對我所做的領域完全的不懂，從寫程式開始的學習，慢慢的從不懂到懂，由於跟應數所共同指導，所以每個禮拜都要做出東西，所以壓力很大，但是要是遇到困難老師會有耐心的教導我，有時候聽不懂得時候老師也會用不同的方法讓我了解，所以讓我每個禮拜都能安全過關，這也要感謝王偉仲老師跟莊曜遠老師不虞遺力的教導。我也要感謝學長姊們以及同學們，在我有困難的時候都會幫忙我，並且蘭屏姊也幫我解決很多的問題，也讓學會很多的事，在這兩年也從舉行研討會從中學習到很多。也感謝家人對我的支持，再我最迷惘的時候拉我一把。兩年裡，發生了許許多多的事，有開心的、有難過的，轉眼間就要結束了，突然覺得時間過的好快，即將要畢業離開熟悉的地方，多少都有些捨不得。最後，還是要謝謝所有幫助過我的教授與同學們，辛苦你們了，謝謝！. 陳怡如撰於高雄大學統計學研究所民國 98 年 7 月.

(3) Surrogate-Assisted Trajectory Based Methodology for Optimization Problems in Computer Experiments. by Yi-Ju Chen Advisor Ray-Bing Chen. Institute of Statistics, National University of Kaohsiung Kaohsiung, Taiwan 811 R.O.C. July 2009.

(4) Contents. Z` zZ`. ii iii. 1 Introduction. 1. 2 Trajectory Based Methodology. 2. 2.1. Computation of local minimum . . . . . . . . . . . . . . . . . . . . .. 2. 2.2. Computation of Decomposition point . . . . . . . . . . . . . . . . . .. 3. 3 The Optimization Algorithm 3.1. 3.2. 4. Surrogate Construction . . . . . . . . . . . . . . . . . . . . . . . . . .. 5. 3.1.1. Generate Grid and Choose Initial Experimental Points . . . .. 6. 3.1.2. Construct Surrogate . . . . . . . . . . . . . . . . . . . . . . .. 6. 3.1.3. Choose the Next Experimental Points . . . . . . . . . . . . . .. 8. Surrogate Assisted Trajectory Based Methodology . . . . . . . . . . .. 9. 4 Numerical Experiments. 12. 5 Conclusion. 35. Reference. 36. Figure. 38. A Appendix. 46. i.

(5) é\@t·;®ÞÝ`«ª° ¼0>0: Wü }ÿ »ñ{..Ù.@~X .ß: WOA »ñ{..Ù.@~X ` ChiangõChu(1996)èª°0´3DTÐóîÝIÁÂF3ª°&Æ ¿àV°0´Á ÂF¬v0´5F ®ìgÝRFQ&ÆÄ6¸à 9ÝÕWÍ0´5F LÍÎÕWÍ8 òÝ`ÎX|&ÆÄ6@1 &ÆÝÕWÍ¸à39 &Æ è`«°xÝ°Î3ÁÂF! C×ÍÝòÝ`«¼ãDTÐó ¬v3`«î0´5Ft ¡&Æ¿à¿ÍóÂ@¼J±Ý`«° n"C:é\@`«ª°. ii.

(6) Surrogate-Assisted Trajectory Based Methodology for Optimization Problems in Computer Experiments Advisor: Dr. Ray-Bing Chen Department of Statistics National University of Kaohsiung. Student: Yi-Ju Chen Institute of Statistics National University of Kaohsiung. ABSTRACT Trajectory-based methodology (TB), proposed by Chiang and Chu (1996) , can be used to identified all local extremes of an objective function. In trajectory-based methodology, we iterative to find a local extreme by a gradient type method, and then to search a decomposition point as the next starting point. However, it might take too much cost in searching decomposition points, especially when the computational cost of function evaluation is very high. Here in order to save the computational cost, a surrogate-assisted method is proposed. The basically idea is to build an inexpensive surrogate surface in a neighborhood of an identified local extreme point, and then the decomposition point search is accomplished on this surrogate surface instead of the true response function. Several numerical experiments are used to demonstrate the performance of this new surrogate-assisted method. Keyword:Computer Experiments, Surrogate, Trajectory based methodology. iii.

(7) 1. Introduction. In the article, we consider the problem of optimization an objective f : <p → < subject to bound constraints, i.e. minimize. f (x) x ∈ [a, b],. subject to. (1.1). where a, x, b ∈ Rp and x ∈ [a, b]. The task is to find the optimal solution to problems of constrained nonlinear optimization. For the optimization problem here, we make the following assumptions: (i) The response function f is unknown or very complicated, (ii) the computational cost of function evaluations is quite expensive. Several methods have been proposed for solving optimization problems. These methods can be divided into two types: the local search methods and the global search methods. Response Surface Methodology (RSM) is a popular representative of the local search methods. The RSM, proposed by Box and Wilson is a popular optimization method employing surrogate models for statistical approximation in [2]. If the response function is smooth, the local response surface can be well-approximated. The global optimization methods depend on the special structure of each problem [7],[11], [3] and [4]. The Activation-Relaxation Technique (ART), introduced to [3], is a powerful method for searching saddle points and transition pathways of a given potential energy surface (PES). The ART method is divided into two main steps: the activation step and the relaxation step. The activation step comprises of moving the system from a local minimum to a saddle point. The relaxation step consists of relaxing the system, from the computed saddle point, to another local minimum. Transition path sampling [5], trajectory-based method and the discrete path sampling [14] are methods subordinate to this method. In the article, we select trajectory-based methodology, proposed by Chiang and Chu in [4] and [10], as the global search method. The proposed methodology can systematically generate a trajectory out of the stability region of a local minimum and find another local minimum. Another popular method for optimization is the Design and Analysis of Computer 1.

(8) Experiments (DACE) that was described in [12]. In DACE, a stochastic normal process is used to model deterministic responses. It constructs a kriging approximation model based on data from a computer experiment, and uses this approximation model as a surrogate for the computer model. In this article, we select Trajectory-based methodology as the global search method. The method employs the decomposition points to reach another local minimums. We need to prespecify two implement parameters, but it might choose inappropriate ones. It costs too much to search for the decomposition point and the decomposition points used to a more computational cost of functional evaluation. In order to solve the problem of high cost, we propose a method that combines surrogate surface with trajectory-based methodology. The main purpose of the algorithm is to apply some experimental points to construct the surrogate surface. We employ the surrogate surface to select the parameter. Finally, we use the surrogate construction to reduce the computational cost of functional evaluation.. 2. Trajectory Based Methodology. Global optimization problems can be described as minimizing y = f (x) subject to x ∈ Rn , where f (x) is supposed to be a bounded below smooth function so that the global minimum exists and its numbers are finite. Chiang and Chu (1996) proposed the TB methodology to identify all local extremes sequentially, and the outlines of this algorithm is shown in Table 1. Here we introduce the details of this algorithm in the following subsections.. 2.1. Computation of local minimum. In TB Methodology, the first step is implemented by integrating the negative quasigradient system to approach a local minimum. The negative gradient vector field related to the objective function f (x) is defined as −gradB f (x) = −B(x)−1 ∇f (x),. 2. (2.2).

(9) Table 1: Algorithm of Trajectory Based Methodology (1)Reach a local minimum. (2)Repeat until the stopping criteria. (2.1)Depart from the local minimum and pass through a decomposition point. (2.2)Approach another local minimum by going to the opposite direction of the unstable manifold of the decomposition point.. where B(x) is a positive definite matrix. The method uses numerical integration of the negative quasi-gradient vector field −gradB f (x) with initial points until it reaches the first local minimum. In this step, four descent methods can be used for seeking the local minimums, 1. 2. 3. 4.. If If If If. B(x) = I, it is the steepest descent method; B(x) = 52 f (x), it is Newton method; B(x) = adj(52 f (x))−1 , it is Branin’s method; B(x) = (1 + || 52 f (x)||)I, it is Treccani’s method.. After identifying a local minimum, the second step of TB methodology is to move along with the direction of the eigenvector of the Jacobian matrix evaluated at local minimum associated with the largest eigenvalue. The method of escaping from a local minimum will be explained in the next subsection.. 2.2. Computation of Decomposition point. In the begining of this subsection, we introduce the definition of the decomposition point. The decomposition point is the type-I equilibrium point which has one eigenvalue with positive real part. In the second step of TB Methodology, we move from a local minimum to a decomposition point. For the details of the equilibrium point, please refer to Chiang and Chu (1996). In terms of how to search the decomposition point is shown as follows. Assume x is a equilibrium point of −gradB f (x) and λ(i) (x) are the eigenvalues of the Hessian matrix ∇2 f (x), for i = 1, · · · , n. Let ui (x) be normalized the eigenvalues 3.

(10) corresponding to λi (x), and vj (x) = uj (x)uTj (x), for j = 1, · · · , n. Next, we define a family of matrices Pk (x) =. k X. vj (x) −. j=1. n X. vj (x). j=k+1. where k = 1, · · · , n. The kth order reflected gradient vector field Dk (x), Dk (x) = Pk (x)∇f (x) There are three stages to reach a decomposition point in this step. We summarize the above stages in Table 2. Table 2: Algorithm of computation of decomposition point (1)Choose starting points nearby local minimum. (2)Repeat the number of starting points. (2.1)Numerically integrate with starting point until kD1 (x)k goes to zero. (2.2)Approach another local minimums by decomposition point.. To implement the steps in Table 2, we need to choose m starting points, qi , i = 1, · · · , m, nearby the local minimum first. Second, we use numerical integration to compute the first order reflected gradient vector field D1 (x) with the starting points qi until kD1 (x)k approaches the equilibrium point. In the last step, we check whether the equilibrium point, the type-I equilibrium point, is found or not. If it is a decomposition point, we will employ the point how to find another local minimum. Above all, the decomposition points computed the unstable eigenvector E u of the Jacobian of −∇f (x). Next, xd ±E u as the initial point involved numerical integration of negative gradient system −gradR f (x) until it approaches the local minimums.. 3. The Optimization Algorithm. In TB methodology, we need to prespecify two implement parameters, the number of starting point, n, and radius of a ball, r, for searching the decomposition point. How4.

(11) ever, it might cost too much searching for the decomposition points especially when functional evaluation is expensive. In Figure 1, procedures for searching decomposition points are demonstrated:(i)The radius is smaller than convergent radius, r=0.2, n=4, and the number of step=8; (ii)The radius is smaller than convergent radius, r=0.2, n=4, and the number of step=1;(iii)The radius is wider than convergent radius, r=0.35, n=4, and the number of step=8;(iv)The radius is wider than convergent radius, r=0.35, n=4, and the number of step=1. We can see that when r is too small or too big, we might fail obtaining those decomposition points. For sake of saving the computational cost, we propose a surrogate-assisted method for seeking decomposition point. The basic idea is that we build a surrogate surface which is close to the true surface, and then implement the searching step on this surrogate instead of true surface. After we identify one decomposition point in surrogate with a particular r and n, we can implement the procedure of searching decomposition point with the true function values according to (r, n).. Figure 1: Compare with the radius and convergent radius.. 3.1. Surrogate Construction. In this section, we discuss the surrogate construction approach in this section. The goal is to construct surrogate which is close to the true surface which is then be 5.

(12) replaced with the surrogate surface for searching for the decomposition point. We give an illustrative outline of this stage in Table 3. Table 3: The outline of surrogate construction (1)Generate grid through the experimental region. (2)Choose the initial experimental points. (3)Repeat until stopping criteria. (3.1)Evaluate the response value. (3.2)Choose the next experimental point according to the surrogate surface. (3.3)Construct the surrogate surface. (4)Approach to the decomposition point.. 3.1.1. Generate Grid and Choose Initial Experimental Points. Above all, we defined a grid P for the experimental region. We create a n-dimensional grid which contains pi points, for i = 1, · · · , n, Thus, it has N = p1 p2 · · · pn grid points. For operational convenience, we connect the grid point to mold a long vector with length N . Taking a 2-dimensional grid for an example, the grid points are marked as Pi1,i2 , for i1 = 1, · · · , p1 and i2 = 1, · · · , p2 and we re-permute the points to form a p1 p2 -by-1 vector [P11 , P12 , · · · , Pp1 ,p2 ]T . Assume that object function is unknown, so we need to choose the initial experimental point from P to get some information. Thus, the initial points Pinit are selected from uniform design, and Pinit ⊂ P. 3.1.2. Construct Surrogate. There are many methods to construct the surrogate surface. Here the Kriging method is employed for surrogate construction in this these. • Kriging Method Assumed that we have observed yi = f (xi ) for i = 1, · · · , n. On the basis of this 6.

(13) information, it is required to construct a global approximation fˆ of f , i.e to want to fˆ(xi ) = f (xi ), for i = 1, · · · , n. In the Kriging Method, the surrogate model is supposed to be: y = a(x)T β + Z(x). (3.3). where a(x) is a q × 1 vector of the q chosen regression function, β is a q × 1 vector of coefficients regression, and Z(x) is a Gaussian stochastic process with µ(x) = 0 and a covariance function c(s, t) = σ 2 rθ (s, t). We assumed that a is a known function, β is an unknown vector, and rθ (s, t) is related to distance between s and t. For instance, a Gauss correlation function is rθ (s, t) = exp(−θj (sj − tj )2 ). (3.4). We give the set S of design sites, denoting by S = {s1 , s2 , · · · , sn }, and define F = {y(s1 ), · · · , y(sn )}T to be a response vector, and regard it as the linear predictor: yˆ(x) = c(x)T F. (3.5). The error is yˆ(x) − y(x) = c(x)T F − y(x). Under this condition, the best linear unbiased predictor (BLUP) is obtained by choosing c(x) to minimize: M SE[ˆ y (x)] = E[(c(x)T F − y(x))2 ]. (3.6). subject to keep the predictor unbiased F T c(x) = f (x). (3.7). Assume that R(θ) is a symmetric n × n matrix [rθ (xi , xj )], A is a n × q matrix [aj (xi )], and r(x; θ) = [rθ (x1 , x), · · · , rθ (xn , x)]T is a n × n matrix of stochastic process correlations between the experimental points. Thus, the surrogate function constructed by the Kriging Method is as: ˆ yˆ(x) = a(x)T βˆ + r(x; θ)T R−1 (θ)(F − Aβ). (3.8). where βˆ = (AT R(θ)−1 A)−1 A0 R(θ)−1 F is the least squares estimate of β. Lastly, the unknown parameter (σ 2 , θ) are estimated by the maximum likelihood method. The 7.

(14) maximum likelihood estimate of σ 2 is σ ˆ 2 (θ) =. 1 ˆ 0 R(θ)−1 [y − Aβ(θ)] ˆ [y − Aβ(θ)] n. (3.9). ˆ it turns out that one must minimize To compute θ, n log σ ˆ 2 (θ) + log det R(θ). (3.10). as a function of θ. 3.1.3. Choose the Next Experimental Points. In this stage, we will demonstrate how to choose the next experimental point since it is a must to generate the next experimental point to update the surrogate surface. Two approaches are suggested here.. • MSE Approach In the approach, the primany purpose is to add the point which is the worst estimation on the surrogate surface. Under the condition the mean squared error of the predictor (3.6) is ϕ(x) = E[(ˆ y (x) − y(x))2 ] = σ 2 (1 + c(x)T Rc(x) − 2c(x)T r(x; θ)) = σ 2 (1 + v T (AT R−1 A)−1 v − r(x; θ)T R−1 r(x; θ)). (3.11). ˆ 0 R(θ)−1 [y − Aβ(θ)]. ˆ Above where v = AT R−1 r(x; θ) − a(x) and σ ˆ 2 (θ) = n1 [y − Aβ(θ)] all, we compute ϕ(x) in all grid points and choose the point of the maximum evaluation to enter for constructing the surrogate surface. We keep seeking a point to construct the surrogate surface until it reaches stopping criteria.. • Maximin Approach Assume that x1 , · · · , xn ∈ G, where G is the total grid. We note xi in the row i of the design matrix X = (xik ) ∈ Mn×2 , where M contain n − m experimental points. 8.

(15) and m new experimental points. Let dij (X) = ||xi − xj ||2 = [. p X. (xik − xjk )2 ]1/2. k=1. Then X 0 ∈ G is a maximin distance design in G if min dij (X 0 ) ≥ min dij (X) i>j. i>j. (3.12). for all X ∈ G. Then X 0 is a maximin design. We select the m experimental points from (3.12) and construct the surrogate surface.. 3.2. Surrogate Assisted Trajectory Based Methodology. It can reduce the computation cost of function evaluation for TB methodology in searching for decomposition point when it cooperates with the surrogate assisted approach. We get an illustrative outline in the Table 4. Table 4: The outline of choosing the parameter (1)Construct surrogate. (2)Choose the initial radius r and the number of point n. (3)Repeat until finding the decomposition point at least on the experimental region. (3.1)Approach the decomposition points on the surrogate surface. (3.2)Define ri+1 = cr1 + αβ, for i = 1, · · · , α, β ∈ R+ , c ∈ Z+ .. Overall, we construct a local surrogate surface in the experimental region. After approaching the local minimum, we give the initial radius and the number of point by the starting point qi that searches the decomposition point on the surrogate surface. If no decomposition point is found, the radius is needed to be expanded until a success for searching the decomposition point. Finally, we can use the parameter to put this method to an end. By so doing, it can be ensured to work by TB methodology and reduce the computational cost of function evaluation. We can. 9.

(16) extend the ideal to reduce the more computational cost in order to assure of searching the true decomposition point of the true function be found. The main method confirms the starting points qi whether the decomposition point can be found by the surrogate surface. If certain point can’t approach the decomposition point, we will disperse the ineffective point in an attempt to approach the decomposition point. The approach can reduce the ineffective of the computational cost, and is called as trajectory based with surrogate assisted parameter tuning approach (TB(SAPTA)). We assumed that the surrogate surface converges to object function in experimental region and we can replace object function with surrogate surface. There are five methods for constructing the surrogate surface to search the decomposition point in Table 5. The method is called as Surrogate assisted trajectory based methodology (SATB).. Table 5: The outline of Algorithm (1)Choose the initial point. (2)Reach a local minimum. (3)Repeat until the stopping criteria. (3.1)Construct the surrogate surface by moving window. (3.2)Depart form the local minimum and move along a decomposition point by surrogate surface. (3.3)Approach another local minimum by TB.. • Surrogate construction in total experimental region (SATB) We decide that the number of experimental point to construct a surrogate surface in the experimental region, then the surrogate surface converges to the object function. After that, we look for the decomposition points completely on the surrogate surface, the process of searching local minimums comes back to the true function. On the other hand, we just replace true decomposition point by surrogate decomposition point when the method doesn’t approach any decomposition point or local 10.

(17) minimum namely, stopping criteria.. • Surrogate construction add the path information (SATBI) In this stage, we employ the path of every step which searches the local minimum, and the points added to surrogate construction. It can update the surrogate surface in every step. The main purpose is understand the relationship between decomposition point and local minimum. Every q points choose one point in every step in the point selection, and we delete the points if their distance is greater than d. Then, p percent of grid point adds to construct the surrogate surface from maximin design or MSE design.. • Window moving (SATBW) We consider that the decomposition point neighbors are on local minimums, so we can use the subsurrogate surface to approach the decomposition point. First, the initial point is used to look for a local minimum, and the center of local minimum opens a window. The boundary size is q percent of the experimental region. If the window boundary size exceeds the experimental region, we make the experimental region as window boundary size. Therefore, the window is not really a square. To add p percent of grid point selected by using maximin design or MSE design, the subsurrogate surface is constructed. While we are searching for a local minimum with every step, we need to construct subsurrogate surface and use individual subsurrogate surface to approach the decomposition point. Though, it can’t search for the decomposition point on the subsurrogate surface, the window boundary size is to be expanded until the decomposition point is successfully found. However, the experimental point is the initial experimental point and a little points from maximin design or MSE design are to construct the subsurrogate surface. The experimental points update the surrogate construction.. • Window moving add the path information (SATBWI) Subsurrogate surface is constructed by second method and third method. We open. 11.

(18) a subregion by the center of local local minimum and add the path points in this subregion. After, we delete the points closely and p percent of grid point added from maximin design or MSE design. The experimental points are employed in constructing the subsurrogate surface until the decomposition point is found.. • Block method (SATBB) TB’s experimental region is Rn , are we require the defined experimental region to construct the surrogate surface. If the region is too wide, the surrogate surface may not satisfy with the assumption. Therefore, we divided the experimental region into n2 subregions. Next, we adopt the first to fourth methods to construct the subsurrogate surface on the individual subregion.. 4. Numerical Experiments. In this section, to illustrate the execution of these methods, different types of testing functions are studied. Then, we introduce two-dimensional and three-dimensional model problems and demonstrate how they can be solved.. Example 1 We consider the M¨ uller Brown function, which has been studied in [9]. The corresponding optimal problem is described by min fM B (x, y) =. 4 X. h 2 2 i Ai exp ai x − x0i + bi x − x0i y − yi0 + ci y − yi0 ,. i=1. where A = (−200, −100, −170, 15), a = (−1, −1, −6.5, 0.7), b = (0, 0, 11, 0.6), c = (−10, −10, −6.5, 0.7), x0 = (1.0, −0.5, −1), y 0 = (0, 0.5, 1.5, 1), s.t.. −1.5 ≤ x ≤ −1 and − 0.5 ≤ y ≤ 2.. We use the surrogate surface to choose the parameters, for instance, radius and the number of points. Above all, we need to construct the surrogate surface. We choose 12.

(19) the initial grid: 13 18 13 18 G = (x, y)|x ∈ −1.5, −1 , −1 , · · · , 1 , y ∈ −0.5, − , − , · · · , 2 46 46 46 46 The grid contains 576(24×24) points. There are three local minimums at (−0.558, 1.442), (0.623, 0.028) and (−0.05, 0.467) in true response function. The local minimum is marked by 4 symbols in Figure 2. Here, 24 experimental points are selected in terms of the two-factor uniform design with 24 levels for each factor and adding 120 experimental points from maximin design. Hence, we use 144 initial experimental points to construct the surrogate surface.. Figure 2: The true response surface in M¨ uller Brown function. We construct the surrogate surface by kriging method and choosing the parameter. The initial radius r is four percent of the experimental boundary and the starting points qi has five points. We adopt the radius and starting points to search for the decomposition point, if no decomposition point is found. Then we update the radius, ri+1 = cr1 for c = 2, · · · , until the decomposition point is found. Lastly, we use the final radius to look for the decomposition point. We choose the number of starting point: n = {5, 10, 15, 20}. Hence, the code chooses the final radius, r = 0.4, which can reach the decomposition point. Then, compare our algorithm with TB. The performances are summarized in Table 6 to 8. We have the following 13.

(20) notations: • NPS : the number of experimental point in the surrogate construction; • NLP : the number of path point of finding the local minimum; • NDP : the number of path point of finding the decomposition point; • TP : the total number of functional evaluation,ie TP=NPS+NLP+NDP; • R : the radius; • NS : the number of starting point. • IS : the inclination of starting point in spherical coordinate system. • NC : the number of circle in the starting point. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {0.4, 0.5, 0.6} and the set of starting point number : n = {3, 5, 10, 15}. Thus, the best experiment result receives 1074 points which use the computational cost of function evaluation in TB. We show the best result in Table 6 with r = 0.6, and the number of starting point, n = {3, 5, 10, 15}. In Table 6 and 7, the result of TB(SAPTA) experiment is better than TB. Here the window size is the experimental region. Compared to the experiment result of TB, the experiment result of SATB is better shown in Table 8. In Figure 10 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figures the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition point of starting point are marked by ∗, ×, · , +, , and 4, respectively. Different parameters are set out in Appendix.. Example 2 Above all, we consider a function which has been studied in [4]. The corresponding optimal problem is described by 2 min fE (x, y) = −(x2 + y 2 − 1)2 − ((2x2 − 1)2 + (2y 2 − 1)2 − )2 3 s.t. −1 ≤ x ≤ 1 and − 1 ≤ y ≤ 1 14.

(21) Table 6: Performance summary of TB in M¨ uller Brown function. R. 0.6. NS. NLP. NDP. TP. 3. 330. 744. 1074. 5. 328. 6234. 6562. 10. 330. 7730. 8060. 15. 384. 8663. 9047. Table 7: Performance summary of TB(SAPTA) in M¨ uller Brown function. R. 0.6. NS NPS. NLP. NDP. TP. 3. 144. 330. 525. 999. 5. 144. 328. 2001. 2473. 10. 144. 330. 3239. 3713. 15. 144. 384. 3336. 3864. We employ the surrogate surface to select the parameters, for instance, radius and the number of points. First, we need to construct the surrogate surface. We choose the initial grid: 19 21 19 21 G = (x, y)|x ∈ −1, − , − , · · · , 1 , y ∈ −1, − , − , · · · , 1 23 23 23 23 The grid include 576(24 × 24) points. There are five local minimums at (0, 0), (0.707, 0.707), (0.707, −0.707), (−0.707, −0.707) and (−0.707, 0.707) in true response function and the local minimum is marked by 4 symbols in Figure 3. Here, 24 experimental points is selected in terms of the two-factor uniform design with 24 levels for each factor and adding 120 experimental points from maximin design. Hence, we use 144 initial experimental points to construct the surrogate surface. We construct the surrogate surface by kriging method and selecting the parameter. The radius r is five percent of the experimental boundary. We choose the number of starting point: n = {10, 15, 20, 30}. We employ the radius and starting points to search for the decomposition point, if no decomposition point were found. Then we update the radius, ri+1 = cr1 for c = 2, · · · , until searching the decomposition 15.

(22) Table 8: Performance summary of SATB in M¨ uller Brown function. R. 0.6. NS NPS. NLP. TP. 3. 144. 328. 472. 5. 144. 322. 466. 10. 144. 324. 468. 15. 144. 326. 470. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) with respect to NS are 768, 5075, 6942 and 11157.. Figure 3: The true response surface in the example 2. point. The initial radius can find the decomposition point. Then, our algorithm compare with TB. The performances are summarized in Table 9 to 11. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {0.1, 0.2, 0.3, 0.5} and the set of starting point number : n = {10, 15, 20, 30}. Thus, the best experiment result is 3921 points which use the computational cost of function evaluation in TB. We show the best result in Table 9 with r = 0.5, and the number of starting point, n = {10, 15, 20, 30}. Though TB experiment result is better than TB(SAPTA) in Table 9 and 10, using the point by searching the decomposition point has been reduce in TB(SAPTA) method. For 16.

(23) all we adopt some points to build the surrogate surface. Then we use the surrogate surface to check the starting point used for searching for the decomposition point. The method truly reduced the cost of functional evaluation. Here the window size is the experimental region. Table 11 shows that the experiment result of SATB is better when compared to the experiment result of TB. In Figure 11 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figures the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition pointof starting point are marked by ∗, ×, · , +, , and 4, respectively. Different parameters are set out in Appendix. Table 9: Performance summary of TB in example 2. R. 0.5. NS. NLP. NDP. TP. 10. 601. 3320. 3921. 15. 567. 5496. 6063. 20. 601. 6640. 7241. 30. 617. 10992 11609. Table 10: Performance summary of TB(SAPTA) in example 2. R. 0.5. NS NPS. NLP. NDP. TP. 10. 144. 601. 3296. 4041. 15. 144. 567. 5462. 6173. 20. 144. 601. 6592. 7337. 30. 144. 617. 10924 11685. 17.

(24) Table 11: Performance summary of SATB in example 2. R. 0.5. NS NPS. NLP. TP. 10. 144. 639. 783. 15. 144. 623. 767. 10. 144. 669. 813. 15. 144. 641. 785. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) with respect to NS are 3349, 5513, 6680 and 11030.. Example 3 Above all, we consider a function, Hump Camel-Back function, which has been studied in [4]. The corresponding optimal problem is described by min fH (x, y) = ax2 + bx4 + cx6 − xy + dy 2 + ey 4 where a = 4, b = −2.1, c = 1/3, d = −4, e = 4. s.t.. −2 ≤ x ≤ 2 and − 2 ≤ y ≤ 2. We use the surrogate surface to choose the parameters, for instance, radius and the number of points. Above all, it is needed to construct the surrogate surface. We choose the initial grid: 19 15 19 15 G = (x, y)|x ∈ −2, −1 , −1 , · · · , 2 , y ∈ −2, −1 , − , · · · , 2 23 23 23 23 The grid contains 576(24×24) points. There are six local minimums at (0.090, 0.713), (1.704, 0.796), (−1.607, 0.569), (−1.704, −0.796), (−0.090, −0.713) and (1.607, −0.569) in true response function and the local minimum is marked by 4 symbols in Figure 4. Here, 24 experimental points are selected in terms of the two-factor uniform design with 24 levels for each factor and adding 100 experimental points from maximin design. Hence, we use 124 initial experimental points to construct the surrogate surface. We construct the surrogate surface by kriging method and choosing the parameter. 18.

(25) Figure 4: The true response surface in Hump camel-back function. The initial radius r is two point five percent of the experimental boundary and the starting points qi has three points. We employ the radius and starting points to search for the decomposition point, if no decomposition point were found. Then we update the radius, ri+1 = cr1 for c = 2, · · · , until searching the decomposition point. The initial radius can find the decomposition point. Then, compare our algorithm with TB. The performances are summarized in Table 12 to 14. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {0.1, 0.2, 0.3, 0.5} and the set of starting point number : n = {3, 5, 10, 15, 20}. Thus, the best experiment result is 3354 points which use the computational cost of function evaluation in TB. We show the best result in Table 9 with r = 0.5, and the number of starting point, n = {10, 15, 20, 30}. Table 12 and 13 show that the result of TB(SAPTA) experiment is better than that of TB, but the latter has ten experimental points more than TB in radius r = 0.1 and the number of starting point n = 3. Other TB(SAPTA) experimental result is better than TB. Here the window size is the experimental region. Comparing two, the experiment result of SATB is better than that of TB shown in Table 14. In Figure 12 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the 19.

(26) Table 12: Performance summary of TB in Hump Camel-Back function. R. NS. NLP. NDP. TP. 3. 744. 2610. 3354. 5. 795. 4814. 5609. 10. 795. 9628. 10423. 15. 791. 13937 14728. 20. 793. 18452 19245. 0.1. three sub-figures the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition point of starting point are marked by ∗, ×, · , +, , and 4, respectively. Different parameters are set out in Appendix. Table 13: Performance summary of TB(SAPTA) in Hump Camel-Back function. R. 0.1. NS NPS. NLP. NDP. TP. 3. 124. 744. 2496. 3364. 5. 124. 795. 4556. 5475. 10. 124. 795. 9112. 10031. 15. 124. 791. 13327 14242. 20. 124. 793. 17552 18469. Example 4 Now we consider the Griewank function, which has been studied in [10], and optimization problem is defined as: min fG (x, y) = a + b1 x2 + b2 y 2 − d cos(c1 x) cos(c2 y) 1 1 where a = 1, b1 = b2 = , c1 = 1, c2 = √ , d = 1. 200 2 s.t. −10 ≤ x ≤ 10 and − 10 ≤ y ≤ 10. 20.

(27) Table 14: Performance summary of SATB in Hump Camel-Back function. R. 0.1. NS NPS. NLP. TP. 3. 124. 722. 846. 5. 124. 765. 889. 10. 124. 733. 857. 15. 124. 777. 901. 20. 124. 779. 903. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) with respect to NS are 2967, 3203, 11229 and 13302.. We use the surrogate surface to choose the parameters, for instance, radius and the number of points. Above all, we need to construct the surrogate surface. We choose the initial grid: 1 2 1 2 G = (x, y)|x ∈ −10, −9 , −8 , · · · , 10 , y ∈ −10, −9 , −8 , · · · , 10 3 3 3 3 The grid contains 900(30×30) points. There are seventeen local minimums at (0, 0), (3.11, 4.36), (−3.11, 4.36), (−3.11, −4.36), (3.11, −4.36), (6.22, 8.71), (0, 8.71), (−6.22, 8.71), (−6.22, 0), (−6.22, −8.71), (6.22, 0), (0, −8.71), (6.22, −8.71), (−9.33, 4.36), (−9.33, −4.36), (9.33, 4.36) and (9.33, −4.36) in true response function and the local minimum is marked by 4 symbols in Figure 5. Here, 30 experimental points are selected in terms of the two-factor uniform design with 30 levels for each factor and adding 150 experimental points from maximin design. Hence, we use 180 initial experimental points to construct the surrogate surface. In this function, the methods base on the preceding examples including the surrogate construction and the radius of tuning. We set up the number of starting point: n = {5, 10, 15, 20}. Then, compare our algorithm with TB. The performances are summarized in Table 15 to 17. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {1, 1.25, 1.5} and the set of starting point number : n = {5, 10, 15, 20}. 21.

(28) Figure 5: The true response surface in Griewank function. Thus, the best experiment result receives 16778 points which uses the computational cost of function evaluation in TB. We show the best result in Table 15 with r = 1.5, and the number of starting point, n = {5, 10, 15, 20}. Table 15 and 16 show that the experiment result of TB(SAPTA) is better than that of TB. Here the window size is the experimental region. Comparing two, the experiment result of SATB is better than that of TB shown in Table 17. In Figure 13 we display the example of Table 15: Performance summary of TB in Griewank function. R. 1.5. NS. NLP. NDP. TP. 5. 6160. 10618 16778. 10. 6142. 19331 25473. 15. 6130. 27357 33487. 20. 6130. 58265 64395. function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figures the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition pointof starting point are marked by ∗, ×, · , +, , 22.

(29) and 4, respectively. Different parameters are set out in Appendix. Table 16: Performance summary of TB(SAPTA) in Griewank function. R. 1.5. NS NPS. NLP. NDP. TP 14638. 5. 180. 6160. 8298. 10. 180. 6142. 16596 22918. 15. 180. 6130. 26398 32708. 20. 180. 6130. 39228 45538. Table 17: Performance summary of SATB in Griewank function. R. 1.5. NS. NPS. NLP. TP. 5. 180. 6103. 6283. 10. 180. 6098. 6278. 15. 180. 6087. 6267. 20. 180. 6094. 6274. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) with respect to NS are 14436, 28871, 33116 and 67864.. Example 5 We consider the Ackley function, which has been studied in [1, 13]. The corresponding optimal problem is described by ( " r # ) 1 1 2 1 min fA (x, y) = −a exp −b (x + y 2 ) − exp (cos cx + cos cy) + a exp (1) + d , f n n where a = 20, b = 0.2, c = 2π, d = 5.7, f = 0.8, n = 2. s.t.. −1.5 ≤ x ≤ 1.5 and − 1.5 ≤ y ≤ 1.5.. We employ the surrogate surface to choose the parameters, for instance, radius and the number of points. Above all, we needed to construct the surrogate surface. We. 23.

(30) choose the initial grid: 6 9 6 9 G = (x, y)|x ∈ −1.5, −1 , −1 , · · · , 1.5 , y ∈ −1.5, −1 , −1 , · · · , 2 24 24 24 24 The grid contains 625(25×25) points. There are nine local minimums at (0.97, 0.97), (0, 0.95), (0.95, 0), (−0.97, 0.97), (0, 0), (0, −0.95), (0.97, −0.97), (−0.95, 0), (−0.95, 0) and (−0.97, −0.97) in true response function and the local minimum marked by 4 symbols in Figure 6. Here, 25 experimental points are selected in terms of the two-factor uniform design with 25 levels for each factor and adding 120 experimental points from maximin design. Hence, we use 144 initial experimental points to construct the surrogate surface.. Figure 6: The true response surface in Ackley function. This function method is based on the preceding examples including the surrogate construction and choosing of radius. The code select the number of starting point: n = {5, 10, 15, 20} and the radius, r = 0.3, which can reach the decomposition point. Then, compare our algorithm with TB. The performances are summarized in Table 18 to 20. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {0.1, 0.2, 0.3, 0.5} and the set of starting point number : n = {5, 10, 15, 20}. Thus, the best experiment result receives 4009 points which uses 24.

(31) the computational cost of function evaluation in TB. The best result is shown in Table 15 with r = 0.5, and the number of starting point, n = {5, 10, 15, 20}. Table 15 and 16 show that the experiment result of TB(SAPTA) is better than that of TB. Here the window size is the experimental region. Comparing two, the experiment result of SATB is better than that of TB shown in Table 20. In Figure Table 18: Performance summary of TB in Ackley function R. 0.5. NS. NLP. NDP. TP. 5. 1606. 2403. 4009. 10. 1645. 4806. 6451. 15. 1558. 7487. 9075. 20. 1638. 9612. 11250. 14 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figures the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition point of starting point are marked by ∗, ×, · , +, , and 4, respectively. Different parameters are set out in Appendix. Table 19: Performance summary of TB(SAPTA) in Ackley function R. 0.5. NS NPS. NLP. NDP. TP. 5. 225. 1606. 1985. 3812. 10. 225. 1645. 3914. 5784. 15. 225. 1588. 8700. 7513. 20. 225. 1638. 7883. 9696. 25.

(32) Table 20: Performance summary of SATB in Ackley function R. 0.5. NS. NPS. NLP. TP. 5. 225. 1603. 1828. 10. 225. 1597. 1822. 15. 225. 1603. 1828. 20. 225. 1474. 1699. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) with respect to NS are 3532, 7632, 21729 and 19968.. Example 6 Above all, we consider a Schwefel function, which has been studied in [8]. The corresponding optimal problem is described by min fS (x, y) = aD −. D X. p xi sin ( |xi |). i=1. where a = 418.9829, D = 2. s.t.. −500 ≤ x ≤ 500 and − 500 ≤ y ≤ 500. There are forty-nine local minimums displayed in Table 21, in true response function and the local minimum marked by 4 symbols in Figure 7.. In this example, we. provide some different methods, since the experimental region is wider. All above, we divided the experimental region into 32 (= 9) subregions and displayed in Table 22. Then we can employ the methods to construct the subregion in last section. In Table 21: The experimental subregion G31. G32. G33. G21. G22. G23. G11. G12. G13. order to construct the subsurrogate surface, we are required to define the grid. We. 26.

(33) Figure 7: The true response surface in Schwefel function.. choose the initial subgrid: b × (i − 1) + 20 b × (i − 1) + 40 bi b × (i − 1) , −a + , −a + , · · · , −a + G xi = x ∈ −a + 3 3 3 3 and b × (j − 1) b × (j − 1) + 20 b × (j − 1) + 40 bj G yj = y ∈ −a + , −a + , −a + , · · · , −a + 3 3 3 3 where a = 500, b = 1000, i, j = 1, 2, 3. The grid contains 2500 (50 × 50) points. Here, 50 experimental points are selected in terms of Latin hypercube design with 50 levels for each factor and adding 350 experimental points from maximin design. Hence, we use 400 initial experimental points to construct the surrogate surface. It is based on the preceding examples including the surrogate construction and the radius of choose. The code selects the number of starting point: n = 5 and the radius, r = 10, which can reach the decomposition point. Then, compare our algorithm with TB. The performances are summarized in Table 23. We choose the parameters that gives the best experiment result in TB. The set of radius : r = {20, 30, 40, 50} and the set of starting point number : n = {5, 10, 15, 20}. Thus, the best experiment result is 17139 points which uses the computational cost of function evaluation in TB. The best result is shown in Table 27.

(34) Table 22: 49 local minimums in Schwefel function. (65.55, -25.88). (203.81, -25.88). (65.55, 5.24). (5.24, -25.88). (420.88, 5.24). (203.81, 5.24). (65.55, 65.55). (5.24, 5.24). (203.81, -124.83). (5.24, -124.83). (65.55, -302.51). (420.97, 5.24). (65.55, 203.81). (5.24, 65.55). (-25.88 ,5.24). (-124.83, -25.88). (-25.88, -124.83). (203.81, -302.52). (5.24, -302.52). (420.97, 65.55). (65.55, 420.97). (5.24, 203.81). (-25.88, 65.55). (-124.83, 5.24). (-25.88, -302.52). (420.97, 203.81). (-124.83, -124.83) (420.97, -302.52) (5.24, 420.97). (-25.88, 203.81). (-124.83, 65.55). (-302.52, 5.24). (-124.83, -302.52). (420.97, 420.97). (-25.88, 420.97). (-124.83, 203.81). (-302.52, -302.52) (-124.83, 420.97) (-302.52, 203.81) (-302.52, 420.97) (420.97, -124.83). (203.81, 203.81). (-302.52, -25.88). (203.81, 420.97). (-302.52, -124.83). (-302.52, 65.55). (65.55, -124.83). (-25.88, -25.88). (420.97, 5.24). 23 with r = 30, and the number of starting point, n = {5, 10, 15, 20}. The experiment result of TB(SAPTA) is better than that of TB. Here the window size is the experimental region. Comparing two, the experiment result of SATBB is better than that of TB. Besides, the window size can be changed. The center of individual local minimum opens a subwindow which occupies seventy percent of the experimental region. If the subwindow exceeds the experimental region, the boundary is the experimental region. In selecting experimental point, we have two methods for constructing the surrogate surface; one is from maximin design and path points and the other is from maximin design. First, we use the first method to select experimental point. In first subsurrogate surface, 50 experimental points were selected in terms of Latin hypercube design with 50 levels for each factor and adding 100 experimental points from maximin design. In the path points, every ten path point choose one as the experimental 28.

(35) Table 23: Performance summary of TB, TB(SAPTA) and SATBB in Schwefel function. method. R. NS. NLP. NDP. TP. TB. 30. 5. 12852. 4287. 17139. method. R. NS. NPS. NLP. NDP. TP. 5. 400. 12407. 4196. 17003. TB(SAPTA) 30 method. R. NS. NPS. NLP. TP. SATBB. 30. 5. 3150. 1140. 14550. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) is 50867 in SATBB.. point. We will delete the point if the distance of experimental point is greater than 1 cm. Hence, we use initial experimental points to construct the first subsurrogate surface. In other subsurrogate surface, there are three cases of adding the number of point which is used to construct the subsurrogate surface. If the number of point is greater than 200, we will add ten points from maximin design. On the other hand, we add 100 points from maximin design when the number of points is less than 30. In another case, we add 38 points from maximin design. We employ the method to construct the subsurrogate surface by which we search for the decomposition point. Hence, compare the algorithm with TB. The performances are summarized in Table 24. Next, we discuss second method of selecting experimental point. In the first subsurrogate surface, the method is the same as preceding paragraph. In another subsurrogate surface, it has two cases of adding the number of points which is used to construct the subsurrogate surface. If the number of adding point is greater than 30, we will add 63 points from maximin design. Otherwise we add 100 points from maximin design. Finally, we use the method to finish our algorithm. The performances are summarized in Table 24.. 29.

(36) Table 24: Performance summary of TB, SATBWI and SATBW in Schwefel function method. R. NS. NLP. NDP. TP. TB. 30. 5. 12597. 7563. 20160. method. R. NS. NPS. NLP. TP. SATBWI. 30. 5. 1485. 14743 16228. method. R. NS. NPS. NLP. SATBW. 30. 5. 3175. 12298 15424. TP. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) are 201411 in SATBWI and 110535 in SATBW.. In table 24, we detected that the point which uses the computational cost of function evaluation is obviously less than TB in our two methods. In this case, our algorithm is greater than TB in any method. In Figure 15 we display the example of functionsolving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figure the initial point, the decomposition point, the point of path, the starting point, the local minimum, and the decomposition point of starting point are marked by ∗, ×, · , +, , and 4, respectively. Different parameters are set out in Appendix.. Example 7 In this example, we discuss a high dimensional function, three dimensional Ackley function, which has been studied in [1, 13]. The corresponding optimal problem is defined as: # 1 2 −a exp −b (x + y 2 + z 2 ) n 1 − exp (cos cx + cos cy + cos cz) + a exp (1) + d , n where a = 20, b = 0.2, c = 2π, d = 5.7, f = 0.8, n = 2.. 1 min fA (x, y, z) = f. s.t.. (. ". r. (4.13). −1.5 ≤ x ≤ 1.5, −1.5 ≤ y ≤ 1.5 and − 1.5 ≤ z ≤ 1.5.. 30.

(37) We use the surrogate surface to choose the parameters, for example, radius and the number of points. First, we need to construct the surrogate surface. We choose the initial grid: G=. (x, y, z)|x, y, z ∈. 3 6 −1.5, −1 , −1 , · · · , 1.5 18 18. The grid contains 5832(18×18×18) points. There are twenty-seven local minimums in true response function in Figure 8.. Figure 8: The true response surface in 3D Ackley function. We build the surrogate surface by the Kriging method and choose the parameters by the surrogate surface. Because the example is a high dimensional function, the center of individual local minimum opens a subwindow as the experimental region which occupies sixty percent of the experimental region. If the window boundary exceeds in the experimental region, the window boundary is the experimental boundary. The steps are based on the preceding examples and we choose that r = 0.5, φ =. π 5. and. n = 7. Here we build a ball by spherical coordinate system as the starting points. In another subsurrogate surface, it has three case of adding the number of points which is used to construct the subsurrogate surface. If the number of adding point is greater than 30, we will add the number of point from maximin design which subtracts the number of adding points from 500. If the number of adding point is 31.

(38) less than 30, we will add 600 point from maximin design. Otherwise we add one point from maximin design. In the surrogate construction, we have other methods to increase the experimental points. The method adding the path points as the experimental points to join the surrogate construction. In the path points, every twenty path point selects one point as the experimental point. We will delete the point if the distance of experimental point is greater than 0.1 mm. Then to select the number of points that are the same in last paragraph. Finally, we use the method to finish our algorithm. The performances are summarized in Table 25. Table 25: Performance summary of TB, TB(SAWIPTA) and SATBWI in 3D Acklry funcrion method. R. NC. IS. NLP. NDP. TB. 0.5. 7. iπ 5. 7926. 31164 39090. method. R. NC. IS. NPS. NLP. NDP. 7. iπ 5. 4828. 7271. 26711 38810. TB(SAWIPTA) 0.5. TP. method. R. NC. IS. NPS. NLP. SATBWI. 0.5. 7. iπ 5. 4979. 13305 18284. TP. TP. † NDP is 27920 by searching the decomposition point on surrogate surface in SATBWI.. In table 25, we detect that the point which uses the computational cost of function evaluation is obviously less than TB in our two methods. The set of radius : r = {0.1, 0.2, 0.3, 0.5} , the set of starting point number : n = {7, 10, 15, 20} and the set of azimuth :. iπ , k. where k = 4, 5, 6 and i = k−1. Thus, the best experiment result. is 39090 points which uses the computational cost of function evaluation in TB, when R is 0.5, IS is. iπ 5. and NC is 7. In this case, our algorithm is greater than TB in any. method. In Figure 16 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figure the initial point, the decomposition point, the point of path and the local minimum are marked by ∗, ×, · and , respectively. 32.

(39) Different parameters are set out in Appendix.. Example 8 We discuss a high dimensional function, three dimensional Hartman function, which has been studied in [6]. The corresponding optimal problem is defined as : ! m n X X min fH (x1 , x2 , x3 ) = − ci exp − aij (xj − pij )2 , i=1. i=1. where x = (x1 , · · · , xn ), pi = (pi1 , · · · , pin ), ai = (ai1 , · · · , ain ). s.t.. −1 ≤ x1 ≤ 1, −1 ≤ x2 ≤ 1 and − 1 ≤ x3 ≤ 1.. Table 26: Parameter for the 3-D Hartman function m = 4, n = 3 i. aij. ci. pij. 1. 3,10,30. 1. 0.3689,0.1170,0.2673. 2 3 4. 0.1,10,35 1.2 3,10,30. 3. 0.4699,0.4387,0.7470 0.1091,0.8732,0.5547. 0.1,10,35 3.2 0.03815,0.5743,0.8828. We use the surrogate surface to select the parameters, for example, radius and the number of points. First, we need to construct the surrogate surface. We choose the initial grid: 1 2 G = (x, y, z)|x, y, z ∈ 0, , , ··· , 1 18 18 The grid contains 5832(18 × 18 × 18) points. There are three local minimums in true response function in Figure 9. We build the surrogate surface by the Kriging method and choose the parameters by the surrogate surface. We employ the experimental region as the window region to surrogate surface. Here, 18 experimental points are selected in terms of the threefactor uniform design with 18 levels for each factor adding 700 experimental points from maximin design. Hence, we use 718 initial experimental points to construct 33.

(40) Figure 9: The true response surface in 3D Hartman function. the surrogate surface. The steps are based on the preceding examples, and then we choose that r = 0.15, φ =. π 5. and n = 5. We build a ball by spherical coordinate. system as the starting points. The performances are summarized in Table 27. Table 27: Performance summary of TB, TB(SAPTA) and SATB in 3D Hartman function method. R. NC. IS. NLP. NDP. TB. 0.15. 5. iπ 5. 235. 11464 11699. method. R. NC. IS. NPS. NLP. NDP. TP. 5. iπ 5. 700. 267. 9807. 10774. TB(SAPTA) 0.15. TP. method. R. NC. IS. NPS. NLP. TP. SATB. 0.15. 5. iπ 5. 750. 1357. 2107. † NDP(S) means the number of points used for searching the decomposition point on surrogate surface and here NDP(S) are 15752 in SATBW.. In table 27, we detected that the point which uses the computational cost of function evaluation is obviously less than TB in our two methods. The set of radius : r = {0.1, 0.15, 0.2, 0.25} , the set of starting point number : n = {3, 5, 10, 15} and 34.

(41) the set of azimuth :. iπ , k. where k = 4, 5, 6 and i = k − 1. Thus, the best experiment. result is 7089 points which use the computational cost of function evaluation in TB, when R is 0.25, IS is. iπ 5. and NC is 3. In this case, our algorithm is greater than. TB in any method. In Figure 18 we display the example of function-solving. The experiment result plotted the true response surface and the surrogate surface are shown in (a), (b) and (c). In the three sub-figure the initial point, the decomposition point, the point of path, the local minimum, and the decomposition point of starting point are marked by ∗, ×, ·, , and 4, respectively. Different parameters are set out in Appendix.. 5. Conclusion. In this thesis, we propose an algorithm that combines surrogate construction and trajectory-based methodology for solving optimization problem. Surrogate construction is first applied to get the trend of true, yet unknown, response surfaces. We then select the useful parameters and the decomposition points from the surrogate surface by trajectory based methodology. Finally, trajectory-based methodology is employed to find out optimal points from the useful parameters and the decomposition points. Our numerical experiments indicate that SATB method does perform well, no matter what type of true function is being modeled. However, to find out the decomposition point is our main concern. If the decomposition point is not successfully being found, we can’t continue with our method. To modify this algorithm to be a sequential procedure may be a potential theme for future studies. That is, we can employ a few starting points to search for the decomposition point. It is expected that this sequential approach will also save computational cost.. 35.

(42) References [1] D. H. Ackley. A connectionist machine for genetic hillclimbing. Kluwer Academic, 1987. [2] G. E. P. Box and K. B. Wilson. On the experimental attainment of optimum conditions. Journal of the Royal Statistical Society. Series B (Methodological), pages 1–45, 1951. [3] E. Canc`es, F. Legoll, M. C. Marinica, K. Minoukadeh, and F. Willaime. Some improvements of the ART method for finding transition pathways on potential energy surfaces. Arxiv preprint arXiv:0806.4354, 2008. [4] H. D. Chiang and C. C. Chu. A systematic search method for obtaining multiple local optimalsolutions of nonlinear programming problems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 43(2):99– 109, 1996. [5] C. Dellago, P. G. Bolhuis, F. S. Csajka, and D. Chandler. Transition path sampling and the calculation of rate constants. The Journal of Chemical Physics, 108:1964, 1998. [6] L. C. W. Dixon and G. P. Szego. The global optimization problem: an introduction. Towards global optimization, 2:1–15, 1978. [7] C. A. Floudas. Deterministic global optimization: theory, methods and applications. Kluwer Academic Pub, 2000. [8] V. S. Gordon and D. Whitley. Serial and parallel genetic algorithms as function optimizers. Citeseer, 1993. [9] K. M¨ uller and L.D. Brown. Location of saddle points and minimum energy paths by a constrained simplex optimization procedure. Theoretical Chemistry Accounts: Theory, Computation, and Modeling (Theoretica Chimica Acta), 53(1):75–93, 1979. 36.

(43) [10] C. Nakazawa, S. Kitagawa, Y. Fukuyama, and H. D. Chiang. A method for searching multiple local optimal solutions of nonlinear optimization problems. In IEEE International Symposium on Circuits and Systems, 2005. ISCAS 2005, pages 4907–4910, 2005. [11] A. Neumaier. Complete search in continuous global optimization and constraint satisfaction. Acta Numerica, 13:271–369, 2004. [12] J. Sacks, W. J. Welch, T. J. Mitchell, and H. P. Wynn. Design and analysis of computer experiments. Statistical science, pages 409–423, 1989. [13] A. Sobester, S. J. Leary, and A. J. Keane. A parallel updating scheme for approximating and optimizing high fidelity computer simulations. Structural and multidisciplinary optimization, 27(5):371–383, 2004. [14] D. J. Wales. Discrete path sampling. Molecular Physics, 100(20):3285–3305, 2002.. 37.

(44) Figure. Figure 10: TB, TB(SAPTA) and SATB in M¨ uller Brown function 38.

(45) Figure 11: TB, TB(SAPTA) and SATB in Example 2. 39.

(46) Figure 12: TB, TB(SAPTA) and SATB in Hump Camel Back function. 40.

(47) Figure 13: TB, TB(SAPTA) and SATB in Griewank function. 41.

(48) Figure 14: TB, TB(SAPTA) and SATB in Ackley Function. 42.

(49) Figure 15: TB, TB(SAPTA) and SATBB in Schwefel function. 43.

(50) Figure 16: TB, TB(SAWIPTA) and SATBWI in 3D Ackley function. 44.

(51) Figure 17: TB, TB(SAPTA) and SATB in 3D Hartman function. 45.

(52) A. Appendix Table 28: Performance summary of TB in M¨ uller Brown function. NS. 3. 5. 15. 20. R=0.4 1250 1693 4030 8676 R=0.5 1300 5201 7401 9511 R=0.6 1074 6562 8060 9047. Table 29: Performance summary of TB(SAPTA) in M¨ uller Brown function. NS. 3. 5. 15. 20. R=0.4 1241 1494 3541 7737 R=0.5 1295 1052 2339 3892 R=0.6. 999. 2473 3713 3864. Table 30: Performance summary of SATB in M¨ uller Brown function. NS. 3. 5. 15. 20. R=0.4 474 470 466 470 R=0.5 472 470 472 470 R=0.6 472 466 468 470. 46.

(53) Table 31: Performance summary of TB in example 2. NS. 10. 15. 20. 30. R=0.1 8479 12484. 16325 24347. R=0.2 7521 18091. 14409 35565. R=0.3 7273 12981. 13873 25321. R=0.5 3921. 7241. 6063. 11609. Table 32: Performance summary of TB(SAPTA) in example 2. NS. 10. 15. 20. 30. R=0.1 8623 12628. 16469 24491. R=0.2 7257 11400. 13737 19038. R=0.3 6061. 7791. 11305 14315. R=0.5 4041. 6173. 7337. 11685. Table 33: Performance summary of SATB in example 2. NS. 10. 15. 20. 30. R=0.1 819 819 795 825 R=0.2 827 833 803 833 R=0.3 823 827 787 829 R=0.5 783 767 813 785. 47.

(54) Table 34: Performance summary of TB in Hump camel-back function. NS. 3. 5. 10. 15. 20. R=0.1 3354 5609 10423 14728 19245 R=0.2 3491 5142. 9372. 15842 18484. R=0.3 3730 4957. 8996. 13675 16945. R=0.5 3893 4130. 7332. 12225 14232. Table 35: Performance summary of TB(SAPTA) in Hump camel-back function. NS. 3. 5. 10. 15. R=0.1 3364 5475 10031 14242. 20 1849. R=0.2 3505 5122. 9208. 13494 18048. R=0.3 3697 4862. 8682. 13372 16403. R=0.5 4013 4230. 7408. 12259 13964. Table 36: Performance summary of SATB in Hump camel-back function. NS. 3. R=0.1 846. 5. 10. 15. 20. 889. 857. 901. 903. R=0.2 962 1014 1018 1020 1016 R=0.3 964 1030 1028 1034 1032 R=0.5 970 1034 1032 1040 1028. 48.

(55) Table 37: Performance summary of TB in Griewank function. NS R=1. 5. 10. 15. 20. 22054 37943. 71576 98869. R=1.25 17047 28050. 45227 76040. R=1.5. 33487 64395. 16778 25473. Table 38: Performance summary of TB(SAPTA) in Griewank function. NS R=1. 5. 10. 15. 20. 21516 36687. 50013 64581. R=1.25 16718 27212. 39978 50750. R=1.5. 32708 45538. 14638 22918. Table 39: Performance summary of SATB in Griewank function. NS R=1. 5. 10. 15. 20. 6255 6247 6212 6237. R=1.25 6249 6292 6292 6279 R=1.5. 6283 6278 6267 6274. 49.

(56) Table 40: Performance summary of TB in Ackley function. NS. 5. 10. R=0.1 4520. 15. 20. 7412. 14545 13276. R=0.2 7394 13245. 15341 25008. R=0.3 4299. 7102. 10739 12668. R=0.5 4009. 6451. 9075. 11250. Table 41: Performance summary of TB(SAPTA) in Ackley function. NS. 5. R=0.1 4347. 10. 15. 20. 6470. 9570. 11511. R=0.2 5440 10707. 10583 16693. R=0.3 3837. 6025. 7892. 10395. R=0.5 3813. 5784. 7513. 9696. Table 42: Performance summary of SATB in Ackley function. NS. 5. 10. 15. 20. R=0.1. 819. 819. 795. 825. R=0.2 1854 1835 1727 1737 R=0.3 1791 1905 1831 1907 R=0.5 1828 1822 1828 1699. 50.

(57) Table 43: Performance summary of TB in Schwefel function. NS. 5. 10. 15. 20. R=20 17777 24076 28310 32588 R=30 17139 23691 28409 32939 R=40 17696 24167 27464 32555 R=50 18051 22980 28316 32364. Table 44: Performance summary of TB(SAPTA) in Schwefel function. NS. 5. 10. 15. 20. R=20 18087 23868 28034 32049 R=30 17003 23881 28104 32530 R=40 17767 24097 27310 32266 R=50. 3813. 5784. 7513. 9696. Table 45: Performance summary of SATBB in Schwefel function. NS. 5. 10. 15. 20. R=20 13428 15023 15631 15360 R=30 14550 16007 16314 16429 R=40 14981 16271 16531 16338 R=50 15746 16743 16066 17107. 51.

(58)