Complexity

The complexity of the ASRM algorithm can be analyzed theoretical as follows.

O(

where d is the dimension of the problem, R is the grid size refining ratio, m is the maximum refinement factor, Nj is number of grid points of each dimension, and Di is the depth.

5 Conclusion and Future Works

In this article, we have proposed a new framework that serves not only to improve the accuracy of finding a given minimum, but that can also find multiple local minima automatically. The ASRM algorithm inherits its asymptotic convergence properties from the pattern search [35]. In addition, the algorithm exhibits theoretical global dense searching to ensure that all local minima and the global minimum can be found. The preliminary numerical results of the ASRM perform well, even for oscillatory high-dimensional problems. We obtained a significant improvement in finding all local minima in a given experimental area.

We do not deny the inadequacy of the present study. There are several further experimental studies that deserve to be undertaken. First, further study might usefully combine the information obtained from the local searching region with the overall experimental area. For example, it may be of interest to merge several local surrogates to construct an overall surrogate. Next, in the aim of achieving a high coverage rate in high-dimensional ASRM applications, it may eventually be beneficial to retrofit the algorithm for parallel processing.

References

[1] Mark A. Abramson and Charles Audet. Convergence of mesh adaptive direct search to second-order stationary points. SIAM Journal on Optimization, 17(2):606–619, 2006.

[2] Mark A. Abramson, Charles Audet, and Jr. J. E. Dennis. Generalized pattern searches with deriva-tive information. Math. Program., 100(1):3–25, 2004.

[3] Charles Audet and Jr. J. E. Dennis. Analysis of generalized pattern searches. SIAM Journal on Optimization, 13(3):889–903, 2002.

[4] Charles Audet and Jr. J. E. Dennis. Mesh adaptive direct search algorithms for constrained opti-mization. SIAM Journal on Optimization, 17(1):188–217, 2006.

[5] S.D. Balkin and D.K.J. Lin. A neural network approach to response surface methodology. Commu-nications in Statistics-Theory and Methods, 29(9):2215–2227, 2000.

[6] Andrew J. Booker, Jr. J. E. Dennis, Paul D. Frank, David B. Serafini, Virginia Torczon, and Michael W. Trosset. A rigorous framework for optimization of expensive functions by surrogates.

Structural and Multidisciplinary Optimization, 17(1):1–13, 1999.

[7] G. E. P. Box. Evolutionary Operation: A Method for Increasing Industrial Productivity. Applied Statistics, 6(2):81–101, 1957.

[8] R.-B. Chen, W. Wang, and F. Tsai. Basis-based response surface method in computer experiments optimization. 2006.

[9] I. D. Coope and C. J. Price. On the convergence of grid-based methods for unconstrained optimiza-tion. SIAM Journal on Optimization, 11(4):859–869, 2001.

[10] Dennis D. Cox and Susan John. Sdo: a statistical method for global optimization. pages 315–329, 1997.

[11] William C. Davidon. Variable metric method for minimization. SIAM Journal on Optimization, 1(1):1–17, 1991.

[12] L. C. W. Dixon and G. P. Szeg¨o. The global optimization problem: an introduction. Towards Global Optimization, 2:1–15, 1978.

[13] Matthias Schonlau Donald R. Jones and William J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492, 1998.

[14] DL Donoho, M. Elad, and VN Temlyakov. Stable recovery of sparse overcomplete representations in the presence of noise. Information Theory, IEEE Transactions on, 52(1):6–18, 2006.

[15] K. B. Wilson G. E. P. Box. On the experimental attainment of optimum conditions. Journal of the Royal Statistical Society, 13(1):1–45, 1951.

[16] H.-M. Gutmann. A radial basis function method for global optimization. 2001.

[17] Robert Hooke and T. A. Jeeves. “direct search” solution of numerical and statistical problems. J.

ACM, 8(2):212–229, 1961.

[18] Patty Hough, Tamara G. Kolda, and Virginia Torczon. Asynchronous parallel pattern search for nonlinear optimization. Journal on Scientific Computing, 23(1):134–156, 2001.

[19] Jr. J. E. Dennis and Virginia Torczon. Direct search methods on parallel machines. SIAM Journal on Optimization, 1(4):448–474, 1991.

[20] D.R. Jones. A Taxonomy of Global Optimization Methods Based on Response Surfaces. Journal of Global Optimization, 21(4):345–383, 2001.

[21] Tamara G. Kolda. Revisiting asynchronous parallel pattern search for nonlinear optimization. SIAM Journal on Optimization, 16(2):563–586, 2005.

[22] Tamara G. Kolda and Virginia Torczon. On the convergence of asynchronous parallel pattern search.

14(4):939–964, 2004.

[23] Robert Michael Lewis and Virginia Torczon. Pattern search algorithms for bound constrained min-imization. SIAM Journal on Optimization, 9(4):1082–1099, 1999.

[24] Robert Michael Lewis and Virginia Torczon. Pattern search methods for linearly constrained mini-mization. SIAM Journal on Optimization, 10(3):917–941, 2000.

[25] Robert Michael Lewis, Virginia Torczon, and Michael W. Trosset. Why pattern search works.

Technical Report TR-98-57, 1998.

[26] Stefano Lucidi and Marco Sciandrone. On the global convergence of derivative-free methods for unconstrained optimization. SIAM Journal on Optimization, 13(1):97–116, 2002.

[27] K. M¨uller and L.D. Brown. Location of saddle points and minimum energy paths by a constrained simplex optimization procedure. Theoretical Chemistry Accounts: Theory, Computation, and Mod-eling (Theoretica Chimica Acta), 53(1):75–93, 1979.

[28] Elijah Polak and Michael Wetter. Precision control for generalized pattern search algorithms with adaptive precision function evaluations. SIAM Journal on Optimization, 16(3):650–669, 2006.

[29] R.G. Regis and C.A. Shoemaker. Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions. Journal of Global Optimization, 31(1):153–171, 2005.

[30] J. Sacks, W. J. Welch, T. J. Mitchell, and H. P. Wynn. Design and Analysis of Computer Experi-ments. Statistical Science, 4(4):409–423, 1989.

[31] C.M. Siefert. Model-assisted pattern search. Master’s thesis, College of William & Mary, 2000.

[32] A. S´obester, SJ Leary, and AJ Keane. A parallel updating scheme for approximating and optimizing high fidelity computer simulations. Structural and Multidisciplinary Optimization, 27(5):371–383, 2004.

[33] W. Spendley, GR Hext, and FR Himsworth. Sequential application of simplex designs in optimisation and evolutionary operation. Technometrics, 4(4):441–461, 1962.

[34] Virginia Torczon. On the convergence of the multidirectional search algorithm. SIAM Journal on Optimization, 1(1):123–145, 1991.

[35] Virginia Torczon. On the convergence of pattern search algorithms. SIAM Journal on Optimization, 7(1):1–25, 1997.

[36] Virginia Torczon and Michael W. Trosset. Using approximations to accelerate engineering design optimization. NASA Langley Research Center, ICASE Technical Report 98-33, 1998.

[37] Michael W. Trosset and Virginia Torczon. Numerical Optimization Using Computer Experiments.

Institute for Computer Applications in Science and Engineering, 1997.

[38] M. Wetter and E. Polak. A convergent optimization method using pattern search algorithms with adaptive precision simulation. Building Services Engineering Research and Technology, 25(4):327, 2004.