Here we describe the Adaptive Search Regions Method step by step. Basically ASRM contains the following steps:
Table 1: Adaptive Search Regions Method (1)Defining the grid.
(2)Choosing initial experimental points.
(3)Constructing the surrogate surface.
(4)Choosing a new point and verifying the stopping criterion.
(5)Shrinking the search region and refining grids.
(6)Stopping criteria for accuracy control.
2.1.1 Defining the Grid.
Given an experimental domain of interest, the grid points is chosen first and de-noted by Gk. Following the ideas in pattern search method the grid can be defined as follows.
To define the grid in ASRM we need two components, a main matrix and a minor matrix. The minor matrix B ∈ Rd×d is a diagonal matrix with positive diagonal elements, where d is the dimension. The main matrix is A ∈ Zd× N, where
N = p1× p2× . . . × pd. We can decompose A as:
A = [M − M L] = [Γ L], (2.2)
where M and −M are determined the boundary of the grid, and L denotes the other points of the grid and contains the column of zeros that is identified as the center of the grid.
Given a grid size 4k∈ R, and 4k is a positive number, we define a trial step in the grid of the form:
gki = 4k· B · ai+ Ck, (2.3) where ai is a column of A = [a1a2· · · aN]. Note that B · ai determines the direction of the step, and 4k is a step length parameter.
Now we want to show that the construction of the grid Gk by matrices A and B, so we discuss the case of dimension-two and the number of grid points is p1 = p2 = 3.
Here we choose the minor matrix B = I2, so the grid size of each dimension is the same.
Furthermore, we determine M and −M
M = The matrix L contains the other points of the grid and the column of zeros, then we will show how to generate the matrix A.
for i = 1 to p1 do
and we let 4k=0.5, Ck=(0, 0), then we have the grid
2.1.2 Choosing Initial Experimental Points.
The choice of initial experimental points is an important issue. We could consider the uniform design (Fang et al., 2000) as a method to pick initial experimental points Pinit∈ Gk. The uniform design seeks design points that are uniformly scattered on the domain. The benefit of using this method to choose initial experimental points is that we will not describe any operator bias into the analysis. We then have a temporarily optimal point Oinit = arg min{f (x)|x ∈ Pinit} after choosing initial experimental points. We will see that there is a specific criterion that allows the algorithm to determine whether the search region should be shrunken or not.
2.1.3 Constructing the Surrogate Surface.
• Kriging
Kriging is a geostatistical method which used to interpolate the surrogate surface of a random field. It is developed in the field of spatial statistics and can be used in application to computer experiments. The surrogate model is denoted by:
S =e
N
X
j=1
βjα(xj) + Z(x), (2.4)
where βj are the regression coefficients of a system of linear equations, and α(xj) are j functions in the regression. Here Z(x) is a random process with mean zero and covariance function of the form:
c(s, t) = σ2Cθ(s, t), (2.5)
where σ2 > 0 is the process variance and Cθ(s, t) is the correlation function.
The best linear unbiased predictor (BLUP) is obtained by choosing the vector w(x) to minimize
M SE[ ef (x)] = E[wT(x)f (Pexp) − f (x)]2, (2.6)
subject to the unbiasedness constraint for the k functions in the regression,
R = [r(x1), . . . , r(xp)]T, (2.9) for the p × k matrix,
C = {C(xi, xj)}, 1 ≤ i, j ≤ p, (2.10) for the p × p matrix of random process correlation between Z’s at the Pexp, and
c(x) = [C(x1, x), . . . , C(xp, x)]T, (2.11) for the correlations between Z’s at the Pexp and an untried point x. The MSE can be denoted as follows:
σ2[1 + wT(x)Cw(x) − 2wT(x)c(x)], (2.12) and the impartial constraint is RTw(x) = r(x). Employing Lagrange multipliers λ(x) for the minimization of the MSE, the weight w(x) of the BLUP has to satisfy
M =
The surrogate surface then can be rewritten in the following form
S = re T(x) eβ + cT(x)C−1(f (Pexp) − R eβ), (2.14) where eβ = (RTC−1R)−1RTC−1f (x) is the common generalized least-squares esti-mate of β.
2.1.4 Choosing a New Point and Verifying the Stopping Criterion.
In this subsection, we employ the surrogate surface eS to determine the next new point Pnew from Gk\ Pexp. Here we apply an optimization method to eS to find Pnew e.g. Pnew = arg min{ eS|x, for all x ∈ Gk\ Pexp}. We then add the point Pnew to Pexp and have a new possible optimal point Ok+1 = arg min{f (x)|s ∈ Pexp}. Then the algorithm is ready to check the stopping criteria for shrinking the search region.
When Ok+1 is identified, the algorithm will check if the function value of Ok+1 is lower than Ok. In other words, ρk = f (Ok) − f (Ok+1) will be computed. If ρk > 0, then let Ok = Ok+1 and keep on adding new point. On the contrary, if ρk ≤ 0, then the searching region should be shrunken and the grids refined. Thus the searching sequence can operate in an ever decreasing.
2.1.5 Shrinking the Search Region and Refining Grids.
If ρk ≤ 0, then the original searching region could not find an improvable Pnew. The searching region should be shrunken and the grids should be refined. The algorithm then attempts to find Pnew with a lower function value in a new search region.
While the algorithm can’t find a point with function value lower than the current optimal point Ok = arg min{f (x)|x ∈ Pnew} in the current grid Gk, we suppose that there is a lower point with smaller grid size nearby current optimal point Ok. Thus we choose as the current optimal point the center Ck+1 = Ok of a new grid Gk+1. The grid’s size control parameter 4k would be refined with a ratio R1, where R ≥ 2. Let 4k+1 = R · 4k, then we have Gk+1 = h
S gk+1(xi)i
, for i = 1, . . . , N , where gk+1(xi) = 4k+1· Bii· ai+ Ck+1. The main reasons that we consider such a grid reduction are as follows. We would like to have function values that have already been computed, that is, for some given experimental points in Pexp. This is because the new experimental region will take the optimal point of Pexp as the center of the region before doing the reduction. Here, we believe that the local minimum is located near the center, even though it is not actually the best point found thus far.
The range of each dimension will become a half of its value in the last experimental region, while still keeping the same number of grid points. However, some parts of
the new domain might not be completely included in the last experimental region;
the algorithm will change their position. Therefore, based on the same principle, the grid size is a factor of R1 times the prior grid size. Also, some grid points of former experimental region are contained in the latter experimental region. The algorithm then uses some of the information obtained. In the new search region, only the points whose function values have already been evaluated will be taken as the initial experimental points.
The following describes the step of grid reduction in detail:
1.The optimal point in Pexp determines the center of the new experimental region.
2.We verify whether the new experimental region is beyond the last experimental region or not with the use of a region checking algorithm. In short, we suppose that the grid point of each dimension is odd.
Algorithm. CHECKING REGION.
for i = 1 to d do
if C(i) − 12(pi − 1) ·124i ≤ x(i)−min,
new−x(i)−min = x(i)−min, new−x(i)−max = x(i)−min + (pi− 1) · 124i. elseif C(i) +12(pi− 1) · 124i ≥ x(i)−max,
new−x(i)−min = x(i)−max − (pi− 1) · 124i, new−x(i)−max = x(i)−max.
else
new−x(i)−min = C(i) −12(pi− 1) ·124i, new−x(i)−max = C(i) +12(pi− 1) ·124i. endif
end
2.1.6 Stopping Criteria for Accuracy Control.
Iterating the procedure until some stopping criteria have been achieved. These stopping criteria may generally be tuned to suit a given experiment and desired degree of accuracy. Some stopping conditions which are in common used :
• The mesh size is less than a certain numerical tolerance;
• Let ρk = f (Ok) − f (Ok+1), and ρk becomes less than a certain numerical tolerance;
• The available computational resources run out.