Chapter IV Computational Testing
4.2 Parameter Testing and Setting
Parameters we need to test include:
1. the power of penalty function (i.e. t) 2. candidate list size
3. tabu list size
4. maximal non-improvable iteration number denoted by nonimprovable_max
5. the parameters related with stopping criterion including:(1) maximal iteration number
(2) maximal consecutive performed times of the intensification and diversification scheme that a better global optimum cannot be found between denoted maximal_
IDS_time
Besides, we also respectively try two kinds of candidate list and insertion strategies including:
1. “the first alternative”: we examine the candidate list and take the first solution which can improves the current solution (i.e. pathop).
2. “the best alternative”: we examine the whole candidate list to select the best solution.
Moreover, we test two kinds of algorithm structure including:
1. the structure with intensification and diversification scheme: this structure has been amplified in Chapter III.
2. the structure without intensification and diversification scheme: this structure has one very great difference from the former, that is, the timing, where candidate list size recovers. The former has two timings, but the later has only one. For the later, candidate list size can be recovered only when an improvable solution is found no matter how many iterations the search has passed. Conversely, for the former, the candidate size is recovered not only when an improvable solution is found but also when the heuristic performs intensification and diversification scheme. Moreover, the former only allows the search to find an improvable solution within certain iterations (i.e.
nonimprovable_max); otherwise the heuristic will perform the intensification and
diversification scheme, but for this, the later doesn’t.In order to test, all parameters are first set respectively as t=1, candidate list size=2m (and candidate list size=8m whenever no improvement happens), tabu status size=7,
nonimprovable_max=7 and Maximal_IDS_time=2. The candidate list and insertion strategy
are all “the best alternative”, and the algorithm structure is set as using intensification and diversification scheme. The way we test is that one parameter or one kind of algorithm structure is changed every time and the others are fixed. From the test, try to understand the sensitivity of each parameter and algorithm structure, and find the most one. Then focus on it to find the optimal values of other parameters or determine the optimal algorithm structure step by step until the optimal values of all parameters and the optimal algorithm structure are all determined. After test, we find the following conclusions:1. For the power of penalty function
Compared with other parameters, in the proposed heuristic, the parameter can be said the most sensitive one. For almost all test problems except problem 1 in Set6, we can all find the proposed optimal solutions no matter what value we set, and most test problems except problem 20 and 21 in Set 5 and problem 10 and 14 in Set6 can be very easily solved the proposed optimal solutions. Thereinafter, we call the measure index of the performance that represents if the proposed optimal solutions can be easily found as “the appearance probability of the proposed optimal solutions”. Although the few test problems cannot be solved the proposed optimal solutions very easily, however, their proposed optimal solutions are already better than the best-known solutions in the literatures. Besides, especially, we find as long as the power of penalty function is larger than 1, for problem 1 in Set6 and the above few test problems, the performance in the appearance probability of the proposed optimal solutions will become better, and such phenomenon is clearer when the power of penalty function is very large, that is, we don’t use the penalty function. Not only it is very clear for those few test problems to find their proposed optimal solutions become easier but also for problem 1 in Set6, it becomes that it can always be found its proposed optimal solution.
2. For candidate list size
We don’t test other change. We are all according to the suggestion in some literatures (e.g. Tang, H and Miller-Hooks, E. (2005), Ben-Daya and Al-Fawzan (1998), and so on), that is, candidate list size = 2m, but whenever no improvement happens, candidate list size becomes 8m. Conversely, the candidate list size recovers 2m.. However, there is a small difference, that is, we use m, not use n. n is the number of all points in the set, and m is the number of points within the ellipse that is constructed by using the start and end point to be its two foci and Tmax to be its length of major axis.
3. For tabu status size
3~18, and for variable form, the ranges we test include 5~9, 5~14, 5~18, 7~14, 7~18, 10~18, and so on. From the test, we find that no matter for the fixed or variable form, we cannot see too large difference and advantage for the search result no matter in the effect or the appearance probability of the proposed optimal solutions. Also, even if there are some differences and advantages, however, these differences and advantages are only in the above probability and for few test problems, and they are few, not clear and not regular. That is, for certain test problems, we may feel certain value or range of tabu status size can have good effect, but for others, they can’t. Therefore, We cannot definitely find the best value or range of tabu status size all along. In other words, the parameter also seems not to be very sensitive for the proposed heuristic. Moreover, it is also sure for using variable form to make the computer program spend more CPU time. So, finally, we still choose the same form and value with first, that is, tabu status size is equal to a fixed value 7.
4. For maximal non-improvable iteration number
We also test two forms of fixed and variable. For fixed form, the values we test are 3~20 etc., and for variable form, the ranges we test include 5~9, 5~14, 5~18, 7~14, 7~18, 10~18, and so on. From the test, we find that the situation of the parameter is very similar to the tabu status size. The parameter also seems not to be very sensitive for the proposed heuristic. In addition, why do we first set the same values with tabu list size? The reason is that we think if we do it, then every tabu move attribute can have at least one opportunity to be selected after it is released so that we may benefit the search to find an improvable solution. Moreover, in the test, an interlude happens. That is, for our computer program, if tabu list size = 7 and we want to reach above purpose, then we should set the parameter = 8, not 7. However, we find that 7 or 8 almost have no difference. Therefore, finally, we set the parameter is a fixed value 7 or 8.
5. For stopping criterion
By reviewing the literatures (e.g. Ben-Daya and Al-Fawzan (1998)), we originally want to take Maximal_IDS_time to be the stopping criterion and let Maximal_IDS_time = 2.
However, we find the parameter seems to be more sensitive. StopInde = 2 is too strict.
Although its sensitivity is also only in the appearance probability of the proposed optimal solutions, however, it is not like the power of the penalty function, is only for few test problems but for more ones. Moreover, we find that although for most problems, we can find the proposed optimal solutions of them very easily when we use this criterion, however, for some problems among them, the used total iteration times are about equal to 500~600 iterations, even higher. Therefore, we change stopping criterion as maximal iteration number
=1000, and it is sure that the above problem can be improved when we use maximal iteration number as stopping criterion. In addition, because the points in test problem Set 5 and Set 6 is more, therefore for the problems of the two data sets, we set maximal iteration number = 2000 so that we can find the proposed optimal solutions easily.
For some problems, maybe maximal iteration number doesn’t need so high values, but we find that for the proposed heuristic, the set values are not very high because the time that we spend for solving any test problem is all not more than one second, and even not more than 0.1 second like 67 test problems in Set 1~Set 4.
6. For candidate list and insertion strategies
We total test four kinds of combinations respectively including best-best, best-first, first-best and first-first. Finally, we find that “best-best” and “first-best” are the best, but finally, we choose “first-best” because it can a very great advantage. That is, for problem 20 and 21 in Set 5 and problem 14 in Set 6, it can have better performance in the appearance probability of the proposed optimal solutions. Moreover, for the algorithm parameters, it can make them be less sensitive, especially for the power of penalty function, and of course, it is more efficient for the computer program.
7. For algorithm structure
From test, we very clearly find that if we don’t use intensification and diversification scheme, then we must use the penalty function, or we not only cannot have good performance in the appearance probability of the proposed optimal solutions, but also cannot find the proposed optimal solutions for most problems. Moreover, even if we use the penalty function, the performance in the above probability is still lower for many problems, especially the problems in Set 6, and the sensitivity of parameters is higher, especially for the power of penalty function. That is, for the problems of different data sets, we must use different power of penalty function so that we can find the proposed optimal solution easily. For example, for Set 1 and Set 4, we must use power 3 and for Set 5, we must use power 1.
According to above, thus we determine and suggest the optimal algorithm parameters and structure is as follow:
1. The penalty function is not used, that is, we don’t accept infeasible solution in the search.
2. Candidate list size = 2m, but whenever no improvement happens, candidate list size becomes 8m. Conversely, once an improvable solution is found, candidate list size recovers 2m.
3. Tabu list size is fixed and its value is equal to 7.
4. Maximal non-improvable iteration number (i.e. nonimprovable_max) is fixed and its
value is equal to 7 or 8.5. Stopping criterion is maximal iteration number = 1000 for Set 1~Set 4 and 2000 for Set5 and Set 6.
6. Candidate list and insertion strategy are all “first-best”, that is, for Candidate list strategy, we use the first alternative and for insertion strategy, we use the best alternative.
7. The algorithm structure is the structure with intensification and diversification scheme.