Chapter IV Computational Testing
4.4 Computational Results
Abbreviations and symbols appearing in the remaining section are listed in Table 4-1.
Table 4-1 Abbreviations and Symbols
Abbreviations
Maximum distance permitted
The proposed tabu search heuristic for solving the OP
Solution time in seconds on Intel Celeron 1.60GHz PC with 504 MB RAM Upper bound on score from Leifer and Rosenwein (1991) (for Set 1, 2 and 3) Tsiligirides algorithm (for data Set 2, 3 and 4 only)
Tsiligirides’s stochastic algorithm coded by Chao et al. (for Set 1, 4, 5 and 6) Golden, Levy, and Vohra algorithm (for Set 1, 2 and 3)
GWL KL ANN CGW GA LS
Golden, Wang, and Liu algorithm (for Set 1, 2 and 3) Keller algorithm (for Set 1, 2 and 3)
Wang, Sun, Golden and Jia’s neural network (for Set 1~Set 4) Chao, Golden, and Wasil algorithm (for Set 1~Set 4)
Tasgetiren and Smith algorithm (for Set 1~Set 4) Liang and Smith Algorithm (for Set 1~Set 4)
Symbols
+ -
Empty cell
TSOP produces a better score TSOP produces a worse score TSOP produces the same score
The best results of the proposed heuristic on each test problem are compared with the best results of other algorithms and the upper bound as follows:
Table 4-2 Comparisons of Results on Test Problem Set 1
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP UB TC GLV GWL KL ANN CGW GA LS TC GLV GWL KL ANN CGW GA ACO
5 0.054 10 10 10 10 10 10 10 10 10 10
10 0.058 15 20 15 15 15 15 15 15 15 15
15 0.031 45 45 45 45 45 45 45 45 45 45
20 0.047 65 70 65 65 65 65 65 65 65 65
25 0.052 90 95 90 90 90 90 90 90 90 90
30 0.050 110 120 110 110 110 110 110 110 110 110
35 0.057 135 140 135 125 135 130 135 135 135 135 + + 40 0.042 155 160 150 140 155 155 155 155 155 155 + +
46 0.046 175 180 170 165 175 175 175 175 175 175 + +
50 0.046 190 195 190 180 190 185 190 190 190 190 + + + 55 0.048 205 210 205 200 205 200 205 205 205 205 + + + 60 0.050 225 230 220 205 225 225 225 225 225 225 + +
65 0.050 240 245 235 220 240 240 240 240 240 240 + + 70 0.052 260 260 255 240 260 260 260 260 260 260 + + 73 0.050 265 270 260 255 265 265 265 265 265 265 + + 75 0.052 270 270 265 260 270 270 270 270 270 270 + + 80 0.052 280 285 275 275 280 280 280 280 280 280 + + 85 0.050 285 285 280 285 285 285 285 285 285 285 +
Summary of New Method vs. Previous Methods + 11 11 0 3 0 0 0 0
- 0 0 0 0 0 0 0 0
Table 4-3 Comparisons of Results on Test Problem Set 2
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP UB T GLV GWL KL ANN CGW GA LS T GLV GWL KL ANN CGW GA ACO 15 0.046 120 145 120 120 120 120 120 120 120 120
20 0.050 200 200 190 200 200 200 200 200 200 200 +
23 0.058 210 215 205 210 205 210 205 210 210 210 + + + 25 0.060 230 240 230 230 230 230 230 230 230 230
27 0.068 230 265 230 230 230 230 230 230 230 230
30 0.068 265 275 250 260 265 260 265 265 265 265 + + + 32 0.068 300 305 275 260 300 300 300 300 300 300 + +
35 0.072 320 350 315 300 320 320 320 320 320 320 + + 38 0.068 360 375 355 355 360 360 360 360 360 360 + +
40 0.068 395 400 395 380 395 380 395 395 395 395 + + 45 0.064 450 450 430 450 450 450 450 450 450 450 +
Summary of New Method vs. Previous Methods + 7 5 1 7 1 0 0 0
- 0 0 0 0 0 0 0 0
Table 4-4 Comparisons of Results on Test Problem Set 3
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP UB T GLV GWL KL ANN CGW GA LS T GLV GWL KL ANN CGW GA ACO 15 0.026 170 175 100 170 170 170 170 170 170 170 +
20 0.036 200 210 140 200 200 200 200 200 200 200 +
25 0.040 260 290 190 250 260 260 250 260 260 260 + + + 30 0.044 320 340 240 320 320 320 320 320 320 320 +
35 0.046 390 395 290 380 390 370 390 390 390 390 + + + 40 0.046 430 445 330 420 430 430 420 430 430 430 + + + 45 0.048 470 490 370 450 470 460 470 470 470 470 + + + 50 0.048 520 535 410 500 520 520 520 520 520 520 + +
55 0.050 550 575 450 520 550 550 550 550 550 550 + +
60 0.050 580 605 500 580 580 570 580 580 580 580 + + 65 0.052 610 635 530 600 610 610 610 610 610 610 + +
70 0.052 640 665 560 640 640 640 640 640 640 640 + 75 0.054 670 695 590 650 670 670 670 670 670 670 + +
80 0.054 710 725 640 690 710 700 700 710 710 710 + + + + 85 0.054 740 750 670 720 740 740 740 740 740 740 + +
90 0.054 770 785 690 770 770 760 770 770 770 770 + + 95 0.056 790 800 720 790 790 790 790 790 790 790 +
100 0.056 800 800 760 800 800 800 800 800 800 800 +
105 0.054 800 800 770 800 800 800 800 800 800 800 + 110 0.056 800 800 790 800 800 800 800 800 800 800 +
Summary of New Method vs. Previous Methods + 20 10 0 5 3 0 0 0
- 0 0 0 0 0 0 0 0
Table 4-5 Comparisons of Results on Test Problem Set 4
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP T TC ANN CGW GA LS T TC ANN CGW GA ACO
5 0.018 10 10 10 10 10 10 10
10 0.018 15 15 15 15 15 15 15
15 0.024 45 45 45 45 45 45 45
20 0.038 65 65 65 65 65 65 65
25 0.044 90 90 85 90 90 90 90 +
30 0.044 110 110 110 110 110 110 110
35 0.042 135 135 135 130 135 135 135 +
40 0.048 155 150 150 155 155 155 155 + +
46 0.046 175 175 175 175 175 175 175
50 0.048 190 190 185 190 190 190 190 +
55 0.048 205 205 200 205 205 205 205 +
60 0.050 225 220 220 220 220 225 225 + + + +
65 0.052 240 240 240 240 240 240 240
70 0.052 260 255 250 260 260 260 260 + +
73 0.052 265 260 265 265 265 265 265 +
75 0.054 275 270 265 270 275 270 275 + + + +
80 0.054 280 275 270 280 280 280 280 + +
85 0.050 285 280 285 285 285 285 285 +
Summary of New Method vs. Previous Methods + 7 8 3 1 1 0 - 0 0 0 0 0 0
Table 4-6 Comparisons of Results on Test Problem Set 5
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP TC CGW TC CGW
5 0.19 10 10 10
*10 0.18 40 40 40
15 0.24 120 100 120 +
20 0.29 205 190 195 + +
25 0.26 290 290 290
30 0.27 400 400 400
35 0.25 465 460 460 + +
40 0.24 575 575 575
45 0.24 650 645 650 +
50 0.24 730 730 730
55 0.24 825 820 825
60 0.23 915 915 915
65 0.24 980 980 980
70 0.24 1070 1070 1070
75 0.24 1140 1140 1140
80 0.25 1215 1215 1215
85 0.25 1270 1265 1270 +
90 0.26 1340 1340 1340
95 0.26 1395 1390 1380 + +
100 0.28 1465 1455 1435 + +
105 0.30 1520 1515 1510 + +
110 0.30 1560 1550 1550 + +
115 0.31 1595 1590 1595 +
120 0.32 1635 1635 1635
125 0.33 1670 1655 1655 + +
130 0.33 1680 1670 1680 +
Summary of New Method vs. Previous Methods + 12 7
- 0 0
Table 4-7 Comparisons of Results on Test Problem Set 6
Previous Methods New Method vs. Previous Methods
Tmax CPU TSOP TC CGW TC CGW
15 0.13 96 90 96 +
20 0.22 294 258 294 +
25 0.21 390 354 390 +
30 0.21 474 432 474 +
35 0.21 576 516 570 + +
40 0.21 714 642 714 +
45 0.21 816 732 816 +
50 0.22 900 828 900 +
55 0.23 984 906 984 +
60 0.25 1062 978 1044 + +
65 0.26 1116 1020 1116 +
70 0.28 1188 1110 1176 + +
75 0.28 1236 1152 1224 + +
80 0.30 1284 1200 1272 + +
Summary of New Method vs. Previous Methods + 14 5
- 0 0
Table 4-8 CPU Time Comparisons for Set 1~Set 4
Set 1 Set 2 Set 3 Set 4
Tmax CGW GA LS TSOP Tmax CGW GA LS TSOP TmaxCGW GA LS TSOP Tmax CGW GA LS TSOP 5 0.67 5.36 0.013 0.028 15 1.29 11.53 0.023 0.021 15 4.37 25.3 0.036 0.035 5 0.22 6 0.014 0.026 10 0.8 10.28 0.011 0.028 20 2.24 15.35 0.402 0.023 20 5.16 31.87 2.194 0.043 10 0.27 10.87 0.013 0.028 15 2.28 14.98 0.207 0.031 23 4.45 19.04 0.685 0.029 25 9.4 39.38 0.209 0.049 15 0.72 16.98 0.752 0.032 20 17.49 20.47 0.064 0.047 25 5.65 20.53 0.111 0.030 30 9.96 44.05 0.465 0.054 20 4.76 23.42 0.056 0.047 25 9.07 25.89 0.733 0.052 27 6.37 22.63 0.152 0.032 35 15.38 47.68 0.164 0.054 25 2.47 29.42 0.119 0.052 30 31.92 31.36 0.08 0.054 30 6.18 23.9 0.172 0.032 40 18.65 52.36 0.711 0.053 30 10.86 35.64 0.081 0.054 35 25.25 38.48 0.808 0.053 32 7.21 26 0.561 0.035 45 26.84 66.89 2.223 0.054 35 14.11 44.57 1.239 0.052 40 16.76 42.06 6.227 0.052 35 7.81 27.74 1.344 0.033 50 28.74 72.78 1.528 0.056 40 21.81 50.51 6.443 0.055 46 21.58 48.68 0.677 0.053 38 6.84 28.74 2.797 0.038 55 30.27 75.4 5.377 0.056 46 21.62 55.58 0.323 0.055 50 24.91 58.94 0.795 0.056 40 7.14 28.71 3.818 0.035 60 27.68 79.01 0.402 0.059 50 22.76 71.78 0.414 0.056 55 24.67 60.39 15.732 0.055 45 0.61 29.01 0.731 0.034 65 25.02 79.2 18.369 0.061 55 24.81 90.31 5.902 0.055 60 24.28 65.55 11.464 0.057 70 29.82 84.69 12.888 0.060 60 20.39 90.74 11.229 0.059 65 23.26 69.56 0.495 0.056 75 29.25 90.24 0.934 0.062 65 26.78 96.45 0.811 0.059 70 25.09 73.56 11.781 0.058 80 30.14 100.42 9.406 0.063 70 25.51 85.44 8.163 0.058
73 25.24 75.29 1.352 0.059 85 28.3 103.64 14.8 0.062 73 27.04 85.16 2.158 0.059 75 28.53 77.72 4.075 0.059 90 24.43 103.91 12.106 0.063 75 27.47 89.59 25.274 0.058 80 26.84 80.58 3.738 0.060 95 22.33 106.47 1.491 0.061 80 28.17 90.12 5.067 0.059 85 21.71 88.12 2.739 0.058 100 0.67 105.55 2.105 0.062 85 21.64 87.77 8.005 0.057
105 0.6 103.01 0.55 0.065
110 0.72 102.27 0.403 0.065
Table 4-9 CPU time for Set 5 and Set 6
Set 5 Set 6
T
maxTSOP T
maxTSOP
5 0.16 15 0.12
10 0.11 20 0.21
15 0.20 25 0.20 20 0.29 30 0.20 25 0.27 35 0.20 30 0.26 40 0.21 35 0.25 45 0.21 40 0.23 50 0.22 45 0.23 55 0.24 50 0.23 60 0.24 55 0.23 65 0.26 60 0.24 70 0.27 65 0.23 75 0.28 70 0.23 80 0.29 75 0.24
80 0.25 85 0.26 90 0.26 95 0.27 100 0.27 105 0.28 110 0.30 115 0.32 120 0.32 125 0.32 130 0.33
From Table 4-2~Table 4-7, we point out that the new heuristic is better in 34 problems of 49 when comparing with T (Tsiligirides (1984)); is better in 45 problems of 76 when comparing with TC (Tsiligirides’s stochastic algorithm coded by Chao et al. (1996)); is better in 26 problems of 49 when comparing with GLV (Golden, Levy, and Vohra (1987)); is better in 1 problem of 49 when comparing with GWL (Golden, Wang, and Liu (1988)); is better in 10 problems of 49 when comparing with KL (Keller (1989)); is better in 7 problems of 67 problems when comparing with ANN (Wang, Sun, Golden and Jia (1995)); is better in 13 problems of 107 when comparing with CGW (Chao, Golden, and Wasil (1996 a)); is better in 1 problem of 67 problem when comparing with GA (Tasgetiren and Smith (2000)); is not better in any problem of 67 problems when comparing with LS (Liang and Smith (2006)), that is, the proposed heuristic has the same performance of the effectiveness in test problem Set 1~Set 4 with LS. As pointed out, the proposed heuristic can find all best-known solutions in all test problems, and we find many new best solutions for test problems in Set 5 and Set 6, problems 4, 7, 19, 20, 21, 22 and 25 in Set 5 and problems 5, 10, 12, 13 and 14 in Set 6. As Tables 4-6~ 4-7, we find out that for problems 4, 7, 19, 20, 21, 22 and 25 in Set 5, their original best solutions are in turn 195, 460, 1390, 1455, 1515, 1550 and 1655, but now we have 205, 465, 1395, 1465, 1520, 1560 and 1670. Moreover, for problems 5, 10, 12, 13 and 14 in Set 6, their best solutions jump from 570, 1044, 1176, 1224 and 1272 to 576, 1062, 1188, 1236 and 1284.
As to the efficiency of the proposed heuristic, for CPU time (in seconds), the total time of every solution in Set 1~Set 4 is rounded to three digits behind the decimal point, and for every one in Set 5 and Set 6, the total time is rounded to two digits behind the decimal point because the value of CPU time is larger. For making a comparison of solid computation, that is very difficult because CPU time varies as hardware, software and coding and so on.
Therefore, we only choose to compare with LS (Liang and Smith (2006)) because it is the
closest to us in all respect according to the following information:
1. CGW is coded in FORTRAN and executed on a SUN 4/370 workstation. (Chao, Golden and Wasil (1996 a))
2. GA is coded in Borland C++ and executed on a Dell 450 PC. (Tasgetiren and Smith (2000))
3. LS is coded in Borland C++ and executed on an Intel Celeron 433 MHz PC with 128 MB RAM. (Liang and Smith (2006))
Besides, because CGW and GA have ever been compared with LS, thus we also present their results of CPU time as Table 4-8. In the Table 4-8, the CPU time of LS ranges from 0.011 seconds to 25.274 seconds and the mean CPU time lower than one second is in only 36 of the 67 problems in Set 1~Set 4, but for us, the CPU time ranges from 0.021 seconds to 0.060 and not only the mean CPU time of 67 test problems in Set 1~Set 4 are lower than 0.1 second but also it of 40 ones in Set 5~Set 6 are lower than one second, that is, it ranges from 0.11 seconds to 0.33. Although our minimum is 0.010 larger than LS, however that is a necessary result that we take a looser condition (i.e. maximal iteration number = 1000) to be my stopping criterion. The reason is that in fact, for many problems, we can find the proposed optimal solutions within 100~200 iterations (even less), especially for the problems whose Tmax value is smaller, and the test problems that generates the minimal CPU time or have better performance are just such problems, that is, problem 1 and 2 in Set 1 and Set4.
Moreover, very clearly, since we can find the proposed optimal solutions of problems in Set 5 and Set6 within one second as Table 4-9, thus it goes without saying that the performance of the proposed heuristic in the efficiency has its superiority.