Enhanced parallel cat swarm optimization based on the Taguchi method

(1)

Enhanced parallel cat swarm optimization based on the Taguchi method

Pei-Wei Tsai

a

, Jeng-Shyang Pan

a,b

, Shyi-Ming Chen

c,d,⇑

, Bin-Yih Liao

a

Department of Electronic Engineering, National Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan, ROC

b

Innovative Information Industry Research Center, Harbin Institute of Technology, Shenzhen Graduate School, Nanshan, Shenzhen City, Guangdong Province, China

c

Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, ROC

d

Graduate Institute of Educational Measurement and Statistics, National Taichung University of Education, Taichung, Taiwan, ROC

a r t i c l e

i n f o

Keywords:

Cat swarm optimization

Enhanced parallel cat swarm optimization Parallel cat swarm optimization Taguchi method

a b s t r a c t

In this paper, we present an enhanced parallel cat swarm optimization (EPCSO) method for solving numerical optimization problems. The parallel cat swarm optimization (PCSO) method is an optimiza-tion algorithm designed to solve numerical optimizaoptimiza-tion problems under the condioptimiza-tions of a small population size and a few iteration numbers. The Taguchi method is widely used in the industry for optimizing the product and the process conditions. By adopting the Taguchi method into the tracing mode process of the PCSO method, we propose the EPCSO method with better accuracy and less compu-tational time. In this paper, five test functions are used to evaluate the accuracy of the proposed EPCSO method. The experimental results show that the proposed EPCSO method gets higher accuracies than the existing PSO-based methods and requires less computational time than the PCSO method. We also apply the proposed method to solve the aircraft schedule recovery problem. The experimental results show that the proposed EPCSO method can provide the optimum recovered aircraft schedule in a very short time. The proposed EPCSO method gets the same recovery schedule having the same total delay time, the same delayed flight numbers and the same number of long delay flights as theLiu, Chen, and Chou method (2009). The optimal solutions can be found by the proposed EPCSO method in a very short time.

1. Introduction

In recent years, some artiﬁcial intelligence (AI) methods have been presented to solve optimization problems. Chu, Tsai, and

Pan (2006) and Chu and Tsai (2007)presented the cat swarm

opti-mization (CSO) method for solving optiopti-mization problems by retaining the natural behaviors of cats.Tsai, Pan, Chen, Liao, and

Hao (2008) presented a parallel cat swarm optimization (PCSO)

method based on the framework of parallelizing the structure of the CSO method. Genetic algorithms (GA) have successfully been used in the internet service (Elliott & Krzymien, 2009) and imped-ance measurements (Janeiro & Ramos, 2009); particle swarm opti-mization (PSO) techniques have successfully been used to design antennas (Wu et al., 2009) and to construct parameters in neural network systems (Lin, Chen, & Lin, 2009); ant colony optimization (ACO) techniques have successfully been used to solve the travel-ing salesman problem (TSP) (Dorigo & Gambardella, 1997) and the routing problem of networks (Pinto, Nägele, Dejori, Runkler,

& Sousa, 2009); artiﬁcial bee colony (ABC) techniques have

successfully been used to solve the lot-streaming ﬂow shop sched-uling problem (Pan, Tasgetiren, Suganthan, & Chua, 2011); cat swarm optimization (CSO) techniques have successfully been used to adjust the parameters of the SVM (Lin et al., 2009). Moreover, in the industry, the Taguchi method (Taguchi, Chowdhury, & Taguchi, 2000) has successfully been used for optimizing the product-line design and the process conditions. Tsai, Liu, and Chou (2004)

successfully adopted the Taguchi method into the GA’s crossover process and presented the hybrid Taguchi-genetic algorithm (HTGA).

Although the parallel cat swarm optimization (PCSO) method presented inTsai et al. (2008)has the ability to ﬁnd the near best solution under more strict conditions, its computational speed is not efﬁcient. It is obvious that to reduce the computational time of the PCSO method and to keep high accuracy results with a small population size simultaneously are the desired goals of the PCSO method. Therefore, in this paper, we propose the en-hanced parallel cat swarm optimization (EPCSO) method by adopting the orthogonal array of the Taguchi method into the tracing mode process of the PCSO method. The proposed EPCSO method can successfully been used for solving optimization problems.

⇑ Corresponding author. Tel.: +886 2 27376417; fax: +886 2 27301081. E-mail address:[email protected](S.-M. Chen).

Contents lists available atSciVerse ScienceDirect

Expert Systems with Applications

(2)

2. Parallel cat swarm optimization

Tsai et al. (2008)have proposed the parallel cat swarm

optimi-zation (PCSO) method for solving optimioptimi-zation problems. The basic idea of the PCSO method utilizes the major structure of the cat swarm optimization (CSO) method proposed byChu et al. (2006). The CSO method has two modes, i.e., the seeking mode and the tracing mode, for simulating the behaviors of cats to move the indi-viduals in the solution space. By adjusting the parameter MR, the ratio of individuals moved by the seeking process and the tracing process can be controlled, where MR 2 [0, 1]. Some methods for splitting a population into several sub-populations to construct a parallel structure have been presented, such as the island-model genetic algorithm (Whitley, Rana, & Heckendorn, 1999), the parallel genetic algorithm (Abramson & Abela, 1992), the ant colony system with communication strategies (Chu, Roddick, &

Pan, 2004) and the parallel particle swarm optimization algorithm

with communication strategies (Chang, Chu, Roddick, & Pan, 2005). Each of the sub-populations evolves independently and shares the information they have occasionally. It results in the reducing of the population size for each sub-population and the beneﬁt of cooper-ation is achieved.

In the PCSO method (Tsai et al., 2008), the individuals are sep-arated into a predeﬁned number of groups in the initial process to construct the virtual parallel space for the individuals. If we let the predeﬁned number of groups be equal to 1, then the PCSO method becomes the CSO method (Chu et al., 2006) due to the fact that there is only one group. The individuals in the same group provide a local near best solution for their group in every generation, and the global near best solution found so far can be discovered by comparing the local near best solutions collected from the parallel groups. The individuals in a group can only access the near best solution discovered by their own group, but when the process of information exchanging is applied, the parallel groups can receive a near best solution from another randomly picked group. The dif-ference between the PCSO method and the CSO method is de-scribed as follows. At the beginning of the PCSO method, N individuals are created and then they are separated into G groups. The calculation of the PCSO method in the tracing mode is different from that of the CSO method and there exists an information exchanging process. The process of the PCSO method is reviewed as follows (Tsai et al., 2008):

Step 1: Create N cats, randomly sprinkle the cats into the M-dimensional solution space within the constrain ranges of the initial value and randomly collect them into G groups. Generate the velocities for each dimension. Set the motion flags of the cats to make them move into the tracing mode or the seeking mode according to the user predefined value of MR, where MR 2 [0, 1]. The cats moved by the seeking mode process present higher exploitative capacity. Contrarily, the cats gain a higher explorative abil-ity when they are moved by the tracing mode process. Hence, MR affects the ratio of artificial agents to work on the exploitation and exploration.

Step 2: Evaluate the fitness values of the cats, respectively, by tak-ing the coordinates into the fitness function, which repre-sents the benchmark and the characteristics of the problem to be solved. After calculating the fitness values of the cats, respectively, record the coordinate xbest and

the ﬁtness value of the cat which has the best ﬁtness value found so far.

Step 3: Move the cats by taking the operations in the seeking mode by Eqs.(1)–(4)or the parallel tracing mode by Eqs.

(5) and (6)according to the statuses of the motion ﬂags.

Step 4: Reset the motion ﬂags of all cats and separate them into statuses that indicating the seeking or the tracing by re-pick [N (1 MR)] cats to move in the seeking mode and (N MR) cats to move in the parallel tracing mode. Step 5: Check whether the number of iterations reaches a

prede-ﬁned iteration number. If the condition is satisﬁed, apply the information exchanging process.

Step 6: Check whether the process satisﬁes the termination condi-tion. If the process is terminated, output the coordinate which represents the found best solution and Stop. Other-wise, go to Step 2.

2.1. The seeking mode process

In the seeking mode, the cat moves slowly and conservatively. It observes the environment before it moves. Four essential factors are defined in the seeking mode, i.e., Seeking Memory Pool (SMP), Seeking Range of the selected Dimension (SRD), Counts of Dimension to Change (CDC) and Self-Position Considering (SPC). SMP is used to define the size of the seeking memory for each cat to indicate the points sought by the cat. The cat will pick a point from the memory pool according to the rules described later; SRD declares the mutative ratio for the selected dimensions; CDC discloses how many dimensions will be varied. In the seeking mode, if a dimension is selected to mutate, the difference between the new value and the old one cannot be out of the range defined by SRD; SPC is a Boolean variable, which decides whether the point in which the cat is already standing will be one of the candidates to move to. No matter the value of SPC is true or false, the value of SMP will not be influenced. These factors are all playing important roles in the seeking mode. The process of the seeking mode is re-viewed as follows (Tsai et al., 2008):

Step 1: Generate j copies of catk, where j = SMP. If the value of SPC

is true, let j = SMP 1 and return the present position as one of the candidates.

Step 2: According to CDC, plus/minus SRD percents of the current value randomly and replace the old one for all copies according to Eqs.(1)–(3): M ¼ Modify [ ð1 ModifyÞ; ð1Þ jModifyj ¼ CDC M; ð2Þ xjd¼ xjd; d R Modify; ð1 þ rand SRDÞ xjd; d 2 Modify; d¼1;2;...;M

8

j; ð3Þ where Modify denotes the elements of the selected dimensions, whose values will be modiﬁed, and rand is a random variable in the range [0, 1].

Step 3: Calculate the ﬁtness values of all candidate points, respectively.

Step 4: If all the ﬁtness values are not exactly equal, calculate the selecting probability Piof each candidate point, shown as

follows: Pi¼

1; if FSmax¼ FSmin;

jFSiFSbj

FSmaxFSmin; where 0 < i < j; otherwise:

(

ð4Þ

If the goal of the ﬁtness function is to ﬁnd the minimum solution, then let FSb= FSmax. Otherwise, let FSb= FSmin, where FSmaxdenotes

the largest FS in the candidates and FSmindenotes the smallest one.

Step 5: Randomly pick the point to move from the candidate points and replace the position of catk.

(3)

2.2. The parallel tracing mode process

The parallel tracing mode process is reviewed as follows (Tsai et al., 2008):

Step 1: Update the velocities for every dimension

v

k,d(t) for the

catkat the current iteration, shown as follows:

v

k;dðtÞ ¼

v

k;dðt 1Þ þ r1 c1 ½xlbest;dðt 1Þ xk;dðt 1Þ;

where d ¼ 1; 2; . . . ; M ð5Þ

where t denotes the iteration number, xlbest;dðt 1Þ denotes the posi-tion of the cat which has the best ﬁtness value at the previous iter-ation in the group that catkbelongs to and M denotes the dimension

of the solution space.

Step 2: Check whether the velocities are in the range of maximum velocity. The new velocity is bounded to the maximum velocity in case the new velocity is over-range.

Step 3: Update the position of catkaccording to Eq.(6):

xk;dðtÞ ¼ xk;dðt 1Þ þ

v

k;dðtÞ: ð6Þ

2.3. The information exchanging process

The information exchange process forces the sub-populations to exchange their information and achieves somehow the coopera-tion. It deﬁned a parameter ECH to control the exchanging of the information between sub-populations. The information exchang-ing process is applied once per ECH iterations. The process of infor-mation exchanging consists of the following four steps (Tsai et al.,

2008):

Step 1: Pick up a group of sub-populations sequentially and sort the individuals in this group according to their ﬁtness values.

Step 2: Randomly select a local best solution from an unrepeat-able group.

Step 3: The individual whose ﬁtness value is the worst in the group is replaced by the selected local best solution. Step 4: Repeatedly perform Step 1 to Step 3 G times to let every

group receives a local best solution from the others. 3. The Taguchi method

The Taguchi method (Taguchi et al., 2000) is an important tool for robust design. It is widely used for optimizing the product-line design and the process conditions due to the fact that it can pro-vide high quality products with low development costs (Tsai

et al., 2004). One of the major tools in the Taguchi method is called

the orthogonal array, which is adopted by the proposed EPCSO method. In the EPCSO method, only the two-level orthogonal array is used to take part in the process. In the following, we brieﬂy review the concept of two-levels orthogonal arrays fromTaguchi

et al. (2000). An array is called orthogonal due to that each column

indicates a value of a considered factor, and the factors listed in the orthogonal array can be evaluated independently. Every row in an orthogonal array represents a set of parameters for one run of the experiment. The two-levels orthogonal array can be described as follows:

Lnð2n1Þ; ð7Þ

where n = 2k_{, k is a positive integer which is greater than 1, and}

n 1 denotes the number of columns in the two-levels orthogonal array. For example, assume that we have two sets of solutions with 7 parameters in our design and assume that we want to ﬁnd the

optimal combination of their values. Then, the L8(27) orthogonal

array is shown inTable 1.

The elements in the orthogonal array shown inTable 1indicate which parameter value should be taken into the experiment on a specific considered factor. The values ‘‘0’’ and ‘‘1’’ in the elements of the orthogonal array shown inTable 1indicate which factor’s value should be used in a run of the experiment, where the value ‘‘0’’ inTable 1means that the factor’s value should be taken from the first set of the solution, and the value ‘‘1’’ inTable 1means that the factor’s value should be taken from the other set of the solu-tion. Therefore, fromTable 1, we can see that the 5th run of the experiment takes the values of the factors A, B, E and F from the first solution set and takes the values of the factors C, D and G from the second solution set to compose the run of the experiment. The orthogonal array provided by the Taguchi method (2000) can reduce the runs of the experiment efficiently. The case given in

Table 1requires 27_{runs of the experiment to explore the full}

com-bination, whereas the Taguchi method in this example reduces the experiment to 8 runs only.

4. The proposed enhanced parallel cat swarm optimization (EPCSO) method

InTsai et al. (2008), we have proposed the PCSO method by

organizing the artiﬁcial agents into a predeﬁned number of groups. When the agent moves in the process of the parallel tracing mode, it collects the local best information found so far by its own group and use the information to update its velocity. Although it presents higher accuracies and faster convergence than the CSO method

(Chu et al., 2006) under the conditions of a few number of artiﬁcial

agents and a short limited computational time, its computational time could be further improved in case that the number of artiﬁcial agents increases due to the fact that the number of artiﬁcial agents moving in the process of parallel seeking mode is much more than those move in the tracing mode in the original design. In this pa-per, in order to overcome the drawbacks of the CSO method (2006) and the PCSO method (2008), we adopt the Taguchi meth-od’s orthogonal array into the process of the parallel tracing mode process to propose the enhanced parallel cat swarm optimization (EPCSO) method.

The orthogonal array in the Taguchi method is quite useful for reducing the run of the experiment. When the artiﬁcial agent moves in the new designed parallel tracing mode, it accesses not only the local near best solution in its own group, but also utilizes the information comes from the global near best solution overall groups. The operation of the enhanced parallel tracing mode is de-scribed as follows:

Step 1: Generate two sets of candidate velocities, which are denoted by the symbols cv1,dand cv2,d, respectively, where

cv1;dðtÞ ¼vk;dðt 1Þ þ r1 c1 ½xgbest;dðt 1Þ xk;dðt 1Þ; where d ¼ 1; 2; . . . ; M;ð8Þ

cv2;dðtÞ ¼vk;dðt 1Þ þ r1 c1 ½xlbest;dðt 1Þ xk;dðt 1Þ; where d ¼ 1; 2; . . . ; M;ð9Þ

Table 1

The L8(27) orthogonal array.

Experiment number Considered factors

A B C D E F G 1 0 0 1 0 1 1 0 2 1 0 0 0 0 1 1 3 0 1 0 0 1 0 1 4 1 1 1 0 0 0 0 5 0 0 1 1 0 0 1 6 1 0 0 1 1 0 0 7 0 1 0 1 0 1 0 8 1 1 1 1 1 1 1

(4)

e = 825), the valid candidate positions, in this case, are the location of 823 and 893. After the seeking operation, the selected element and the elements locate after it are swapped with the calculated location. The tracing operation is also brought to the location index to insure every movement still retains the consistency between the destination of the former ﬂight and the origin of the current ﬂight. L ¼ flij^p0e;j¼ p0i;j1g; 1 6 i 6 M; 1 6 j 6 N; 1 6 li6M: ð28Þ

In our experiment, the population size of the proposed EPCSO meth-od is set to 16 and the populations are divided into four groups. The maximum iteration is set to 2000 and the experiment is repeated with 25 runs. The experiment is run on an Intel Core 2 2.4G CPU with the GCC compiler for C with a Freebsd 7.1-Release operating system. The experimental results show that the optimal recovery schedules can be found in every run by delay 24 flights with totally delay 595 min, and only 5 flights will be delayed relatively longer (more than half hour). To fully run the 2000 iterations costs less than 4.6 s and the optimum solutions can be found with 24 itera-tions in average.Fig. 6shows four of the recovered schedules found by the proposed EPCSO method, where the darker zone is the dis-turbance period of the closure of the airport ‘‘TSA’’. The proposed EPCSO method gets different recovered schedules with stable fitness values at final in all 25 runs. For all 25 runs, the standard deviation of the final fitness values is 4.547 1013_{and the total}

delay time is stabilized on 595 min.Fig. 5shows the recovery re-sults delay 24 ﬂights with totally 595 min delay, and 5 ﬂights are delayed more than 30 minutes. The standard deviation of the statis-tical total delay time for every recovery schedule is 14.

7. Conclusions

In this paper, we have presented the enhanced parallel cat swarm optimization (EPCSO) method for solving optimization

problems. In this paper, five test functions are used to evaluate the accuracy of the proposed EPCSO method. The experimental re-sults show that the proposed EPCSO method gets higher accuracies with less computational time than the existing methods. We also have applied the proposed method to solve the aircraft schedule recovery problem. Aircraft schedule recovery contains many essen-tialities and should be solved within a short period of time due to the disrupted aircraft schedule implies departure time delay for many flights. The experimental results show that the proposed EPCSO method can provide the optimum recovered aircraft sche-dule in a very short time. The proposed EPCSO method gets the same recovery schedule having the same total delay time, the same delayed flight numbers and the same number of long delay flights as theLiu et al.’s method (2009). The optimal solutions can be found by the proposed EPCSO method in a very short time, i.e., 24 iterations in average.

Acknowledgement

The authors thank Mrs. Szu-Ping Hao, Department of Mechani-cal Engineering, National Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan, for her help during this work.

References

Abramson, D. & Abela, J. (1992). A parallel genetic algorithm for solving the school timetabling problem. In Proceedings of 15 Australian computer science conference (pp. 1–11), Hobart, Australia.

Chang, J. F., Chu, S. C., Roddick, J. F., & Pan, J. S. (2005). A parallel particle swarm optimization algorithm with communication strategies. Journal of Information Science and Engineering, 21(4), 809–818.

Chen, X., & Li, Y. M. (2007). A modiﬁed PSO structure resulting in high exploration ability with convergence guaranteed. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 37(5), 1271–1289.

(5)

Chu, S. C., Tsai, P. W., & Pan, J. S. (2006). Cat swarm optimization. In Proceedings of the 9th Paciﬁc rim international conference on artiﬁcial intelligence (pp. 854–858), Guilin, China.

Chu, S. C., Roddick, J. F., & Pan, J. S. (2004). Ant colony system with communication strategies. Information Sciences, 167(1–4), 63–76.

Chu, S. C., & Tsai, P. W. (2007). Computational intelligence based on behaviors of cats. International Journal of Innovative Computing, Information and Control, 3(1), 163–173.

Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), 53–66.

Elliott, R. C., & Krzymien, W. A. (2009). Downlink scheduling via genetic algorithms for multiuser single-carrier and multicarrier MIMO systems with dirty paper coding. IEEE Transactions on Vehicular Technology, 58(7), 3247–3262. Janeiro, F. M., & Ramos, P. M. (2009). Impedance measurements using genetic

algorithms and multiharmonic signals. IEEE Transactions on Instrumentation and Measurement, 58(2), 383–388.

Lin, C. J., Chen, C. H., & Lin, C. T. (2009). A hybrid of cooperative particle swarm optimization and cultural algorithm for neural fuzzy networks and its prediction applications. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 39(1), 55–68.

Liu, T. K., Chen, C. H., & Chou, J. H. (2009). Optimization of short-haul aircraft schedule recovery problems using a hybrid multiobjective genetic algorithm. Expert Systems with Applications, 37(3), 2307–2315.

Pan, Q. K., Tasgetiren, M. F., Suganthan, P. N., & Chua, T. J. (2011). A discrete artiﬁcial bee colony algorithm for the lot-streaming ﬂow shop scheduling problem. Information Sciences, 181(12), 2455–2468.

Pinto, C., Nägele, A., Dejori, M., Runkler, T. A., & Sousa, J. M. C. (2009). Using a local discovery ant algorithm for bayesian network structure learning. IEEE Transactions on Evolutionary Computation, 13(4), 767–779.

Taguchi, G., Chowdhury, S., & Taguchi, S. (2000). Robust engineering. New York: McGraw-Hill.

Tsai, P. W., Pan, J. S., Chen, S. M., Liao, B. Y., & Hao, S. P. (2008). Parallel cat swarm optimization. In Proceedings of the seventh international conference on machine learning and cybernetics (pp. 3328–3333), Kunming, China.

Tsai, J. T., Liu, T. K., & Chou, J. H. (2004). Hybrid Taguchi-genetic algorithm for global numerical optimization. IEEE Transactions on Evolutionary Computation, 8(4), 365–377.

Whitley, D., Rana, S., & Heckendorn, R. B. (1999). The island model genetic algorithm: On separability, population size and convergence. Journal of Computing and Information Technology, 7(1), 109–125.

Wu, H., Geng, J., Jin, R., Qiu, J., Liu, W., Chen, J., et al. (2009). An improved comprehensive learning particle swarm optimization and its application to the semiautomatic design of antennas. IEEE Transactions on Antennas and Propagation, 57(10), 3018–3028.