OPSO: Orthogonal Particle Swarm Optimization and
Its Application to Task Assignment Problems
Shinn-Ying Ho, Member, IEEE, Hung-Sui Lin, Weei-Hurng Liauh, and Shinn-Jang Ho
Abstract—This paper proposes a novel variant of particle swarm optimization (PSO), named orthogonal PSO (OPSO), for solving intractable large parameter optimization problems. The standard version of PSO is associated with the lack of a mechanism responsible for the process of high-dimensional vector spaces. The high performance of OPSO arises mainly from a novel move behavior using an intelligent move mechanism (IMM) which ap-plies orthogonal experimental design to adjust a velocity for each particle by using a systematic reasoning method instead of the con-ventional generate-and-go method. The IMM uses a divide-and-conquer approach to cope with the curse of dimensionality in determining the next move of particles. It is shown empirically that the OPSO performs well in solving parametric benchmark functions and a task assignment problem which is NP-complete compared with the standard PSO with the conventional move behavior. The OPSO with IMM is more specialized than the PSO and performs well on large-scale parameter optimization problems with few interactions between variables.
Index Terms—Orthogonal experimental design (OED), orthog-onal PSO (OPSO), particle swarm optimization (PSO), task assignment.
I. INTRODUCTION
P
ARTICLE swarm optimization (PSO) is one of the evolu-tionary computation techniques. It is a population-based search algorithm that exploits a population of individuals to probe promising regions of the search space. In this context, the population is called a swarm, and the individuals are called particles. Each particle moves with an adaptable velocity within the search space and retains in its memory the best position it ever encountered. The standard version of PSO is briefly described here [1]–[3].Assume a D-dimensional search space S⊂ RDand a swarm consisting of Np particles. The position X = [x1, x2, . . . , xD]T
of a particle in the search space S is treated as a can-didate solution. The current position of the ith particle is a D-dimensional vector Xi= [xi1, xi2, . . . , xiD]T∈ S. The
Manuscript received September 26, 2004; revised June 5, 2005 and December 29, 2005. This paper was recommended by Associate Editor G. C. Calafiore.
S.-Y. Ho is with the Department of Biological Science and Technology and the Institute of Bioinformatics, National Chiao Tung University, Hsinchu 300, Taiwan, R.O.C.
H.-S. Lin and W.-H. Liauh are with the Department of Information Engi-neering and Computer Science, Feng Chia University, Taichung 407, Taiwan, R.O.C.
S.-J. Ho is with the Department of Automation Engineering, National Formosa University, Yunlin 632, Taiwan, R.O.C. (e-mail: [email protected]).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSMCA.2007.914796
velocity of this particle is also a D-dimensional vector Vi=
[νi1, νi2, . . . , νiD]T∈ S. The best position encountered by the
ith particle is denoted as Pi = [pi1, pi2, . . . , piD]T∈ S.
As-sume g to be the index of the particle that attained the best position found by all particles in the ith particle’s neighbor-hood. The swarm is manipulated in some form resembling the following equations:
νid(t + 1) = wνid(t) + c1· rand() · (pid(t)− xid(t))
+ c2· rand() · (pgd(t)− xid(t)) (1)
xid(t + 1) = xid(t) + νid(t + 1) (2)
where i = 1, 2, . . . , Np is the particle’s index, d = 1, 2, . . . , D is the dimension index, and t = 1, 2, . . . indicates the iteration number. The variable w is a parameter called inertia weight, c1
and c2are positive constants, which are referred to as cognitive
and social parameters, respectively, and rand() is a function which generates a random number that is uniformly distributed within the interval [0,1]. There are two commonly used versions of PSO, i.e., the global version and the local version. In the local version of PSO, each particle’s neighborhood includes a limited number of social neighbors, whereas in the global version of PSO, it includes all the particles in the swarm.
The inertia weight w plays the role of balancing the global and local searches. It can be a positive constant (e.g., 0.9) or even a positive linear or nonlinear function of time [2], [3]. Although sometimes proper fine-tuning of the parameters c1
and c2 leads to an improved performance, an extended study
of the cognitive and social parameters is proposed in [4], and
c1= c2= 2 were proposed as default values.
Recently, Clerc and Kennedy [7] indicated that a constriction factor χ may help ensure convergence. The new velocity of a particle is manipulated by the following equation:
νid(t + 1) = χ [νid(t) + d1· rand() · (pid(t)− xid(t))
+d2· rand() · (pgd(t)− xid(t))] . (3)
The typical value of χ is
χ = 2κ
2 − φ −φ2− 4φ,
where φ = d1+ d2> 4, and κ = 1. (4)
Generalized models and techniques for controlling the con-vergence properties of PSO by fine-tuning its parameters are 1083-4427/$25.00 © 2008 IEEE
analyzed in [7]. Generally, d1= d2= 2.05 are often used, and
thus, χ = 0.72984 from (4). Therefore, (3) is equivalent to (1) with w = 0.72984 and c1= c2= 1.4962.
To prevent an explosion of the velocity, the usual method is simply to define a parameter Vmaxand prevent the velocity from
exceeding it on each dimension d for particle i If νid > Vmax, then νid = Vmax;
else if νid<−Vmax, then νid=−Vmax. (5)
The initialization of the swarm and velocities is usually performed randomly in the search space, although more sophis-ticated initialization techniques can enhance the overall perfor-mance of PSO [5]. The initial velocities are often distributed randomly over [−Vmax, Vmax]. According to the suggestion by
Eberhart and Shi [6], it is good to limit the maximum velocity
Vmaxto the upper value of the dynamic range of search.
Many intractable engineering problems, such as task as-signment problem [8]–[10], are characterized by a nonlinear multimodal search space and a large number D of system parameters. For a high-dimensional vector used as a potential solution, there is a large probability that some partial vectors move closer to the optimal solution, whereas others move away from the optimal solution. Because the velocity is adjusted on a full D-dimensional vector, a drawback of the standard PSO is associated with the lack of a mechanism responsible for the process of the high-dimensional vector space. van den Bergh and Engelbrecht [11] proposed a cooperative particle swarm optimizer which uses multiple swarms to optimize different components of the solution vector cooperatively.
In this paper, we propose an entirely different approach to improve the PSO performance for overcoming curse of dimensionality. Although several improved versions of PSO have been proposed, many of which have been based on the original move behavior in (1). Different from the improvement of convergence and regulation of tradeoff between the local and global exploration abilities of swarm by tuning control parameters, the proposed orthogonal PSO (OPSO) employs a novel move behavior to significantly improve the performance of PSO. The OPSO uses an intelligent move mechanism (IMM) which applies orthogonal experimental design (OED) [12]–[14] to adjust a velocity for each particle. Each particle using IMM can adjust its flying velocity from the viewpoint of individual parameters (partial vectors) rather than the full D-dimensional vector.
The OED-based IMM uses a divide-and-conquer technique to efficiently determine the next move of a particle. Generally, a
D-dimensional vector of the next move is divided into N (≤ D)
partial vectors. The IMM spends at most 2N objective function evaluations to reason a potentially good solution to be the next move consisting of N good partial vectors. The IMM generates the next move by using a systematic reasoning method instead of the generate-and-go method of the conventional PSO, re-sulting in economically obtaining an accurate solution to the intractable engineering problems.
OED with both orthogonal array (OA) and factor analysis is a representative method of quality control, which is also an
efficient technique in the Taguchi method [15]. Tsai et al. [16] proposed a hybrid Taguchi–genetic algorithm (GA) for global numerical optimization where the Taguchi method is inserted between the crossover and mutation operations of a GA. Tanaka [17] proposed an orthogonal design algorithm using GA-encoding and OED with no recombination or mutation. Zhang and Leung [18] proposed an orthogonal GA (OGA). Leung and Wang [19] proposed an improved OGA with quan-tization (OGA/Q) using an OA-based initial population for global numerical optimization. Both the OGA and the OGA/Q use OA without factor analysis. Ho et al. [20]–[22] proposed population-based evolutionary algorithms with an OED-based recombination to efficiently solve large parameter optimization problems. In addition, the OED also performs well in cooperat-ing with the simulated annealcooperat-ing (SA) [23], [24]. The original contribution of this paper is to apply the OED to the PSO rather than the GA and the SA. The resultant OPSO is a population-based algorithm with no recombination that is completely new with respect to published research works.
The task assignment problem, which is known to be NP-complete [25], is that of assigning tasks of a program among different processors of a distributed computer system in order to reduce the program turnaround time and to increase the system throughput [8]–[10]. In this paper, the local version of PSO with a constriction factor for parametric optimization functions and the global version of PSO for the task assignment problem [8] are selected for comparisons with the OPSO. Performance comparisons show that the OPSO with IMM is better than the PSO with the conventional move behavior in a limited amount of computation time using these test problems.
The rest of this paper is organized as follows. Section II presents the OED used for the IMM of OPSO. Section III gives the proposed OPSO with IMM. The performance comparisons of OPSO using the parametric optimization functions are pre-sented in Section IV. Section V describes the application of OPSO to the task assignment problems. Finally, Section VI concludes this paper.
II. ORTHOGONALEXPERIMENTALDESIGN
For self-completeness, the concise description of the used OED is given. More information for the efficient use of OED can be found in [20]–[24].
A. Concepts of OED
An efficient way to study the effect of several factors simul-taneously is to use the OED with both the OA and the factor analysis. The factors are the variables (parameters), which af-fect response variables, and a setting (or a discriminative value) of a factor is regarded as a level of the factor. A “complete facto-rial” experiment would make measurements at each of all pos-sible level combinations. However, the number of level combi-nations is often so large that this is impractical, and a subset of level combinations must be judiciously selected to be used, resulting in a “fractional factorial” experiment [12]–[14]. The OED utilizes properties of the fractional factorial experiments
to efficiently determine the best combination of factor levels to be used in design problems.
The OED can provide near-optimal quality characteristics for a specific objective. Furthermore, there is a large saving in the experimental effort. The OED uses well planned and controlled experiments in which certain factors are systematically set and modified, and then, the main effect of factors on the response can be observed. The OED specifies the procedure of drawing a representative sample of experiments with the intention of reaching a sound decision [15]. Therefore, the OED using the OA and the factor analysis is regarded as a systematic reasoning method.
B. Orthogonal Array
OA is a fractional factorial array, which assures a balanced comparison of levels of any factor. OA is an array of numbers arranged in rows and columns where each row represents the levels of factors in each combination, and each column represents a specific factor that can be changed from each combination. The term “main effect” designates the effect on response variables that one can trace to a design parameter. The array is called orthogonal because all columns can be evaluated independently of one another, and the main effect of one factor does not bother the estimation of the main effect of another factor [12], [13].
The used OA in IMM is described next. Let there be N factors with two levels each. The total number of level com-binations is 2N for a complete factorial experiment. To use
an OA of N factors, we obtain an integer M = 2log2(N +1)
where the bracket represents an upper ceiling operation, build an OA LM(2M−1) with M rows and M− 1 columns, use the
first N columns, and ignore the other M− N − 1 columns. For example, if N∈ {4, 5, 6, 7}, then M = 8, and L8(27) is used.
The numbers one and two in each column indicate the levels of the factors. Each column has an equal number of ones and twos. The array is orthogonal when the four pairs, i.e., (1, 1), (1, 2), (2, 1), and (2, 2), appear the same number of times in any two columns.
The OA can reduce the number of level combinations for factor analysis. The number of OA combinations required to analyze all individual factors is only M = O(N ), where N + 1≤ M ≤ 2N. A simple algorithm for constructing the used two-level OAs can be found in [22]. After a proper tabulation of experimental results, the summarized data are analyzed using the factor analysis to determine the relative level effects of factors.
C. Factor Analysis
Factor analysis using the OA’s tabulation of experimental re-sults can allow the main effects to be rapidly estimated, without the fear of distortion of results by the effects of other factors. Factor analysis can evaluate the effects of individual factors on the evaluation function, rank the most effective factors, and determine the best level for each factor such that the evaluation function is optimized.
Let fzdenote a function value of the combination z, where
z = 1, . . . , M . Define the main effect of factor j with level k as Sjk, where j = 1, . . . , N , and k = 1, 2 Sjk= M z=1 fz· Fz (6)
where Fz= 1 if the level of factor j of combination z is k;
otherwise, Fz= 0. Considering the case that the optimization
function is to be maximized, the level 1 of factor j makes a better contribution to the function than the level 2 of factor j when Sj1> Sj2. If Sj1< Sj2, level 2 is better. On the contrary,
if the function is to be minimized, level 2 (level 1) is better when Sj1> Sj2(Sj1< Sj2). If Sj1= Sj2, levels 1 and 2 have
the same contribution. The main effect reveals the individual effect of a factor. The most effective factor j has the largest main effect difference MED =|Sj1− Sj2|.
Note that the main effect holds only when no or weak in-teraction exists, and that makes the experiment meaningful. An actual experiment result is estimated based only on the factors with the major effect. The difference between the estimated and experimental results is the degree of interactions among factors. In order to achieve an effective design, experiments should be prepared so as to reduce the interactions among factors. For generalized optimization problems with parameters having more or less strong interactions, the OPSO can still work well because the evolution ability of the PSO can compensate the OED excluding the study of interactions.
After the better one of two levels of each factor is determined, an intelligent combination consisting of all factors with the better levels can be easily derived. The OED is effective for the development design of efficient search for the intelligent combination of factor levels, which can yield a high-quality objective function value, compared with all values of generated combinations.
III. PROPOSEDOPTIMALPSO
IMM is the main phase in adjusting the particles’ velocities of OPSO. The major concerns of efficiently using the IMM are the following: 1) how to encode system parameters into the position X of a particle and 2) how to effectively divide X into
N partial vectors where each partial vector is treated as a factor
of the OED. The representation X plays an important role in making the IMM efficient, as described in Section III-A. How to efficiently use the IMM involving the division of X is given in Section III-B. The proposed OPSO is given in Section III-C.
A. Particle Representation
A suitable way of encoding the system parameters into the position X of a particle is important in efficiently using the IMM. An experienced engineer of using IMM tries to use ap-propriate parameter transformation for handling constraints in order to confine searches within feasible regions. Considering the properties of OED, the main effects of individual factors are more accurate for unconstrained problems than constrained problems if the constraints are not properly handled. Therefore,
problem-specific particle representation and specialized oper-ations are generally more efficient in solving the constrained problems than any other methods using penalty approaches. The necessity of maintaining feasibility can make the OED-based operation more efficient than the penalty approach [22].
To accurately estimate the main effect of OED, the position of a particle should be encoded so as to reduce the degree of epistasis among parameters xi and maintain the feasibility
of all conducted combinations. A proper encoding scheme of particles where the parameters with strong interactions (if prior knowledge is available) are encoded together would enhance the effectiveness of particle division. An illustrative example using an effective parameter transformation to reduce the de-gree of epistasis and confine searches within feasible regions for designing genetic-fuzzy systems can be found in [20]. Consequently, the experiments can be prepared in order to reduce the interactions among factors by properly assigning partial vectors to individual factors.
B. Intelligent Move Mechanism
For the position of each particle Xi(t) = [xi1(t), . . . ,
xiD(t)]T, the IMM generates two temporary moves H =
[h1, h2, . . . , hD]Tand R = [r1, r2, . . . , rD]Tcorresponding to
the cognitive and social parts, respectively
hd= xid(t) + wνid(t) + c1· rand() · (pid(t)− xid(t)) (7)
rd= xid(t) + wνid(t) + c2· rand()·(pgd(t)−xid(t)). (8)
The IMM aims at efficiently combining good partial vectors of
H and R to generate the next move Xi(t + 1), as described
here.
Divide the D-dimensional vector of Xi into N
nonover-lapping partial vectors using the same division scheme for H and R. The proper value of N is problem-dependent. The larger the value of N , the more efficient it is the IMM if the interactions among partial vectors are weak. If the existing interactions among partial vectors are not weak, the smaller the value of N , the more accurate it is the estimated main effect of individual partial vectors. Considering the tradeoff, an efficient division criterion is to minimize the interactions among partial vectors while maximizing the value of N . To efficiently use all columns of OA excluding the study of intractable interactions, the used OA is LN +1(2N), and the largest value of N is
equal to 2log2(D+1)− 1 where the bracket represents a lower
ceiling operation. The N− 1 cut points are randomly specified from the D− 1 candidate cut points which separate individual parameters. Note that the parameter N at each call of the following IMM operation can be a constant or variable value. For example, a coarse-to-fine strategy using a variable value of N is sometimes more efficient [21] (for a more detailed study of efficiently assigning parameters to factors, please refer to the OED-based optimization algorithms with real-world applications [20]–[24]).
How to perform an IMM operation using Xi(t) with an
objective function f is described as follows.
Step 1) Generate two temporary moves H and R for Xi(t)
using (7) and (8).
Step 2) Divide each of H and R into N partial vectors where each partial vector is treated as a factor. Step 3) Use the first N columns of an OA LM(2M−1),
where M = 2log2(N +1).
Step 4) Let levels 1 and 2 of factor i represent the ith partial vectors of H and R, respectively.
Step 5) Compute the objective function value fz of the
combination z, where z = 1, . . . , M .
Step 6) Compute the main effect Sjk, where j = 1, . . . , N ,
and k = 1, 2.
Step 7) Determine the better one of two levels of each factor based on the main effect.
Step 8) The next move Xi(t + 1) is formed using the
com-bination of the better partial vectors.
Step 9) Verify that Xi(t + 1) is superior to Xi(t) and the
M sampling solutions derived from the OA
combi-nations. If it is not true, let Xi(t + 1) be the best one
among Xi(t) and these M sampling solutions.
The OED has been proven optimal for additive and quadratic models, and the selected combinations are good representations for all of the possible combinations [12]. The OA specifies a small number M of representative combinations that are uni-formly distributed over the neighborhood of the position Xi(t).
The number of objective function evaluations is M + 1 per IMM operation, which includes M evaluations in Step 5) and one in Step 9). The IMM spends M + 1 function evaluations, whereas the move mechanism of PSO spends one function evaluation using the generate-and-go method to determine the next move. However, the M sampling solutions and the factor analysis make the IMM more efficient in obtaining the next move. If interactions among partial vectors are weak, Xi(t + 1)
is a potentially good approximation to the best one of all the 2N
combinations in the neighborhood of Xi(t).
C. OPSO Algorithm
The simple OPSO algorithm with IMM for the optimization function f is described here. All efficient strategies of control-parameter specification and heuristics which are good for PSO may be also good for OPSO.
Step 1) Randomly initialize Xi(t) and Vi(t), and compute
f (Xi), where i = 1, . . . , Np, and t = 0.
Step 2) Compute Pi(t) for each particle and Pg(t) of the
swarm.
Step 3) Apply the IMM to generate Xi(t + 1) for each
Xi(t), i = 1, . . . , Np.
Step 4) Vi(t + 1) = Xi(t + 1)− Xi(t), i = 1, . . . , Np.
Step 5) If a prespecified stopping condition is met, stop the algorithm; otherwise, increase the value of t by one and go to Step 2).
The overhead of IMM in preparing the OA experiments and the factor analysis is relatively small compared with the cost of function evaluations. Let G be the number of total iterations. The number of IMM operations is Np· G. The time complexity of OPSO is Np + Np· G(M + 1) function evaluations, where M + 1 is the number of function evaluations per IMM operation. The complexity of PSO is Np + Np· G
TABLE I
PERFORMANCECOMPARISONS OFVARIOUSALGORITHMS FOR APARAMETRICFUNCTIONWITH AGLOBALOPTIMUM OF121.598
function evaluations. When the OPSO is compared with the PSO using the same number of function evaluations, the OPSO uses a smaller number of iterations.
Generally, research works limited the value of Np to the range of 20–60 [6]. It is suggested that even though there is a slight improvement to the optimal solution with increasing swarm size, it increases the number of function evaluations to converge to an error limit. Since the IMM can efficiently exploit the neighborhood of a particle, the OPSO generally uses a relatively small value of Np.
IV. PERFORMANCECOMPARISONS OFOPSO Kennedy [27] indicated that the local version of PSO might perform better on complex problems, whereas the global version of PSO would perform better on simple problems. Mendes et al. [26] indicated that one cannot find an efficient and general topology structure for any kinds of functions. Similarly, there are two versions of OPSO. To avoid from involving the specification of neighborhood topology and evaluate its gen-eralization ability, the global version of OPSO with constants
w = 0.9 and c1= c2= 2 is used for all performance
com-parisons in this paper. Let the optimization functions have D parameters. The used OA is LN +1(2N), where N =
2log2(D+1)− 1.
Four versions of PSO using benchmark functions (see Table II) are evaluated: global versus local version (with a neighborhood size of five) and inertia weight (w = 0.9 and
c1= c2= 2) versus constriction factor (w = 0.72984 and c1= c2= 1.4962). The best version of PSO is selected for
com-parisons with OPSO, which uses the local version with a constriction factor. In this section, the PSO uses Np = 20, and the OPSO uses Np = 5. Both the OPSO and the PSO do not use any local search or heuristics which can improve their performances for specific problems. For fair comparisons, the OPSO and the PSO use the same prespecified number of function evaluations (Neval) as the stopping condition.
A. Large Parameter Optimization Problem
KrishnaKumar et al. [28] proposed three approaches, namely, stochastic GA, sensitivity GA, and GA-local search, and provided reasonable success on the large parameter
opti-mization problems using the test function f (X) with D = 100 as follows: max f (X) =− D i=1 sin(xi) + sin 2xi 3 (9)
where X = [x1, . . . , xD]T, and variable xi∈ [3, 13]. The
per-formances of the aforementioned three methods and a simple GA (SGA) are obtained from [28]. To demonstrate a high performance of OPSO, the OPSO is additionally compared with the following popular GAs: SAGA [29], GA with adaptive probabilities of crossover and mutation (AGA) [30], and SGA with an elitist strategy using one-point crossover (OEGA) [31]. The results of SAGA, AGA, and OEGA are obtained from [22]. Since the performance of the stochastic GA is very good, both the PSO and the OPSO use Neval= 12 000 for a direct
com-parison. The average performances of all compared algorithms using ten independent runs are shown in Table I. The results reveal that the OPSO can efficiently obtain the best solution 116.183 where the solution of PSO is 113.705.
B. Parametric Optimization Function
To evaluate the efficiency of OPSO for solving optimization functions with various dimensions, 12 benchmark functions gleaned from the literature, including unimodal and multimodal functions, continuous and discontinuous functions, smooth and nonsmooth functions, and function with and without interac-tions among parameters, are tested in the experimental study. The test function, the variable domain, and the global optimum for each function with D parameters are obtained from [22] and listed in Table II.
The average, best, and worst performances of PSO and OPSO using 30 independent runs for D = 10 and 100 are listed in Tables III and IV. A paired Wilcoxon test is applied to the results of PSO and OPSO, and the p values are also given. The experimental results reveal that the OPSO performs well and is significantly better than the PSO for most functions in a limited amount of computation time.
In order to show that the proposed IMM is effective, the simple PSO and OPSO without heuristics are further com-pared with the following GAs with elitist strategy and the associated commonly used crossover, which are denoted as
TABLE II BENCHMARKFUNCTIONS
TABLE III
FUNCTIONVALUECOMPARISONS FORD = 10
OEGA (one-point), TEGA (two-point), UEGA (uniform), and BLXGA (BLX-α) [32]. Each parameter is encoded using 10 bit for all test functions except that functions f9 and f12
use 24 bit. The dimensions of the 12 test functions vary on D = 10, 20, . . . , 100 in the experiments. The stopping condition uses
Neval= 10 000 for all compared algorithms.
To illustrate the performance comparisons on various num-bers of parameters, we use a distance function Dist(D) for describing the mean distance between the optimal solution
TABLE IV
FUNCTIONVALUECOMPARISONS FORD = 100
fopt(D) and the obtained best solution fbest(D) for one
pa-rameter as follows:
Dist(D) =|fopt(D)− fbest(D)| /D. (10)
The results of average Dist(D) for all test functions with
D = 10, 20, . . . , 100 using 30 independent runs are shown in
Fig. 1 and Tables V and VI. The results of all GAs are obtained from [22]. Fig. 1 shows that the mean-distance values Dist(D)
TABLE V
FUNCTIONVALUES ANDRANKS FORFUNCTIONSWITHD = 10
TABLE VI
FUNCTIONVALUES ANDRANKS FORFUNCTIONSWITHD = 100
of OPSO and PSO slightly increase while D increases from 10 to 100 compared with other algorithms. Tables V and VI reveal that the OPSO and the PSO have the best and second performances with final ranks 1 and 2, respectively. From these experimental results, it reveals that the OPSO performs well in efficiently solving small and large parameter optimization problems in a limited amount of computation time.
Recently, two OED-based algorithms, namely, IEA [22] and OSA [24], which are efficient for solving the large parameter optimization problems were proposed. All the three OED-based algorithms have the same superiority for the large parameter optimization problems. The common scenario is that the simple IEA, OSA, and OPSO can significantly improve the conven-tional GAs, SA, and PSO, respectively, for large parameter optimization problems. Due to different aims and merits, the comparisons among the three OED-based algorithms are not discussed in this paper.
V. TASKASSIGNMENTPROBLEMS
A. Problem Description [8]
Task assignment is one of the core steps to effectively exploit the capabilities of distributed or parallel computing systems.
Fig. 2. TIG instance on a homogeneous system.
The task assignment problem is that of assigning tasks of a program among different processors of a distributed computer system in order to reduce the program turnaround time and to increase the system throughput. A distributed program is char-acterized by task interaction graph (TIG) G(VT, ET), where VT={1, . . . , m} represents the tasks of a program, and ET
models the interaction between these tasks. The edge weight
eij between nodes i and j denotes the information exchange
between these pair of tasks. The node weight wij corresponds
to the work to be performed by task i on processor j. In a homogeneous system, processors have the same computing power, and each task takes the same amount of time to execute on each processor. Fig. 2 shows an instance of the problem on a homogenous system.
The underling computing system is represented by a proces-sor interaction graph (PIG) (VP, EP), where VP={1, . . . , n}
represents the set of processors in the system, and EPrepresents
the set of communication links between these processors. An edge weight dkjbetween any two nodes k and j represents the
length of the shortest path between the corresponding proces-sors. The assignment problem can be formally represented as follows:
A : VT→ VP (11)
where A is a function that maps the set of tasks VTto the set
of processors VP. A(i) = j if task i is assigned to processor j. Let m be the total number of tasks and n be the total
number of processors, where 1≤ i ≤ m, 1 ≤ j ≤ n, and then, the objective function (a minimax cost model) is described next. Find a mapping instance A, such that when estimating the total time required by each processor, the largest time among all processors is minimized. Let Cexe(A)kbe the total execution
time of all tasks assigned to processor k, such that
Cexe(A)k =
i
wik ∀A(i) = k (12)
and let Ccom(A)kbe the total interaction time between the tasks
assigned to processor k and those that are not assigned to that processor in an assignment A, i.e.,
Ccom(A)k= i j dA(i)A(j)eij
Fig. 3. Performance comparisons of PSO and OPSO for various numbers m of tasks. (a) m = 50. (b) m = 100. (c) m = 150. (d) m = 200. (e) m = 250. (f) m = 300.
Then, for a given assignment A, Ctotal(A)k, which is the total
time of a processor k, will be equal to the sum of the execution time and the interaction time for processor k, i.e.,
Ctotal(A)k= Cexe(A)k+ Ccom(A)k. (14)
Then, the final cost of an assignment will be dominated by the processor with the highest Ctotal(A)k, i.e.,
Cost(A) = max (Ctotal(A)kfor 1≤ k ≤ n) . (15)
The goal of the task assignment problem is to find an assign-ment A which has the minimum cost for a given TIG on a given PIG, i.e.,
minimize Cost(A). (16)
B. Performance Comparison
The source code of the compared PSO and data sets is obtained from Salman et al. [8]. A particle is represented as
X = [x1, . . . , xm]T, where m is the number of tasks. Each
variable xi has a discrete set of possible values belonging to
{1, . . . , n}, where n is the number of processors in the
distrib-uted system under consideration. The particle representations (mapping between problem solution and particle) of PSO and OPSO are the same. The control parameters of OPSO are as follows: Np = 30, w = 0.9, and c1= c2= 2.0. If m > 100,
the OA table uses L16(215) with N = 15; otherwise, L8(27)
with N = 7. The stopping condition uses the same number of Neval with that of PSO using Np = 2m and 150
gene-rations [8].
The test data consist of six randomly generated TIGs. The size of TIG is varied from m = 50 to 300 nodes with an
TABLE VII
COSTCOMPARISONS FORTASKASSIGNMENTPROBLEMSWITHm TASKS
increment of 50. We compare the PSO with the OPSO using ten fully connected homogeneous processors. Therefore, there is no variable which is independent in the optimization problem. For each TIG, both the PSO and the OPSO conducted 30 independent runs. Fig. 3 shows the average convergences of PSO and OPSO for various numbers m of tasks. Table VII shows the cost results of PSO and OPSO.
The performance of the PSO in solving the task assignment problem is evaluated in comparison with the GA for a number of randomly generated mapping problem instances in [8]. The results showed that the solution quality of PSO is better than that of GA in most of the test cases. According to the results of convergence and cost value (Fig. 3 and Table VII), we can see that the OPSO is significantly better than the PSO in solving task assignment problems.
The task assignment problem with up to 300 tasks is a large parameter optimization problem which is NP-complete. Fur-thermore, there are strong interactions among the parameters of the task assignment problem. These interactions will result in the so-called linkage of GA. Linkage represents the ability of building blocks to bind tightly together. For the problem with strong interactions, the conventional crossover operator of GA usually disrupts building blocks and, thus, reduces the GA performance. The move mechanism can probe a wider region of search space than the OEGA. Furthermore, the probed space of the move mechanism is smoother than that of the crossover of GA in the late evolution. It would increase the chance to find a better solution than the current best solution. As results, the PSO is better than the GA in solving the task assignment problem, and the OPSO using the divide-and-conquer strategy is better than the PSO in solving the large task assignment problem.
VI. CONCLUSION
This paper has proposed a novel variant of PSO, which is named orthogonal PSO. OPSO is more specialized than PSO and performs well on large-scale parameter optimiza-tion problems with few interacoptimiza-tions between variables. In this paper, it is shown that the OPSO performs well in solving the real-valued parametric functions and the large discrete task assignment problem which is NP-complete in a limited amount of computation time. This is achieved by the proposed novel move behavior in the swarm: an IMM using a
divide-and-conquer strategy. The OPSO-based method can be widely used to efficiently solve various applications of optimization problems. We believe that the problem-dependent learning strategies and auxiliary techniques can further advance the performance of OPSO. From our computer simulation results, the intelligent multiobjective version of OPSO is also efficient in solving multiobjective large-scale parametric optimization functions compared with the existing multiobjective versions of PSO [33].
REFERENCES
[1] J. Kennedy and R. C. Eberhart, “Particle swarm optimization,” in Proc.
IEEE Int. Conf. Neural Netw., Perth, Australia, Dec. 1995, vol. IV,
pp. 1942–1948.
[2] Y. Shi and R. C. Eberhart, “Parameter selection in particle swarm optimization,” in Evolutionary Programming, vol. VII, V. W. Porto, N. Saravanan, D. Waagen, and A. E. Eiben, Eds. Berlin, Germany: Springer-Verlag, 1998, pp. 591–600.
[3] Y. Shi and R. C. Eberhart, “A modified particle swarm optimizer,” in Proc.
IEEE Conf. Evol. Comput., Anchorage, AK, 1998, pp. 69–73.
[4] J. Kennedy, “The behavior of particles,” in Evolutionary Programming, vol. VII, V. W. Porto, N. Saravanan, D. Waagen, and A. E. Eiben, Eds. Berlin, Germany: Springer-Verlag, 1998, pp. 581–590.
[5] K. E. Parsopoulos and M. N. Vrahatis, “Initializing the particle swarm optimizer using the nonlinear simplex method,” in Advances in
Intelli-gent Systems, Fuzzy Systems, Evolutionary Computation, A. Grmela and
N. Mastorakis, Eds. Interlaken, Switzerland: WSEAS Press, 2002, pp. 216–221.
[6] R. C. Eberhart and Y. Shi, “Comparing inertia weights and constriction factors in particle swarm optimization,” in Proc. IEEE Int. Conf. Evol.
Comput., 2000, vol. 1, pp. 84–88.
[7] M. Clerc and J. Kennedy, “The particle swarm—Explosion, stability, and convergence in a multidimensional complex space,” IEEE Trans. Evol.
Comput., vol. 6, no. 1, pp. 58–73, Feb. 2002.
[8] A. Salman, I. Ahmad, and S. Al-Madani, “Particle swarm optimization for task assignment problem,” Microprocess. Microsyst., vol. 26, no. 8, pp. 363–371, Nov. 2002.
[9] C.-C. Chiu, Y.-S. Yen, and J.-S. Chou, “A fast algorithm for reliability-oriented task assignment in a distributed system,” Comput. Commun., vol. 25, no. 17, pp. 1622–1630, Nov. 2002.
[10] J. W. Chinneck, V. Pureza, R. A. Goubran, G. Karam, and M. Lavoie, “A fast task-to-processor assignment heuristic for real-time multipro-cessor DSP applications,” Comput. Oper. Res., vol. 30, no. 5, pp. 643–670, Apr. 2003.
[11] F. van den Bergh and A. P. Engelbrecht, “A cooperative approach to particle swarm optimization,” IEEE Trans. Evol. Comput., vol. 8, no. 3, pp. 225–239, Jun. 2004.
[12] Q. Wu, “On the optimality of orthogonal experimental design,” Acta
Math. Appl. Sin., vol. 1, no. 4, pp. 283–299, 1978.
[13] A. Dey, Orthogonal Fractional Factorial Designs. New York: Wiley, 1985.
[14] A. S. Hedayat, N. J. A. Sloane, and J. Stufken, Orthogonal Arrays: Theory
[15] T.-P. Bagchi, Taguchi Methods Explained: Practical Steps to Robust
Design. Englewood Cliffs, NJ: Prentice-Hall, 1993.
[16] J.-T. Tsai, T.-K. Liu, and J.-H. Chou, “Hybrid Taguchi-genetic algorithm for global numerical optimization,” IEEE Trans. Evol. Comput., vol. 8, no. 4, pp. 365–377, Aug. 2004.
[17] H. Tanaka, “A comparative study of GA and orthogonal experimental design,” in Proc. IEEE Int. Conf. Evol. Comput., 1997, pp. 143–146. [18] Q. Zhang and Y.-W. Leung, “An orthogonal genetic algorithm for
mul-timedia multicast routing,” IEEE Trans. Evol. Comput., vol. 3, no. 1, pp. 53–62, Apr. 1999.
[19] Y.-W. Leung and Y. Wang, “An orthogonal genetic algorithm with quan-tization for global numerical optimization,” IEEE Trans. Evol. Comput., vol. 5, no. 1, pp. 41–53, Feb. 2001.
[20] S.-Y. Ho, H.-M. Chen, S.-J. Ho, and T.-K. Chen, “Design of accurate classifiers with a compact fuzzy-rule base using an evolutionary scatter partition of feature space,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 2, pp. 1031–1044, Apr. 2004.
[21] H.-L. Huang and S.-Y. Ho, “Mesh optimization for surface approxima-tion using an efficient coarse-to-fine evoluapproxima-tionary algorithm,” Pattern
Recognit., vol. 36, no. 5, pp. 1065–1081, May 2003.
[22] S.-Y. Ho, L.-S. Shu, and J.-H. Chen, “Intelligent evolutionary algorithms for large parameter optimization problems,” IEEE Trans. Evol. Comput., vol. 8, no. 6, pp. 522–541, Dec. 2004.
[23] S.-Y. Ho, S.-J. Ho, Y.-K. Lin, and W. C.-C. Chu, “An orthogonal simulated annealing algorithm for large floorplanning problems,” IEEE
Trans. Very Large Scale Integr. (VLSI) Syst., vol. 12, no. 8, pp. 874–877,
Aug. 2004.
[24] S.-J. Ho, S.-Y. Ho, and L.-S. Shu, “OSA: Orthogonal simulated anneal-ing algorithm and its application to designanneal-ing mixed H2/H∞optimal
controllers,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 34, no. 5, pp. 588–600, Sep. 2004.
[25] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to
the Theory of NP-Completeness. New York: Freeman, 1979.
[26] B. Mendes, J. Kennedy, and J. Neves, “The fully informed particle swarm: Simpler, maybe better,” IEEE Trans. Evol. Comput., vol. 8, no. 3, pp. 204– 210, Jun. 2004.
[27] J. Kennedy, “Small worlds and mega-minds: Effects of neighborhood topology on particle swarm performance,” in Proc. Conf. Evol. Comput., Washington DC, 1999, pp. 1931–1938.
[28] K. KrishnaKumar, S. Narayanaswamy, and S. Garg, “Solving large pa-rameter optimization problems using a genetic algorithm with stochastic coding,” in Genetic Algorithms in Engineering and Computer Science, G. Winter, J. Périaux, M. Galán, and P. Cuesta, Eds. Hoboken, NJ: Wiley, 1995, pp. 287–303.
[29] H. Esbensen and P. Mazumder, “SAGA: A unification of the genetic algorithm with simulated annealing and its application to macro-cell placement,” in Proc. IEEE Int. Conf. VLSI Des., 1994, pp. 211–214. [30] M. Srinivas and L. M. Patnaik, “Adaptive probabilities of crossover and
mutation in genetic algorithms,” IEEE Trans. Syst., Man, Cybern., vol. 24, no. 4, pp. 656–667, Apr. 1994.
[31] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine
Learning. Reading, MA: Addison-Wesley, 1989.
[32] L. J. Eshelman and J. D. Schaffer, “Real-coded genetic algorithms and interval schemata,” in Foundation of Genetic Algorithm-2, L. D. Whitley, Ed. San Mateo, CA: Morgan Kaufmann, 1993. [33] S.-J. Ho, W.-Y. Ku, J.-W. Jou, M.-H. Hung, and S.-Y. Ho, Intelligent
Particle Swarm Optimization in Multi-Objective Problems, vol. 3918.
Berlin, Germany: Springer-Verlag, 2006, pp. 790–800.
Shinn-Ying Ho (M’00) was born in Taiwan, R.O.C.,
in 1962. He received the B.S., M.S., and Ph.D. degrees in computer science and information en-gineering from the National Chiao Tung Univer-sity, Hsinchu, Taiwan, in 1984, 1986, and 1992, respectively.
From 1992 to 2004, he was with the Department of Information Engineering and Computer Science, Feng Chia University, Taichung, Taiwan. He is cur-rently a Professor with the Department of Biolog-ical Science and Technology and the Institute of Bioinformatics, National Chiao Tung University. His research interests include evolutionary algorithms, soft computing, image processing, pattern recognition, bioinformatics, data mining, machine learning, computer vision, fuzzy classi-fier, large-scale parameter optimization problems, and system optimization.
Hung-Sui Lin was born in Taiwan, R.O.C., in
1977. He received the B.S. degree in information management from the National Defense of Man-agement College, Taipei, Taiwan, in 1999, and the M.S. degree in information engineering and com-puter science from Feng Chia University, Taichung, Taiwan, in 2004.
He is currently an Officer with the Office of the Deputy Chief of the General Staff for Communi-cations, Electronics and Information, Ministry of National Defense, Taiwan. His research interests in-clude evolutionary algorithms, large parameter optimization problems, system optimization, software development, and information security.
Weei-Hurng Liauh was born in Taiwan, R.O.C.,
in 1960. He received the B.S. degree in computer engineering from the Chung Cheng Institute of Tech-nology, Taoyuan, Taiwan, in 1987, and the M.S. de-gree from Feng Chia University, Taichung, Taiwan, in 2004.
His research interests include evolutionary com-putation and system optimization.
Shinn-Jang Ho was born in Taiwan, R.O.C., in
1960. He received the B.S. degree in power mechanic engineering from the National Tsing Hua University, Hsinchu, Taiwan, in 1983, and the M.S. and Ph.D. degrees in mechanical engineering from the National Sun Yat-Sen University, Kaohsiung, Taiwan, in 1985 and 1992, respectively.
He is currently a Professor with the Department of Automation Engineering, National Formosa University, Yunlin, Taiwan. His research interests include optimal control, fuzzy systems, evolutionary algorithms, and system optimization.
