單目標與多目標最佳化演算法的改良與其搜尋策略之研究

行政院國家科學委員會專題研究計畫 成果報告

單目標與多目標最佳化演算法的改良與其搜尋策略之研究

研究成果報告(精簡版)

計 畫 類 別 ： 個別型

計 畫 編 號 ： NSC 97-2218-E-161-001-

執 行 期 間 ： 97 年 01 月 01 日至 97 年 10 月 31 日

執 行 單 位 ： 亞東技術學院通訊工程系

計 畫 主 持 人 ： 謝昇達

計畫參與人員： 碩士班研究生-兼任助理人員：蔡宗穎

博士班研究生-兼任助理人員：林君玲

報 告 附 件 ： 出席國際會議研究心得報告及發表論文

公 開 資 訊 ： 本計畫可公開查詢

中 華 民 國 98 年 02 月 05 日

行政院國家科學委員會補助專題研究計畫

■ 成 果 報 告

□期中進度報告

單目標與多目標最佳化演算法的改良與其搜尋策略之研究

計畫類別：■ 個別型計畫 □ 整合型計畫

計畫編號：NSC 97－2218－E－161－001－

執行期間：2008 年 1 月 1 日至 2008 年 10 月 31 日

計畫主持人：謝昇達

共同主持人：

計畫參與人員：蔡宗穎、林君玲

成果報告類型(依經費核定清單規定繳交)：■精簡報告 □完整報告

本成果報告包括以下應繳交之附件：

□赴國外出差或研習心得報告一份

□赴大陸地區出差或研習心得報告一份

■出席國際學術會議心得報告及發表之論文各一份

□國際合作研究計畫國外研究報告書一份

處理方式：除產學合作研究計畫、提升產業技術及人才培育研究計畫、

列管計畫及下列情形者外，得立即公開查詢

□涉及專利或其他智慧財產權，□一年□二年後可公開查詢

執行單位：亞東技術學院 通訊工程系

中 華 民 國 9 8 年 1 月 2 2 日

一、 前言

數十年來，數學規劃法運用在各種領域最佳化問題的研究，已有許多具體成果，其中

利用梯度方向做為搜尋方向的資訊，對問題的極值求解具有很好的計算效率。此種方法的

特點為初始點位在最佳化問題的最佳解附近，便很容易搜尋到；或設計空間凸區域的問題

其極值亦為全域最佳解。然而，遇到含多極值點的最佳化問題時，全域最佳解的搜尋，則

端賴初始點的選擇，如果初始點位於局部最佳解處附近時，則容易收斂於局部最佳值。

為了改善傳統數學規劃法的缺點，近年來已有越來越多的研究採取仿生物智慧的概

念，利用在各種領域最佳化問題的研究上，其主要的共同特點為：非線性、類似生物推理

思考的方法、能利用自我學習(Self-learning)、對雜訊更具強建性(Robust)。

然而，為了應付工程上複雜的問題以及更高維度的挑戰，多目標最佳化的研究也顯得

相當的重要。在一些工程上，常常碰到希望能夠找到一些參數的最佳值，但在調整的過程

中，參數之間也將互相的影響，例如在建造橋樑時，會希望整體建材的重量越低但這也使

得堅固的程度降低，而不是預期中的提高。在飛機設計時，希望能夠最佳化燃油使的效率，

整體的載重量以及整架飛機的重量這幾個參數中取得有效平衡的解。除了在工程上，商業

上也希望藉由最佳化來使得風險最小，但希望同時所得到的財務回收能夠最大化。如何在

這麼多的變數中做取捨及調整，就必需藉由好的多目標最佳化的演算法來達成。

最佳化 (Optimization) 是在一個有許多限制和條件相互衝突的環境下找尋一個最合適

解決方式的過程。因此最佳化是一個複雜度和解決結果好壞的平衡點；最適當的答案表示

最好的妥協。在實際應用上，改進是相當重要的。甚至連極微小的改進都非常值得的。想

想看在工廠作業上和股票市場中，

5％的改善會有多大的影響。即使絕對的最好解答無法被

找到，一個跟這個答案相當接近的解決方案對原本的問題也有相當多的改善。

縱使目前電腦普遍擁有強大的運算能力，但許多較複雜的NP-complete問題，仍無法在

合理的時間內求得最佳解，因此有待發展出更有效率的演算法以解決日常生活中種種最佳

化問題。

二、 研究目的

本計畫提出以粒子群最佳化演算法，並針對原PSO架構提出以下缺失改良方法，以期

能在高維度問題提高求解效率及正確性。

1. 改善其它演算法容易陷入局部最佳解的缺點

當初始粒子分佈不均勻或過於集中時，加上演化策略不健全，則易形成狹隘的搜尋

解空間，這將會使粒子群在搜尋空上產生盲點而受限於局部最小值，需經過較多世

代演化才有機會跳脫，但卻無法保證一定能跳脫，大多只搜尋到次佳解即完成收斂。

因此針對此一缺失，我們在研究方法中提出改良辦法。

2. 提高求解效率，節省計算時間

初始粒子分佈不均勻或過於集中時，除了易造成狹隘的粒子搜尋空間，也會使粒子

群花費較多時間在解空間的探測(Explore)與開發(Exploit)上，降低了整體搜尋的效率

與效能。

三、 文獻探討

在一開始研究多目標最佳化時，都利用權重的方法，將多個目標函數結合起來，簡化

為一個單目標的問題，但是權重值的決定就是一個很大的困難，除非可以以經驗法則去調

整，否則則要以嘗試錯誤的方法找出適合的權重值。因此，若使用遺傳演算法來處理多目

標的問題，可以使用

Pareto based 的方法來解決，就不需要使用權重的方法，省去了尋找

適當權重值過程所需花費的時間及資源。

多目標最佳化問題的演化式解決方法，大致可以分為早期的以基因演算法為基礎，以

及近年以粒子族群演算法(Particle Swarm Optimization Algorithm, PSO)為基礎二種。基因演

算法

genetic algorithm (GA)是在 1975 年 John Holland[1]基於達爾文的”適者生存”概念所發

展出的演化式計算法則，藉由基因的競爭、交配與突變等方式來產生子代，而表現較好的

基因則可以存活下來，藉以產生更多較好基因。

而後，許多研究的發展，是基於這樣的概念而有了新的演化架構與機制。在交配的機

制方面，雙點交配(two-point)、多點交配(multi-point)以及統一交配(uniform )[2]等等，陸續

被提出來改良二元編碼基因演算法(binary-coded GA)的效率。另外，探索(exploration)、開

採(exploitation)[3]以及多重交配(multi-cross-over) [4]也隨之被提出用以改良實數編碼型基

因演算法(real-coded GA)。此外，基因演算法也可藉由突變的機制，讓基因群得以探索具開

發潛力或尚未探索的區域而不至於陷入局部最佳化的困境。在[5]中，動態突變被提出來以

調整各種不同形式的突變機制。另外，也有利用適應性機率來搭配突變機制的演算法[6]，

它是藉由

k-mean 與模糊系統來調整突變的發生率。

由

Leung 和 Wang 所提出 OGA/Q[7]是利用正交矩陣的特性並針對搜尋區域加以量化

為數個子區，在各個區域中產生子代。另外也有混合

Taguchi 方法的基因演算法 hybrid

Taguchi genetic algorithm (HTGA) [8]，HTGA 可以更有效率的產生子代並且有著比 OGA/Q

更好的求解能力。近期中，也有利用模糊推論的方式來產生子代，藉以判斷交配率與突變

率的大小。

相較於其他不同的計算智慧方法，諸如模糊理論、類神經網路等等，基因演算法可以

有效的避免局部搜尋，並可以增交找到整域最佳解的機會。並且已成功的被應用至機器學

習 [9][10]、數值最佳化 [11]-[14]、訊號處理等領域 [15]-[18]。

自

1995 年 Kennedy 和 Eberhart 提出了粒子群最佳演算法 (Particle swarm optimization,

PSO) [19]演算法的架構後，許多的研究也陸續投入，藉由不同的方式來改善 PSO 的求解效

能。其中一個被提出的參數被稱之為慣性權重 ( inertia weight) 在[20]中由 Shi and Eberhart

所提出並加入速度(velocity)的更新的方程式中。這個新的速度更新演算法陳述如下：

)]

(

)

(

)[

(

)]

(

)

(

)[

(

)

(

)

1

(

, 2 2 , , 1 1 , , , ,

g

x

g

gbest

g

r

c

g

x

g

pbest

g

r

c

g

wv

g

v

j i j j i j i j i j i j i j i













(3.1)

它扮演的角色是用以平衡整與搜尋和局部搜尋的比重。它可以是正值的常數或時變的

限性或非線性變化的正數。一般而言，它的設定是在演化過程中由

0 到 1 之間做線性的變

化。以特性而言，慣性權重與類神經網路演算法中梯度下降法的動量項類似。

在族群為為主的最佳化演算法中，局部和全域的搜尋方式將會直接影響到找尋全域最

佳解的效率。PSO 已經被應用來解決許多靜態問題的最佳化，且其求解性能也被證明是極

具效率的。然而，多數的的實際問題與應用是屬於非線性的動態系統。Eberhart 和 Shi 發

現：在

PSO 的參數設定中，固定、或線性遞增的慣性權重對於動態系統的求解較無效率。

考慮實際應用中的特性，他們在[21] 以隨機的方式來設定慣性權重，藉以更適切的應用於

動態問題求解。

改善

PSO 效能的另一個有趣的方法，由 Angeline [22]所提出。考慮粒子當下的適應函

數，而採用競賽的方式來做取捨。這個方法是將粒子群中表現較優的前百分之五十選出來，

將其內涵資訊複製出來，用以取代粒子群中表現較差的（後百分之五十）粒子，但是，並

不去改變被取代粒子的個人最佳解（personal best）。

在局部最佳版的

PSO，lbest 對應的是環狀的鄰域架構（neighborhood structure）。粒子

們都將會被其鄰近粒子的最佳位置所影響正如他們各自的過去最佳解也會影響自己的移動

一般。一旦 lbest 的收斂速度比 gbest 慢，代表的是 lbest 有較佳的求解結果且其搜尋了大

部分的搜詢空間[23]。

近期中，Kennedy 研究了其他鄰域拓墣架構發現馮紐曼拓墣架構可以產生較好的效能

與結果[24]。在[25]中，Parsopoulos 和 Vrahatis 提出了聯合式粒子群最佳化演算法（Unified

particle swarm optimizer, UPSO），它是結合了”全域版（Global version）”與”局部版（Local

version）”的 PSO 演算法。 而合作式粒子群最佳化演算法（Cooperative particle swarm

optimizer, CPSO）也於 [26]中被提出，它包含了兩種版本：CPSO-S 和 CPSO-H。其中，CPSO-S

是將

CCGA 直接套用在標準 PSO 上；而 CPSO-H 則是結合標準 PSO 和 CPSO-S，並且將

二者在每次的演化是代中交替執行。CPSO 將解向量空間分成數個子空間，每一個子空間

個別使用一組粒子族群（Swarm）來求解；也就是說，CPSO 使用的是一維空間的粒子族群

來求解單一維度問題，待所有維度求解完成，再將其合併。另外，Peram 等人提出了以適

應距離率為主的粒子群最佳化演算法（Fitness-distance-ratio based particle swarm optimization,

FDR-PSO）[27]，該演算法在粒子族群中定義了 n 個距離最接近其自身粒子的其他粒子為

鄰域，而此距離的計算，即為粒子間的歐幾里得距離（Euclidean distance）。在 In [28]中，,

轉向因子( turn-around factor)被提出來，並加入原始 PSO 架構之中以解決動態問題，如：

隱蔽式訊號分離問題( blind source separation)。在動態系統中的時變型最佳化問題中，由於

最佳解的所在地將會隨時間而改變，這將會使得粒子群無所適從(上一次的最佳經驗或搜尋

方向，在下個時間點可能不適用)，假如僅僅依據先前的經驗求解，將可能會導致搜尋結果

不如預期。El-Abd 和 Kamel 提出了階層式合作粒子群最佳化演算法( hierarchal cooperative

particle swarm optimizer) [29]，該演算法結合了兩種前期提出的演算法 CONPSO [30]和

CPSO-S。這樣的結合方式是讓兩種演算法各自的粒子群各自求解，並進行資訊交換已得到

較好的收斂性。在近期，有一個廣泛式學習粒子群最佳化演算法( comprehensive learning

particle swarm optimizer, CLPSO) [31]被提出來。它的學習策略使捨棄全域最佳解的資訊，

取而代之的是其他粒子的過去最佳解，並藉以決定每一個粒子的下一個移動速度向量。

在

多模態問題中，CLPSO 可以有效的改善原始 PSO 的求解效能。

由於有越來越多的

PSO 版本與變形陸續被提出來，然而那一個版本可以被稱之為”標

準”則難有定論，因此制定”標準 PSO”的構想也被提出來。這個構想是由 James Kennedy 與

Maurice Clerc 所主導的。將現有的 PSO 每年整理一次，並制定哪一個版本可稱之為標準以

免研究學者在比較時無所適從。標準

PSO 的制定並非在提出一個現有最佳的演算法，而是

參考文獻

[1] J. H. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press,

1975.

[2] G. Syswerda, “Uniform cross-over in genetic algorithms,” in Proc. of 3rd. International

Conference on Genetic Algorithms, pp. 2-9, 1989.

[3] Z. Michalewicz, T. Logan and S. Swaminathan, “Evolutionary operations for continuous

convex parameter spaces,” in Proc. of the 3

rd

Annual Conference on Evolutionary

Programming, pp. 84-97, 1994.

[4] W. D. Chang, “Application of a novel multi-cross-over genetic algorithm to FIR filters

estimation,” in Proc. of The joint conference on AI, Fuzzy System, and Gray System, pp.

A066: 1-6, Taipei, Taiwan, Dec. 2003.

[5] T. P. Hong and H. S. Wang, “A dynamic mutation genetic algorithm,” in Proc. of IEEE

International Conference on Systems, Man, and Cybernetics, vol. 3, pp. 2000-2005, 1996.

[6] J. Zhang, H. S. H. Chung and B. J. Hu, “Adaptive probabilities of cross-over and mutation

in genetic algorithms based on clustering technique,” in Proc. of Congress on Evolutionary

Computation (CEC2004), vol. 2, pp. 2280-2287, 2004.

[7] Y. W. Leung and Y. Wang, “An orthogonal genetic algorithm with quantization for global

numerical optimization,” IEEE Trans. on Evolutionary Computation, vol. 5, pp. 41-53, Feb.

2001.

[8] J. T. Tsai, T. K. Liu and J. H. Chou, “Hybrid Taguchi-genetic algorithm for global numerical

optimization,” IEEE Trans. on Evolutionary Computation, vol. 8, issue 4, pp. 365-377,

2004.

[9] L. B. Booker, D. E. Gokdberg and J. H. Holland, Classifier Systems and Genetic Algorithms,

Technical Report, No. 8, University of Michigan, 1978.

[10] H. Liu, Z. Xu and A. Abraham, “Hybrid fuzzy-genetic algorithm approach for crew

grouping,” in Proc. of 5th International Conference on Intelligent Systems Design and

Applications (ISDA '05), pp. 332-337, 2005.

[11] D. E. Gokdberg, Genetic Algorithms in Search, Optimization & Machine Learning, Addison

Wesley, 1989.

[12] Z. Tu and Y. Lu, “A robust stochastic genetic algorithm (StGA) for global numerical

optimization,” IEEE Trans. on Evolutionary Computation, vol. 8, issue 5, pp. 456-470,

2004.

[13] D. Jong, “Adaptive system design: a genetic approach,” IEEE Trans. on System, Man and

Cybernetics, vol. 10, pp. 566-574, 1980.

[14] H. Muhlenbein, M. Schomisch and J. Born, “The parallel genetic algorithm as function

optimizer,” Parallel Computing, vol. 17, pp. 619-632, 1991.

[15] T. Guo and C. Mu, “Blind separation of instantaneous mixed Gaussian sources via genetic

algorithms,” in Proc. of the 4th world congress on Intelligent Control and Automation, vol.

3, pp. 1849-1852, 2002.

[16] Y. Yue and J. Mao, “Blind separation of sources based on genetic algorithm,” in Proc. of the

4th world congress on Intelligent Control and Automation, vol. 3, pp. 2099-2103, 2002.

[17] M. Li and J. Mao, “A new algorithm of evolutional blind source separation based on genetic

algorithm,” in Proc. of WCICA 5th world congress on Intelligent Control and Automation,

vol. 3, pp. 2240-2244, 2004.

[18] P. Zheng, Y. Liu, L. Tian and Y. Cao, “A blind source separation method based on

diagonalization of correlation matrices and genetic algorithm,” in Proc. of WCICA 5th

world congress on Intelligent Control and Automation, vol. 3, pp. 2127-2131, 2004.

[19] J. Kennedy and R. Eberhart, “Particle Swarm Optimization,” in proceeding of the Fourth

IEEE International Conference on Neural Networks, pp. 1942-1948, 1995.

[20] Y. Shi and R. C. Eberhart, “A modified particle swarm optimizer,” in Proc. of IEEE World

Congress on Computational Intelligence, pp. 69-73, May 1998.

[21] R. C. Eberhart and Y. Shi, “Tracking and optimizing dynamic systems with particle

swarms,” in Proc. of IEEE world congress on Evolutionary Computation 2001 (CEC 2001),

pp. 94-97, May 2001.

[22] P. Angeline, “Using selection to improve particle swarm optimization,” in Proc. of

International joint conference on Neural Networks (IJCNN’99), pp. 84-89, Jul. 1999.

[23] A. P. Engelbrecht, Computational intelligence an introduction, Wiley, pp.185-195, 2002.

[24] J. Kennedy and R. Mendes, “Population structure and particle swarm performance,” in Proc.

of IEEE world congress on Evolutionary Computation 2001 (CEC 2002), pp. 1671-1676,

May 2002.

[25] K. E. Parsopoulos and M. N. Vrahatis, “UPSO—A unified particle swarm optimization

scheme,” in Lecture Series on Computational Sciences, pp. 868-873, 2004.

[26] F. van den Bergh and A. P. Engelbrecht, “A cooperative approach to particle swarm

optimization,” IEEE Trans. on Evolutionary Computation, vol. 8, pp. 225-239, Jun. 2004.

[27] T. Peram, K. Veeramachaneni and C. K. Mohan, “Fitness-distance-ratio based particle

swarm optimization,” in Proc. of Swarm Intelligence Symposium, pp. 174-181, 2003.

[28] C. L. Lin, S. T. Hsieh, T. Y. Sun and C. C. Liu, “PSO-based learning rate adjustment for

blind source separation,” in Proc. of International Symposium on Intelligent Signal

Processing and Communications Systems (ISPACS), pp. 181-184, Dec. 2005.

[29] M. El-Abd and M. S. Kamel, “A Hierarchal Cooperative Particle Swarm Optimizer,” in

Proc. Swarm Intelligence Symposium, pp. 43-47, 2006.

[30] S. Baskar and P. N. Suganthan, “A novel concurrent particle swarm optimization,” in Proc.

of IEEE Congress on Evolutionary Computation, vol. 1, pp. 792-796, 2004.

[31] J. J. Liang, A. K. Qin, P. N. Suganthan and S. Baskar, “Comprehensive learning particle

swarm optimizer for global optimization of multimodal functions,” IEEE Trans. on

Evolutionary Computation, vol. 10, pp. 281-296, Jun, 2006.

[32]

http://particleswarm.info/

四、 研究方法

本研究以粒子群最佳化演算法[20]為基礎，針對其搜尋特性加以研究，由於在(3.1)式

velocity 的更新方程式中，粒子的移動向量，將會根據自身先前的移動向量以及過去最佳的

位置以及群中最佳位置來前往新的地點進行最佳解的搜尋。由於群中最佳解極有可能僅只

是搜尋空間的局部最佳解(Local optimum)。當一群粒子江搜尋焦點集中在某一個局部最佳

解時，對於廣泛探索以求取新解是沒有幫助的，因此，如果可以適度的加入擾動的機制，

使得部份粒子可以被允許跳脫目前的蒐尋動作，將有利於探索整域最佳解的所在。因此，

改善粒子群在求解過程中容易陷入局部最佳解的缺點，當粒子群彼此適應函數達到一定接

近程度時，將會對粒子群加入擾動機制，隨機選擇一粒子跳出其原本依經驗所繼承的移動

位置，打亂其原本的搜尋軌跡。如圖

4.1 所示。

圖4.1 對粒子的移動路徑加入擾動

由上圖可發現：加入擾動後，粒子群會改變原本的位置，跳脫原來的搜尋解空間，再

重新做搜尋，擾動後的結果，經實驗證明將，將使得粒子更有機會搜尋到全域最佳解。

其次，由於在搜尋複雜問題時，gbest這一項往往對於”廣度搜尋”以求取整域最佳解是

沒有幫助的，而這一項的最大功能，在於找到一個最佳解可能存在的區域，進行”深度搜

尋”，因此在計畫的演算法中，也將3.1方程式改成

)]

(

)

(

)[

(

)]

(

)

(

)[

(

)

(

)

1

(

2 2 1 1

g

x

g

pbest

g

r

c

g

x

g

pbest

g

r

c

g

wv

g

v

i r i i i i i i













(4.1)

上式中的 r 是隨機值，且 r 不等於 i，當 r = 2 時，則挑選第二顆粒子的 pbest 做為一動向量

的參考。如此一來粒子與粒子之間將有機會互相參考彼此的最佳紀錄，以做為該次移動向

量的依據。

最後，在PSO演算法中，粒子會繼承前次粒子的個體最佳及群體最佳移動位置做為自

身移動方向與速度的依據，以此種方式在解空間中做最佳化的搜尋，如圖4.2所示。此種移

動機制基於演化法則似乎是合理的，但我們必須考量到搜尋解空間的完整性。若只是以前

次粒子移動經驗做為下次粒子的追隨依據，那麼當初始粒子分佈不均勻或過於集中時，則

會形成狹隘的搜尋解空間，這將會使粒子群花費較多時間在解空間的探測(Explore)與開發

(Exploit)上，甚至錯過了發現最佳解的機會，降低了整體搜尋的效率與效能。

Current Particle

Global Optimum

Local Optimum

Particle 1

Particle 2

Global best

solution

_{Past best}

solution

圖4.2 粒子移動

因此，為了擴展搜尋解空間的範圍，以提升PSO整體搜尋能力及加速探測與開發時間，

我們構想出了新的PSO移動機制:

)]}

(

)

(

)[

(

)]

(

)

(

)[

(

)

(

{

)

1

(

2 2 1 1

g

x

g

gbest

g

r

c

g

x

g

pbest

g

r

c

g

wv

T

g

v

i i i i i i i













(4.2)

其中T為轉向因子(turn-around factor),令所有奇數粒子的T為1,偶數粒子的T為-1，如圖4.3

所示，可以發現加入轉向因子(T =-1)後，粒子會做一個與目前學習經驗相反的移動，以圖

中例子說明，此機制會較有效率的搜尋到最好的解。

Global best solution

Past best

solution Particle’s next position by (4.2) Particle’s current position

Particle’s next position

by (3.1) or (4.2) Optimal solution

圖3.7 加入轉向因子的粒子移動空間

五、 結果與討論

為了驗證計畫中所提出的方法在搜尋最佳解中的效能改進，計畫中採用了

CEC 2005

Benchmark 的測試函數(詳參附錄)，並挑選 Std. PSO 2006 (標準 PSO 2006 版)做為比較驗證

的對象。在測試函數中，最佳解值越小越好，而初始設定與最佳解的位置詳列於附錄中。

計畫中挑選

CEC 2005 Benchmark 其中的 12 個函數，從複雜問題以致於高度複雜的混

合問題，以做為驗證演算法搜尋能力優劣的依據。我們針對每一個問題都值型

25 次的搜

值、標準差以及平均搜尋時間，所得知結果如表

5.1 所列：

表 5.1 實驗結果

f

01

f

02

f

03

f

04

f

05

f

06

最大值

8022.239 10730.7 -118.942 -317.965 -126.137

1020

最小值

1359.775 2996.434 -119.286 -329.876 -128.234 147.6717

平均值

4217.353 7649.073 -119.042

-324.8 -127.208 437.5193

標準差

2274.752 1792.661

0.0682

2.8634

0.5531 167.5072

ISP-PSO

時間

(秒) 156.0346 70.6922

95.8272

79.1057 161.4777 6672.605

最大值

83623.65 15118.86 -118.953

-92.207 1564.862 877.6264

最小值

25999.41 5018.683 -119.319 -254.383 -126.899 451.4314

平均值

48749.82 8813.52 -119.094 -198.189

-3.5319 653.1605

標準差

14658.54 2732.566

0.0812

35.0698 405.6627 98.1663

PSO 2006

時間

(秒) 161.0608

79.084

94.57

78.6121 162.2096 7030.617

f

07

f

08

f

09

f

10

f

11

f

12

最大值

722.6948 935.8466 1037.353 881.6507 1627.457 1639.864

最小值

306.8108 831.3842 832.7767

830.666 479.7946 477.0848

平均值

571.0373 844.0245 853.8822 845.7688 755.6903 1023.373

標準差

109.928 19.8121

38.0548

10.995 468.8602 551.3749

ISP-PSO

時間

(秒) 6415.788 6612.999 6595.825

6605.01 6490.571 6484.281

最大值

834.193 914.7198 1055.584 945.2832 1570.416 1597.304

最小值

382.9505 837.9491 844.6737

838.89 500.8369 531.0579

平均值

610.9881 865.9746 885.7798 871.5774 1080.393 1296.561

標準差

143.5315 21.2356

42.8264

24.8745 454.8028 363.6214

PSO 2006

時間(秒) 6767.576 6910.264 6919.249 6895.039 6786.398 6781.242

在上列數據中，可以發現：計畫中所改良的演算法的整體搜尋時間，略低於

Std. PSO

2006，顯示在整體計算量上，所提出的方法並未增加處理器的負擔。而且所搜尋到的最佳

解也都優於

Std. PSO 2006 的整體表現，足以顯示在計畫中所發展出來的演算法在於跳脫局

部最佳解以及開發新的為探索空間上，有較佳的搜尋能力，而演算法所求得的解與整域最

佳解的相差範圍也非常小，並且，演算法的求解標準差也都可以維持在較小的範圍，可以

得知演算法的穩定性與強健性是非常好的。

計畫中的演算法發展皆如預期的成果，不但改良了既有的缺點，也不會造成額外計算

量，只是在經費採購上，校方的行政流程耗費太多時間，以至於相關的設備無法於期待的

時間內完成，是美中不足的地方。

附錄

A. 測試函數

1) Shifted Schwefel Problem 1.2 with Noise in Fitness

]

,...,

,

[

,

_

)

)

1

,

0

(

4

.

0

1

(

*

1 2 1 1 1 N i i j j

x

x

x

bias

f

N

z

f









































 

 

x

o

x

z

1 2 N

(12)

2) Schwefel Problem 2.6 with Global Optimum on Bounds

0

*)

(

],

3

,

1

[

*

,

...

1

},

5

2

,

7

2

max{

_{1 2 1 2}

















x

f

x

n

i

x

x

x

x

f

Extend to N dimensions:

]

...,

,

,

[

,

...

1

,

_

}

max{

2 1 2 2 N i i

x

x

x

N

i

bias

f

f















x

o

x

z

B

x

A

₍₁₃₎

where A is a N*N matrix, aij are integer random numbers in the range [-500, 500],

, A

i

is the i

th

row of A. B

i

=A

i

*o, o is s N*1 vector, o

i

are random in the range[-100,

100]. After load the data file, set o

i

=-100, for

0

)

det(

A



] 4 / [ ,..., 2 , 1 N i

, o

i

=100, for

i[3N/4],...,N

.

3) Shifted Rotated Ackley Function with Global Optimum on Bounds

] ..., , , [ , * ) ( _ 20 ) 2 cos( 1 exp 1 2 . 0 exp 20 2 1 3 1 1 2 3 N N i i N i i x x x bias f e z N z N f                        





  x M o x z



(13)

where M is the linear transformation matrix, condition number=100.

4) Shifted Rastrigin Function

] ..., , , [ , _ ) 10 ) 2 cos( 10 ( 2 1 4 1 2 2 4 N N i i i i x x x bias f z z z f       



 x o x z



₍₁₄₎

5)

Shifted Expanded Griewank plus Rosenbrock Function

Griewank Function:

_

_

 







N i N i i i a

j

x

x

f

1 1 2

1

)

cos(

4000

Rosenbrock Function:

















      1 1 2 2 1 2 ₁ 100 N i i i i b x x x f

] ..., , , [ , 1 _ )) , ( ( )) , ( ( ... )) , ( ( )) , ( ( 2 1 5 1 1 3 2 2 1 5 N N b a N N b a b a b a x x x bias f z z f f z z f f z z f f z z f f f            x o x z

(16)

6)

Hybrid Composition Function 1

f

6

is composed using ten different benchmark functions: two Rastrigin functions, two Weierstrass functions,

7)

Rotated Hybrid Composition Function 1 with Noise in Fitness

Let

f₆f _ bias₆

be G(x), then

f₇G(x)*(10.2N(0,1)) f _bias₇

. All setting are the same as f

6

.

8)

Rotated Hybrid Composition Function 2

f

8

is composed using ten different rotated benchmark functions: two Ackley Functions, two Rastrigin functions,

two Weierstrass functions, two Griewank functions, and two Sphere functions.

9)

Rotated Hybrid Composition Function 2 with Narrow Global Optimal Basin []

In f

9

, all setting are the same as f

8

except with different coverage range and function stretch compressor [].

10) Rotated Hybrid Composition Function 2 with the Global Optimum on Bounds []

In f

10

, all setting are the same as f

18

except after load the data file, set o

1(2, j)

, for

j1 ,2 ,...,[ N/2]

11) Rotated Hybrid Composition Function 4 []

f

11

is composed using ten different benchmark functions: Weierstrass function, rotated expanded Scaffer F6

function, expanded Griewank plus Rosenbrock function, Ackley function, Rastrigin function, Griewank

function, non-continuous expanded Scaffer F6 function, non-continuous Rastrigin function, high conditioned

Elliptic function and Sphere function with noise in fitness.

12) Rotated Hybrid Composition Function 4 without Bounds

In f

12

, all settings are the same as f

11

except no exact search range set for this

test function.

where

o



[

o

1

,

o

2

,

...,

o

N

]

, which is used to shift global optimum,

x

*



o

,

, and k is

the function number.

*)

(

_

bias

k

f

k

x

B 函數初始設定

f

全域最佳解

初始範圍

搜尋範圍

函數偏移植

f

1

-450 [-100,

100]

N

[-100,

100]

N

-450

f

2

-310 [-100,

100]

N

[-100,

100]

N

-310

f

3

-140 [-32,

32]

N

[-32,

32]

N

-140

f

4

-330 [-5,

5]

N

[-5,

5]

N

-330

f

5

-130 [-3,

1]

N

[-3,

1]

N

-130

f

6

120 [-5,

5]

N

[-5,

5]

N

120

f

7

120 [-5,

5]

N

[-5,

5]

N

120

f

8

10 [-5,

5]

N

[-5,

5]

N

10

f

9

10 [-5,

5]

N

[-5,

5]

N

10

f

10

10 [-5,

5]

N

[-5,

5]

N

10

f

11

260 [-5,

5]

N

[-5,

5]

N

260

f

12

260 [-2,

5]

N

No

Boundary 260

全文完

出席國際學術會議心得報告

計畫編號 NSC

97-2218-E-161-001

計畫名稱

單目標與多目標最佳化演算法的改良與其搜尋策略之研究

出國人員姓名

服務機關及職稱

亞東技術學院 通訊工程系 助理教授 謝昇達

會議時間地點

時間：2008 年 6 月 1 日至 2008 年 6 月 6 日

地點：香港 香港會議展覽中心

會議名稱

2008 IEEE World Congress on Computational Intelligence (WCCI 2008)

發表論文題目

Solving Large Scale Global Optimization Using Improved Particle Swarm

Optimizer

一、參加會議經過

根據大會的安排，本人的論文發表時間如下：

Session ID: O4B2Q

Session Name: Special Session: Large Scale Global Optimization I

Date: June 5

Time: 16:30 -18:30

本次的 WCCI 2008 的會議安排在香港會議展覽中心(Hong Kong Convention and

Exhibition Centre)舉辦，報到當天即可領取論文集摘要與議程表，前幾天的幾場 Keynote

speech 相當精彩，尤其是 Jim Bezdek 主講的 Visual Cluster Validity、Witold Pedrycz 主講

的 Collaborative Architectures of Fuzzy Modeling 以及 Hans-Paul Schwefel 主講的 Simulated

Evolution Under Multiple Criteria Conditions Revisited。圖一為剛到會場時的剪影。

圖一 會場剪影

在 6 月 5 號當天，早上有 Takeshi Yamakawa 主講的 Bio-inspired Self-Organizing

Relationship Network as Knowledge Acquisition Tool and Fuzzy Inference Engine，也引發會

場學者的熱烈討論，模糊推論引擎的相關研究與方法也被充分的交流。

下午安排本人的報告是屬於 Special Session 之一，是針對高維度複雜問題所開設的主題，

首先大家都針對不同的最佳化方法加以說明，並導入各自所提出的新方法，並於報告後

給予 10~30 分鐘的問題討問，也因為討論非常熱烈，因此結束時間也遠超過原本的議程。

圖二是論文報告的剪影，本人的報告時間大約為 15 分鐘。

圖二 論文報告

論文報告後，不少學生與學者針對相關的議題提出討論，圖三是針對提出的問題，

本人返回之前所報告的投影片加以說明

圖三 問題討論

會議過程中，也有與外國學者針對先前的議題交流心得與討論，並且互留名片以供

日後交流，圖四為意見交流過程之剪影。

圖四 會場內的意見交流

二、與會心得

WCCI 是 2 年舉辦一次的超大型國際會議，會議結合了三個原本規模就不小的研討會。

本次出的研討會，的確得以觀摩許多優秀的研究與的論文，當中有些方法極富巧思，參考他

人的理論以及解決問題的技巧，也很能激發許多在未來研究方向的想法，在會中的晚宴，更

是促進交流的最佳管道，與一些在會場中認識的國際友人討論他們的論文內容、方法與理論，

或是閒話家常的對話，無形中也交流了一些不同的價值觀點。有趣的是，在會場也遇到台灣

的教授帶領學生一同出席，也有一些以往曾在會議中認識的人一同參與會議，不但能夠了解

不同地區或是不同國家的學者在做研究上面的態度，更能了解自我存在的渺小以及尚待努力

加強的地方。

在本會議期間，包學生所發表的論文在內，台灣所發表的論文只有寥寥幾篇，相較之下，

大陸的學者在相關的研究領域就有比較多的貢獻。可能是因為在香港舉辦，所以大陸的學者

投稿也比較踴躍。其實，看到其他研究單位對於一個研究總是很願意投入充分的時間、人力

和經費，如此一來也可以累積較多的研究成果。總有時候回頭看看自己的研究資源，的確是

還有很多可以努力的空間，如果可以有教充足的經費與人力支援，相關的研究也可以推展得

更順利。

Solving Large Scale Global Optimization Using Improved Particle

Swarm Optimizer

Sheng-Ta Hsieh, Tsung-Ying Sun, Member, IEEE, Chan-Cheng Liu and Shang-Jeng Tsai



Abstract—As more and more real-world optimization

problems become increasingly complex, algorithms with more capable optimizations are also increasing in demand. For solving large scale global optimization problems, this paper presents a variation on the traditional PSO algorithm, called the Efficient Population Utilization Strategy for Particle Swarm Optimizer (EPUS-PSO). This is achieved by using variable particles in swarms to enhance the searching ability and drive particles more efficiently. Moreover, sharing principals are constructed to stop particles from falling into the local minimum and make the global optimal solution easier found by particles. Experiments were conducted on 7 CEC 2008 test functions to present solution searching ability of the proposed method.

I. INTRODUCTION

ARTICLE swarm optimization (PSO) is a population-based self-adaptive search optimization technique that was proposed by Kennedy and Eberhart [1] in 1995, where the population is referred to as a swarm. The PSO is based on simulations of social behaviors such as fish in a school, birds in a flock etc. A swarm in PSO consists of a number of particles. Each particle represents a potential solution of the optimization task. All of the particles iteratively discover a probable solution. Each particle moves to a new position according to the new velocity and the previous positions of the particle. The PSO has ability of fast convergence to local and/or global optimal solutions over a small number of generations.

Similar to other population-based optimization methods such as genetic algorithms (GA), PSO algorithm starts with a random population initialization of particles in the search space [2]. Unlike other evolutionary optimization methods, particles in PSO do not recombine genetic material directly between individuals during the search, but works under social behavior in swarms instead. Therefore, it finds the global best solution by simply adjusting the moving vector of each individual according to personal best and the global best positions of particles in the entire swarm at each time step (generation), in other words, the search process allows

particles to stochastically return toward previously successful regions in the search space. The PSO method is becoming more popular due to its simplicity to implement and its ability to quickly converge to a reasonably acceptable solution on a plethora of problems in science and engineering. However, due to the particles’ searching behavior, all particles’ movement will also refer to global best positions. One the global best positions locate on local minimum. Other particles in the swam may also be trapped into while solving complex multimodal problems [3][4].

Manuscript received December 12, 2007.

Sheng-Ta Hsieh was with the Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan, R.O.C. He is currently with the Department of Communication Engineering, Oriental Institute of Technology, Taipei County, Taiwan, R.O.C. ([email protected])

Tsung-Ying Sun, Chan-Cheng Liu and Sheng-Jeng Tsai are with the Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan, R.O.C. (E-mail: [email protected]).

For the sake of improving PSO’s searching ability and efficiency, this paper presents the efficiency population utilization strategy for particle swarm optimizer (EPUS-PSO). EPUS-PSO is incorporated with the population manager to eliminate redundant particles and hire new ones or maintain particle numbers according to the solution searching status to make the process more efficient. The proposed method also has two built-in sharing strategies based on the sharing principle, which can prevent the solutions from falling into the local minimum and enhance the proposed approach’s searching ability.

II. PARTICLESWARMOPTIMIZATION(PSO) In the PSO, each particle moves to a new position according to the new velocity and the previous positions of the particle. This is compared with the best position generated by previous particles in the cost function, and the best one is kept; so each particle accelerates in the direction of not only the local best solution but also the global best position. If a particle discovers a new probable solution, other particles will move closer to it to explore the region more completely in the process [5].

A. Original PSO

Let s denote the swarm size. In general, there are three attributes, current position xi,current velocity viand past best

position Pbesti, for particles in the search space to present

their features. Each particle in the swarm is iteratively updated according to the aforementioned attributes. Assuming that the function f is the objective function of an optimization problem and the function f need to be minimized, the new velocity of every particle is updated by

)] ( ) ( )[ ( )] ( ) ( )[ ( ) ( ) 1 ( , 2 2 , , 1 1 , , , , g x g gbest g r c g x g pbest g r c g v g v j i j j i j i j i j i j i j i      ₍₁₎

P

1777 978-1-4244-1823-7/08/$25.00 c2008 IEEE

For all , is the velocity of j-th dimension of the

i-th particle, the c1and c2denote the acceleration coefficients,

r1and r2are elements from two uniform random sequences in

the range (0, 1), and g is the number of generations. The new position of a particle is calculated as follows:

N j1... vi,j ) 1 ( ) ( ) 1 ( , , , g x g v g xi j ij i j (2)

The past best position of each particle is updated using







 























 





¯ ® ­    t   g pbest f g x f g x g pbest f g x f g pbest g pbest i i i i i i i 1 if , 1 1 if , 1





(3)

and the global best position gbest found from all particles during the previous three steps is defined as:

s i g pbest f g gbest i i pbest  d d 1) argmin ( ( 1)), 1 ( (4)

The value of moving vector vican be restricted to the range

[-vmax, vmax] to prevent particles from moving out of the search

range

B. Some Variants of PSO

Since Kennedy and Eberhart [1] introduced PSO in 1995, many researchers have worked on improving its performance in various ways. One of the variants introduces a parameter called inertia weight of velocity w into the original PSO, introduced by Shi and Eberhart [6]. The new velocity update algorithm is shown as follows:

)] ( ) ( )[ ( )] ( ) ( )[ ( ) ( ) 1 ( , 2 2 , , 1 1 , , , , g x g gbest g r c g x g pbest g r c g wv g v j i j j i j i j i j i j i j i      (5)

It plays the role of balancing the global search and local search. It can be a positive constant or even a positive linear or nonlinear function of time. This value is typically setup to vary linearly from 1 to 0 during the course of a training run. The inertia weight is similar to the momentum term in a gradient descent neural network training algorithm.

In population-based optimization methods, how the local and global exploration is controlled will influence efficiency on finding the optimal solution directly. PSO has been proven to be effective while applied to optimizing static problems in earlier developments. Even so, most real-world applications are identified as nonlinear dynamic systems. Eberhart and Shi [7] found that the fixed or liner decreased inertia weight for PSO is not very effective for tracking dynamic systems. Considering the dynamic nature of real-world applications, a random inertia weight factor was proposed for tracking dynamic systems instead.

Another interesting approach to improve PSO performance was proposed by Angeline [8] who used a tournament selection process based on the particles’ current fitness. This method copies the current positions and velocities of the better half of the population to replace the worse half, without changing the “personal best” values of any of the particles in current step.

The local best version of PSO, lbest, reflects the circle neighborhood structure. Particles are influenced by the best position within their neighborhood, as well as their own past experience. While lbest is slower in convergence than gbest,

lbest results in much better solutions and searches a larger

part of the searching space [9].

More recently, Kennedy investigated other neighborhood topologies, finding that the von Neumann topology resulted in superior performance [10]. In [11], Parsopoulos and Vrahatis proposed a unified particle swarm optimizer (UPSO) which combined both the global version and local version. A cooperative particle swarm optimizer (CPSO) [3] which include two versions: CPSO-S and CPSO-H were also proposed. The CPSO-S model is a direct application of Potter' s CCGA model to the standard PSO, while the CPSO-H model combines the standard PSO with the CPSO-S model, and performs them both in each generation. CPSO split the space (solution vector) into several sub-spaces (smaller vectors) where each sub-space is optimized using a separate swarm, i.e. CPSO use one-dimensional swarms to search each dimension separately, the results are then integrated. Peram et al. proposed the fitness-distance-ratio based particle swarm optimization (FDR-PSO) [12], which defines the “neighborhood” of a particle as its n closest particles of all particles in the population (measured in Euclidean distance). In [13], the turn-around factor was involved in original PSO to solve dynamic problems such as blind source separation. Due to the time-varying optimal solutions in dynamic systems, it may be harder for the particles to catch up to various variations in each time slot, and may produce undesirable results if they move according to previous experiences only. El-Abd and Kamel proposed a hierarchal cooperative particle swarm optimizer [14] which combines two previously proposed models, CONPSO [15] and CPSO-S. The combination is achieved by having two swarms searching for a solution concurrently for solution exchange and performing better convergence. Recently, a comprehensive learning particle swarm optimizer (CLPSO) [4] was proposed. Its learning strategy abandons the global best information, and all other particles’ past best information is used to update particles’ velocity instead. CLPSO can significantly improve the performance of the original PSO on multimodal problems. There are more and more improved variants of PSO have been proposed, but it is hard to define which variants of PSO is the standard one. Thus, the idea of the standard PSO is to define a real standard at least for one year, validated by some researchers of the field, in particular James Kennedy and Maurice Clerc. This PSO version does not intend to be the best one on the market (in particular there is no adaptation of the swarm size nor of the coefficients) but simply very near of the original version (1995) with just a few improvements based on some recent works [16].

III. EFFICIENCTPOPULATIONUTILIZATIONSTRATEGY FOR

PARTICLESWARMOPTIMIZER(EPUS-PSO) Although PSO algorithms have been applied to a wide range of optimization problems and numerous variants of the PSO exist, solving high complexity problems with efficient

1778 2008 IEEE Congress on Evolutionary Computation (CEC 2008)

searching and rapid convergence are still an active area of research. In order to allow the better particles from exchanging their information, the population manager and solution sharing strategy were introduced to PSO. Also, to improve the particles’ searching abilities, an effective searching range sharing strategy was proposed to prevent solutions from falling into the local minimum. The sharing concept, which includes solution sharing and search range sharing, will make particles exchange their experience each other.

A. Population Manager

The first thing to apply the PSO algorithm to solve specific applications is to decide several initial parameters of PSO, which includes a choice of the particle number according to users’ experience or complexity of the problem. Having more particles can extend the searching space and increase the probability of finding the global optimal solution, but it will spend more time in each generation. The problem is that, until now, there is no way to know how many particles is suitable for solving the current problem.

Here, a population manager is introduced to EPUS-PSO to enhance its searching ability. The population manager will increase or decrease particle numbers according to the solution searching status, thus the population size in EPUS-PSO is variable. If the particles cannot find a better solution to update the global best solution gbest, particles may be trapped into the local minimum during the searching process, or need a competent guide to lead them toward the potential area. Thus, the information (experience) of existing particles may be too little to handle the current solution searching procedures. To keep gbest updated and to find better solutions in the current generation, new particles should be added into the swarm and share their up-to-date information to speed up the solution searching progress. In order to avoid unlimited increase in particles, a boundary for population size should be predefined. Therefore, the population manager will adjust population size in following three conditions:

1) If the gbest has not been updated in k consecutive generations, and if the current population size doesn’t exceed the boundary, a new particle will be added into the swarm. The new particle will be created from combining the information of the past best solutions of two randomly selected particles, through a crossover-like information combination strategy to provide some useful information for this particle, meaning, this particle will be placed at a beneficial position to the swarm. The new particle will be involved in the solution searching process in the following generation.

2) If particles can find one or more solutions to update the global best solution gbest in k consecutive generations, the existing particles may prove to be too many to handle the current solution searching procedures. For saving some time on finding the global optimal solution, and keep gbest updated, the redundant particles should be expelled from the swarm PSO to conserve their evolution time for speeding up the solution searching progress. If gbest has

been updated in two consecutive generations, and the current population size is more than one, a particle with poor performance in the current generation will be removed from the swarm. In the following generation, the particle number will be one less the original.

3) If the particles cannot find any better solutions to update global best solution, but the population size is equal to the boundary, a particle with poor performance in the current generation will be removed to reserve a space to accommodate a new potential particle.

where k is the population manager activating threshold which is a positive integer number. While k is set as 1, the population size will be adjusted in each generation. Finally, if the gbest has been updated intermittently, population size will remain the same. Setting a higher k value for the population manager will give the population adjustment lower sensitivity; while a lower k value for the population manager will increase its sensitivity so it could better process solution searching states (adjust population on time).

The possibility that the new particle may be eliminated instantly because it was born with a poor solution counteracts the purpose of the population manager. To prevent this situation, the value k should set more than 1. This will ensure the new born particle will live for more than one generation. In this paper, the k value is set as 2 in order to prevent the instant elimination of a newborn particle, and to keep population manager high sensitivity.

An example of the change in population size is showed in Fig. 1. It’s for solving the Quadric function [17] with ten dimensions. The initial population size is set as 1 and the maximum population size is set as 30. Population manager will dynamically adjust the particle numbers according to the current solution searching state.

Fig. 1 An example of the change in population size while solving the Quadric function with 30 dimensions.

B. Solution Sharing

In the proposed solution sharing strategy, the following equation was used to update the velocity:

2008 IEEE Congress on Evolutionary Computation (CEC 2008) 1779

° ° ¯ ° ° ® ­       t     i j i j r j i j i j i i j i j j i j i j i j i Ps rand if g x g pbest g r c g x g pbest g r c g wv Ps rand if g x g gbest g r c g x g pbest g r c g wv g v i i i i )] ( ) ( )[ ( )] ( ) ( )[ ( ) ( )] ( ) ( )[ ( )] ( ) ( )[ ( ) ( ) 1 ( , , 2 2 , , 1 1 , , 2 2 , , 1 1 , , (6)

where Psi denotes the solution sharing probability, whose

definition will have a detail description later, of the ith particle. It will decide if the moving vector of the third item in the velocity update equation is referring to the gbest or another particle’s pbest (pbestr). The pbestrcan be any other

particle’s pbest, not including its own.

In [4], Liang et al. found that different Learning Probability values will affect the results for the same problem if the same value of learning probability was used for all the particles in the swarm. Thus, each particle will be given a unique sharing probability. Before the particle’s new velocity is calculated, a random number will be generated. If this number is larger than or equal to Psi, the guide of the third

item of the velocity update equation will be the gbest. On the other hand, if this number is smaller than Psi, the particle will

learn from another particle’s pbest. In other words, the guide of the third item (gbest) of the velocity update equation will be replaced by pbestr. The selection of pbestr is stated as

follows.

If the random number is smaller than Psi

1) Randomly choose two particles’ pbest from the population (Except the pbesti).

2) Compare the fitness values of both pbest and select the better one as pbestr.

3) pbestrwill share its own information of all dimensions.

Thus, the solution searching of all the particles refer to not only their own pbest but also have the chance for learning from other particles’ pbest.

Since the dimensional information is incorporated in calculating the particles’ solution sharing probability. The definition of learning probability in [4] was referred to define the solution sharing probability for each particle:





N s i N Psi 1 1 1 exp 1 5 . 0  ¸ ¹ · ¨ © §   u  u (7)

where N denotes the dimension of problems and s is the population size. The solution sharing strategy will be not necessary while solving one-dimensional problems. The solution sharing probability will be set as zero due toN1 0.

There are two main differences between the solution sharing strategy and the comprehensive learning strategy [4]. 1) The solution sharing strategy not only learns from other particles’ experiences but also refers to gbest’s information.

2) For each particle, the solution sharing strategy picks another particle’s pbest as one of the guides for current movement instead of fine tuning dimensions one by one.

C. Searching Range Sharing (SRS)

According to the searching behavior of PSO, the gbest will be an important clue in leading particles to the global optimal solution. But it is unavoidable for the solution to fall into the local minimum while particles try to find better solutions. In fact, after several generations, particles will gather in several clusters, or even just one cluster, which is the local minimum. Each particle in the cluster may perform a local search to follow evolution, but not be able to explore other better solutions.

In order to allow the solution exploration in the area to produce more potential solutions, and find unsearched solution space. The searching range sharing (SRS) strategy, a mutation-like evolutionary strategy, was introduced to the EPUS-PSO algorithm. The SRS strategy can be classified into two versions: local and global. The two versions of SRS are classified according to a restricted boundary. Similar to mutation operation, the SRS activates under a predefined SRS rate which can be setup to vary linearly from 0 to 0.2 during the course of a training run. Note that this is similar to the temperature adjustment schedule found in Simulated Annealing (SA) algorithm.

In the local version, the particle’s new position will be restricted in the boundary of all past best solution of all particles ( ). Through local SRS, the perturbed particles will start solution searching in the reduced range. This will increase efficiency of solution searching for particles. Similarly, in the global version, the particles’ new positions will be restricted in the search boundary as an initial state ( ). Through global SRS, the perturbed particles are randomly distributed in the initial search range. It will increase the probability of finding potential solutions in un-searched areas. The local version can share particles’ searching ranges to make solution searching for particles more efficient. The global version can prevent solution from being trapped in the local optimum.

] , [pbestmin pbestmax

] , [X_min X_max

TABLE I

SEARCHRANGESHARINGSTRATEGY

Local version Global version

For Particles

To perturb selected particle and place them in reduced

range_{[ , ]}

max minpbest

pbest

To perturb particle and place them in initial range

] , [X_min X_max For Dimensions To perturb selected dimension (d1) of current particle to another selected dimension (d2)’s searching range_{[ , ]} max min 2 2 d d pbest pbest To Perturb selected dimension (d1) of current particle to another selected dimension (d2)’s searching range[Xmin,Xmax]

The SRS not only can share searching range between particles but also share searching range between dimensions. For particles, the particles’ current positions (solution) for all dimensions will be perturbed. This particle will be placed at a new position in the restricted boundaries while the SRS is activated. For dimensions, a dimension will be picked up randomly; all particles’ corresponding dimensions will be

1780 2008 IEEE Congress on Evolutionary Computation (CEC 2008)

perturbed and restricted as another dimension’s past best solution for all particles. For example, for a dimension’s local SRS, a randomly selected dimension (d1) will incorporate another randomly selected dimension (d2)’s searching range, the corresponding d1 in the particles i will be perturbed in the range between , where and are the minimal and maximal past best solution respectively of corresponding d2 of all particles. The dimension’s SRS will ignore other dimensions’ but reconfigure solutions of dimensions one by one in the particles. The SRS strategy can be summarized in Table I.

] ,

[pbestd2_min pbestd2_max pbestd2min

max

2

d

pbest

Fig. 2 Flowchart of the EPUS-PSO.

i = i+1 g = g+1 Population Manager Y Y N Y

Initial particle’s position, associated position v, pbest and gbest of the population, set g = 1, cd= 0, ci= 0

rand > Pr

i = 1

Range Sharing

Perturb the information of the random selected dimension or all

dimensions of current particle

Solution Sharing Execute (6) and (2) Pbesti= xi gbest = xi cd= cd+1 ci= 0 i<s ci= ci+1 cd= 0 cd>1 s = s-1 cd= 0 ci>1 s = s+1 ci= 0 s<max_ps g<max_gen End s>1 s=max_ps N Y Y N Y N Y Y N Y N N N N Y N

max_ps: maximum population size s: current population size pbesti: ithparticle’s past best solution

gbest: global best soltution

g: generation counter from 1 to max_gen ci: counter for increasing particle of elitism

cd: counter for decreasing particle of elitism

i: particle’s id from 1 to s xi:ithpartuicle’s value

: population manager Fit(xi)<Fit(gbest)

Fit(xi)<Fit(pbest)

Similar to the mutation rate for of GAs, different SRS rate will affect EPUS-PSO’s performance directly. After numerous generations, particles are gathering in several clusters, and therefore the demands of SRS will also be increasing. Due to a lower SRS activating rate may not be able to rescue particles that were trapped into local minimal, and a higher one would interference particles’ convergence deeply. To keep better solutions will be found efficiently and also to avoid particles perform local search. The concept of linearly variation inertia weight for PSO is adopted for the SRS rate. The activating probability of SRS for each generation is defined as following:

max_gen g g

Pr( ) 0.03 0.07* ₍₈₎

where g denotes the generation number and max_gen is maximum generations. The probability of SRS will keep increased linearly, from 0.03 to 0.1, while solution searching is in progress. Note that if each dimension has different bounds by strict constraint, the SRS can just share searching range between particles but dimensions.

D. EPUS-PSO Algorithm

The complete flowchart of the EPUS-PSO is given in Fig. 2. After initializing all the parameters of EPUS-PSO, each particle will be conducted by either solution sharing or range sharing strategy. Which sharing strategy will be chosen is dependent on Pr. For each particle i, a random number will be generated for it. If this number is larger than Pr, the sharing strategy will be solution sharing; otherwise it will be range sharing. The fitness evaluation will then update the particle’s

pbest and gbest. After all particles in the population have

moved, the population manager will determine the elimination of a redundant particle with poor performance, the hiring a new one, or the maintenance of the current population size according to the solution searching status. All particles in the population will perform the next iteration after the population manager. The clamping for restrict particles to stay in the searching space is not necessary duo to the poor particles will be eliminated by the population manager. Note that each update step (particle’s movement) in the EPUS-PSO is performed on a full N-dimensional vector. This will save a considerable amount of computational consumption when performing each particle’s movement.

2008 IEEE Congress on Evolutionary Computation (CEC 2008) 1781