Using Optimization Algorithms to Select Parameters of Support Vector Regression 楊舜麟、王正賢 ; 白炳豐
E-mail: 9509855@mail.dyu.edu.tw
ABSTRACT
Organized approaches in selection support vector regression (SVR) parameters are lacking. Furthermore, the determination of SVR parameters affects the prediction accuracy a lot. This research applied four algorithms, namely Ant colony system, Tabu search, Immune algorithm and Particle swarm optimization to choose SVR parameters. For each algorithms, the forecasting errors are treated as objective functions. In additional, many examples from real word are used to demonstrate the performance of the proposed models
Keywords : Support vector machine(SVM) ; Support vector regression(SVR) ; Ant colony system ; Tabu search ; Immune algorithm
; Particle swarm pptimization ; system
Table of Contents
封面內頁 簽名頁 授權書 iii 中文摘要 iv ABSTRACT v 誌謝 vi 目錄 vii 圖目錄 x 表目錄 xi 第一章 緒論 1 1.1 研究背景與動機 1 1.2 研究方法以及目的 2 1.3 研究資料 3 第二章 文獻探討 5 2.1 支援向量迴歸 5 2.2 啟發式演算法 8 2.2.1 禁忌搜尋法 8 2.2.2 螞蟻演算法 10 2.2.3 免疫演算法 11 2.2.4 粒子群體演算法 13 第三章 研究方法 16 3.1 支援向量迴歸 16 3.2 啟發式演算法結合 支援向量迴歸架構 22 3.2.1 禁忌搜尋法 23 3.2.2 免疫演算法 26 3.2.3 螞蟻演算法 28 3.2.4 粒子群體演算法 32 第四章 實例分 析 35 4.1 實例一－泰山收費站 35 4.1.1 支援向量迴歸 36 4.1.2 ARIMA方法 40 4.1.3 GRNN方法 41 4.1.4 整合比較各方法預 測結果 43 4.2 實例二－台灣電力用量 45 4.2.1 支援向量迴歸 46 4.2.2 ARIMA方法 50 4.2.3 GRNN方法 51 4.2.4 整合比較各 方法預測結果 53 4.3 實例三－系統可靠度資料 55 4.3.1 支援向量迴歸 56 4.4 實例四－Orland機場租車使用量 62 4.4.1 支援 向量迴歸 64 4.4.2 ARIMA方法 67 4.4.3 GRNN方法 68 4.4.4 整合比較各方法預測結果 70 4.5 實例五－複合材料疲勞試驗－
楊氏係數 71 4.5.1 支援向量迴歸 71 4.5.2 ARIMA方法 75 4.5.3 GRNN方法 75 4.5.4 整合比較各方法預測結果 77 第五章 結論 及未來研究 78 5.1 結論 78 5.2 未來研究 79 參考文獻 81
REFERENCES
【1】白炳豐,王正賢,雷賀君“前十字韌帶傷害快速診斷系統-以粗略集合、基因演算法與倒傳遞網路為工具”,(2004) 大葉大學工業工程 學系碩士論文 【2】陳朝順,吳家駿 ”應用免疫演算法於配電系統運轉策略之研究”,(2001)國立中山大學碩士論文 【3】PC home 電子報 / 記者:陳世煌 日期﹕民國 92 年 6 月 【4】 Armentano, Vinicius A., Debora P. Ronconi “Tabu search total tardiness minimization in flowshop scheduling problems”,Computers & Operations Research, Vol. 26,(1999)219-235. 【5】Armentano, Vinicius A., Scrich Cintia Rigao
“Tabu search for minimizing total tardiness in a job shop”, International Journal of Production Economics, Vol. 63,(2000)131-140. 【6】Clerc , M., J. Kennedy, “The particle swarm explosion, stability, and convergence in a multidimensional complex space”, IEEE Transaction on Evolutionary Computation, 6, (2002), 58-73. 【7】Dorigo, M. “Optimization, learning and natural algorithms”, (1992)Ph.D Dissertation, Dipartimento di Elettronica, Politecnico di Milano, IT, Italian 【8】Dorigo Macro, Gambardella, Luca Maria ”Ant colonies for the travelling salesman problem” ,BioSystem 43(1997)73-81 【9】Glover, F. “Tabu search─Part II” ORSA Journal on Computing, Vol. 2, (1990)4-32 【10
】Glover F. “Tabu search-Part I”, ORSA Journal on Computing(1990) 【11】Hong, Wei-Chiang, Ping-Feng Pai, Chen-Tung Chen, and Ping-Teng Chang,“Recurrent support vector machines in reliability prediction” accepted by Lecture Notes in Computer Sciences, (Ref . No.
P2810), 2005 【12】Hong, Wei-Chiang, Ping-Feng Pai, and Yu-Ying Huang, and Sheng-Lin Yang,“Application of support vector machines in predicting employee turnover based on job performance”, accepted by Lecture Notes in Computer Sciences, (Ref. No.P0537), 2005, 【13】
http://www.freeway.gov.tw/02_05_05.asp 【14】Hajela, P. and J. Lee, Constrained genetic search via schema adaptation: an immune network solution, Structural Optimization , 12, 11-15, 1996. 【15】Jia-Hroung Wu and Feng-Min Lai, Composite Materials & Structure Laboratory at National Chiao Tung University, Taiwan. 【16】John, E. Hunt and Denise E. Cooke, Learning using an artificial immune system, Journal of Network and Computer Applications. 19, 189-212,1996. 【17】Kecman, V.(Vojislav), “Learning and soft computing: support vector machines, neural networks, and fuzzy logic models”, Massachusetts Institute of Technology, pp.121-189, 2001. 【18】Kennedy, J., R. C. Eberhart,
“Particle swarm optimization”, Proceedings IEEE Int’l. Conf. on Neural Networks, IV, (1995), 1942-1948. 【19】Kennedy, J., R. C.
Eberhart, “A discrete binary version of the particle swarm algorithm”, Proceedings IEEE Int’l. Conf. on Systems, Man, and Cybernetics, (1997), 4104-4108. 【20】Lu , Weizhen, Wenjian Wang, Andrew Y T Leung, Siu-Ming Lo, Richard K K Yuen, Zongben Xu, Huiyuan Fan,
”Air Pollutant Parameter Forecasting Using Support Vector Machines”, IEEE, 2002. 【21】Luh, Guan-Chun, Chung-Huei Chueh,
“Multi-modal topological optimization of structure using immune algorithm”, Comput. Methods Appl. Mech. Engrg. 193 (2004) 4035–4055
【22】Luh, Guan-Chun and Chung-Huei Chueh, “Multi-objective optimal design of truss structure with immune algorithm”, Computers and Structures 82 (2004) 829–844 【23】Mohandes, M.A., T.O. Halawani, S. Rehman, Ahmed A. Hussain, “Support vector machines for wind speed prediction”, Renewable Energy, 29, pp. 939-947, 2004 【24】Mr. M. Blair and Y.J. Lai, Dollar Thrifty Automobile Group, USA 【25
】Pai, Ping-Feng, Chih-Shen Lin, Wei-Chiang Hong, and Chen-Tung Chen,“A hybrid support vector machine regression for exchange rate prediction”,accepted by International Journal of Information and Management Sciences,(Ref No. 688), 2005 【26】Parsopoulos , K. E., M. N.
Vrahatis, “Recent approaches to global optimization problems through particle swarm optimization”, Natural Computing, 1 (2002), 235-306.
【27】Pai, Ping-Feng, “System reliability forecasting by support vector machines with genetic algorithms”, accepted by Mathematical and Computer Modeling, (Ref No. MCM 4969), 2005, 【28】Pai, Ping-Feng and Wei-Chiang Hong, “Support vector machines with simulated annealing algorithms in electricity load forecasting'”, 2005, Volumn 46, Issue 17 , 2669-2688, Energy Conversion and Management 【29
】Singpurwalla, Nozer D., Pefik Soyer, “Assessing (Software) reliability growth using a random coefficient autoregressive process and its ramifications”, IEEE transactions on software engineering, Vol. SE-11, NO.12, pp. 1456-1464, 1985 【30】Shigenori , N., G. Takamu, Y.
Toshiku, and F. Yoshikazu, “A hybrid particle swarm optimization for distribution state estimation”, IEEE Transaction on Power Systems, 18 (2003), 60-68. 【31】Salman, A., I. Ahmad, and S. Al-Madani, “Particle swarm optimization for task assignment problem”, Microprocessors and Microsystems, 26 (2002), 363-371. 【32】Shelokar, P.S., Jayaraman V.K. and Kulkarni, B.D. ”An ant colony approach for clustering”, Analytica Chimica Acta 509 (2004) 187–195 【33】Tay , Francis E. H., Lijuan Cao, ”Application of support vector machines in financial time series forecasting”, The International Journal of Management Science, Omega 29, pp.309-317, 2001. 【34】Tay , Francis E. H., Lijuan Cao,
”Modified support vector machines in financial time series forecasting”, Neurocimputing, 48 ,pp.847-861, 2002. 【35】Trafalis, Theodore B., Huseyin Ince, “Support Vector Machine for Regression and Application to Financial Forecasting”, IEEE, 2000. 【36】Trelea, I. C., “The particle swarm optimization algorithm: convergence analysis and parameter selection”, Information Processing Letters, 85 (2003), 317-325. 【37
】Thissen , U., R. van Brakel, A.P. de Weijer, W.J. Melssen, L.M.C. Buydens, “Using support vector machines for time series prediction”, Chemometrics and Intelligent Laboratory Systems, 69, pp. 35-49, 2003. 【38】Yoshida, H., K. Kawata, Y. Fukuyama, and Y. Nakanishi, “A particle swarm optimization for reactive power and voltage control considering voltage stability”, Proceedings Int’l. Conf. on Intelligent System Application to Power Systems, (1999), 117-121.
