局部最小值問題為類神經網路研究中之重點議題。本研究將傳統倒傳遞類神經網 路、多項式神經網路以及所提出之方法分別應用於多筆不同類型之訊號學習上；由第 四章的實驗結果得知，傳統倒傳遞類神經網路與多項式神經網路兩者之結果皆存在落 入局部極小值的問題，而需以多次嘗試錯誤的方式尋找相對較佳之結果，儘管如此，
亦無法驗證是否已得到該系統之最佳解；相較於前述兩種神經網路，本研究所提出之 機制不僅在實際訊號應用上可獲得較佳之表現外，更可明確檢視多項式神經網路於現 存架構中是否已落入局部最小值。因此，本研究提出之機制相信可大大幫助類神經網 路跳脫局部最小值，進而達到網路最佳學習之目的，使得類神經網路於日後之各類實 際應用時更能提升其應用之準確性。
本論文之未來研究方向包含以下兩部分；第一，利用本研究所發展之學習機制與 多項式神經網路間相通之數學關係，回過頭來找出多項式神經網路之最佳化模型。第 二，應用於各類實際訊號以驗證此方法之實用性與一般性，提供從事類神經網路研究 人員一重要之參考指標。
參考文獻
[1] S. Chen, S. Billings, and P. Grant, “Non-linear System Identification Using Neural Networks”, International Journal of Control, Vol. 51, pp. 1191-1214, 1990.
[2] A. Khotanzad, R. C. Hwang, A. Abaye, and D. Maratukulam, “An Adaptive Modular Artificial Neural Network: Hourly Load Forecaster and Its Implementation at Electric Utilities”, IEEE Transactions on Power Systems, Vol. 10, pp. 1716-1722, 1995.
[3] Vermaak J. and Botha E. C., “Recurrent Neural Networks for Short Term Load Forecasting,” IEEE Trans. On Power Systems, Vol. 13, No. 1, pp. 126-132, 1998.
[4] B. Zhang, M. Fu, H. Yan, and M. A. Jabri, “Handwritten Digit Recognition by Adaptive-Subspace Self-Organizing Map (ASSOM),” IEEE Trans. on Neural Networks , Vol. 10, No. 4, 1999.
[5] C. Wohler and J. K. Anlauf, “An Adaptive Time-Delay Neural-Network Algorithm for Image Sequence Analysis”, IEEE Transactions on Neural Networks, Vol. 10, pp.
1531-1536, 1999.
[6] Huang H. C., Hwang R. C. and Hsieh J. G., “A New Artificial Intelligent Peak Power Load Forecaster Based on Non-Fixed Neural Networks,” International Journal of Electrical Power and Energy Systems, Vol. 24, Issue 3, pp. 245-250, March 2002.
[7] Sun, Junping, Grosky, William I.; Hassoun, Mohamad H., “Heuristic and Hybrid Methods for Finding he Global Minimum of the Error Function in Artificial Neural Networks”, Modeling and Simulation, Proceedings of the Annual Pittsburgh Conference, Vol. 21, No. Part 2, pp. 521-525, 1990.
[8] B. Cetin, J. Burdick, and J. Barhen, “Global Descent Replaces Gradient Descent to Avoid Local Minima Problem in Learning with ANN,” Proceedings of IEEE International Conference on Neural Network, Vol. 2, pp. 836-842, 1993.
[9] X. H. Yu and G. A. Chen, “On the Local Minima Free Condition of Backpropagation Learning,” IEEE Transactions on Neural Networks, Vol. 6, pp. 1300-1303, 1995.
[10] Shang, Yi, Wah, Benjamin W., “Global Optimization for Neural Network Training”, Computer, Vo. 29, No. 3, pp. 45-54, Mar 1996.
[11] Lawrence, Steve, Tsoi, Ah Chung; Giles, C.Lee, “Local Minima and Generalization”, IEEE International Conference on Neural Networks, Vol. 1, pp. 371-376, 1996.
[12] D. S. Huang, “The Local Minima-Free Condition of Feedforward Neural Network for Outer-Supervised Learning,” IEEE Transactions on System, Man, and
Cybernetics-Part B: Cybernetics, Vol. 28, No. 3, pp. 447-480, June 1998.
[13] K. Fukumizu and S. I. Amari, “Local Minima and Plateaus in Multilayer Neural Networks,” Artificial Neural Networks, Conference Publication No. 470, pp. 597-602, Sept., 1999.
[14] I. Sprinnhuizen-Kuyper and E. Boers, “A Local Minimum for the 2-3-1 XOR Network,” IEEE Transactions on Neural Networks, Vol. 10, No. 4, pp. 968-971, 2002.
[15] Erdogmus, Deniz, Fontenla-Romero, Oscar; Principe, Jose C.; Alonso-Betanzos, Amparo; Castillo, Enrique, “Linear-Least-Squares Initialization of Multilayer Perceptrons Through Backpropagation of the Desired Response”, IEEE Transactions on Neural Networks, Vol. 16, No. 2, pp. 325-337, March 2005.
[16] Kryzhanovsky, Boris, Magomedov, Bashir; Fonarev, Anatoly, “On the Probability of Finding Local Minima in Optimization Problems”, IEEE International Conference on Neural Networks-International Joint Conference on Neural Networks 2006, p 3243-3248, 2006.
[17] Hu, Xiaolin; Wang, Jun, “A Recurrent Neural Network for Solving Nonconvex Optimization Problems”, IEEE International Conference on Neural Networks (IJCNN’06), pp. 4522-4528, 2006.
[18] Li, Hong-Ru, Li, Hai-Long, “A Global Optimization Algorithm Based on Filled-Function for Neural Networks”, Dongbei Daxue Xuebao/Journal of Northeastern University, Vol. 28, No. 9, pp.1247-1249, September 2007.
[19] Cho, Yong-Hyun, Hong, Seong-Jun, “An Efficient Global Optimization of Neural Networks by Using Hybrid Method”, Proceedings of the Frontiers in the Convergence of Bioscience and Information Technologies, FBIT 2007, pp. 807-812, 2007.
[20] Atakulreka, Akarachai, Sutivong, Daricha, “Avoiding Local Minima in Feed Forward Neural Networks by Simultaneous Learning”, Advances in Artificial Intelligence - 20th Australian Joint Conference on Artificial Intelligence, Lecture Notes in Computer Science, Vol. 4830 LNAI, pp. 100-109, 2007.
[21] Hayakawa, Yoshihiro ; Nakajima, Koji, “Parameter Analysis for Removing the Local Minima of Combinatorial Optimization Problems by Using the Inverse Function Delayed Neural Network,” Lecture Notes in Computer Science, Vol. 5506 LNCS, No.
PART 1, pp. 875-882, 2009, Advances in Neuro-Information Processing - 15th International Conference, ICONIP 2008.
and Soft Computing - ICAISC 2008 -9th International Conference, Lecture Notes in Computer Science, Vol. 5097 LNAI, pp. 89-100,2008.
[23] Ninomiya, H., “A Hybrid Global/Local Optimization Technique for Robust Training of Microwave Neural Network Models”, IEEE Congress on Evolutionary Computation, 2009, pp. 2956 – 2962, 18-21 May 2009.
[24] 葉怡成, ”類神經網路模式應用與實作, ”
儒林圖書有限公司
, 1998.[25] 龔志剛, ”類神經網路於邏輯電路之設計應用, ”
私立義守大學電機工程研究所碩 士論文
,2008.[26] Hagan, Demuth, and Beale, “Neural Network Design,” PWS, 1996.
[27] A. J. Patrikar, Neural Network Paradigms for Adaptive Signal Processing and Control, Ph.D. Thesis, Southern Method University, Texas, U.S.A, 1992.
[28] 黃銘湖, ”基於 FPGA 架構之主動式噪音控制, ”