許多的研究證實，組間分散矩陣和組內分散矩陣的定義以及正規化技術是 在設計一個小樣本問題的關鍵。在本研究中，闡明了特徵值分解是另一個重要 的關鍵。在高維度影像資料中，這三個特徵萃取的效果分別在 ill-posed 以及 poorly-posed 兩種小樣本情況下被探討。
由實驗結果可以得到以下：
1. 加入特徵值分解對於特徵萃取是有所改善的，不同的正規化方法需要配合不 同的特徵值分解，以達到最佳的效能。
2. RFE 較其他的正規化技術更加有效果。
3. NWFE 形式的分散矩陣在 ill-posed 以及 poorly-posed 下均較 LDA 形式的分 散矩陣穩定。
4. 在小樣本辨識問題中最佳的特徵萃取組合為 RFE 正規化 NWFE 使用 EIG 或 QZ 分解。
圖5-1 Washington DC Mall 資料部份的影像
圖5-2 PCA 分類結果
圖5-3 LDAGSVD 分類結果
圖5-4 LDA_PINV_EIG 分類結果
圖5-5 LDA_PINV_SVD 分類結果
圖5-6 LDA_PINV_QZ 分類結果
圖5-7 LDA_RDA_EIG 分類結果
圖5-8 LDA_RDA_SVD 分類結果
圖5-9 LDA_RDA_QZ 分類結果
圖5-10 LDA_ MECS_EIG 分類結果
圖5-11 LDA_ MECS_SVD 分類結果
圖5-12 LDA_ MECS_QZ 分類結果
圖5-13 LDA_ NLDA_EIG 分類結果
圖5-14 LDA_ NLDA_SVD 分類結果
圖5-15 LDA_ NLDA_QZ 分類結果
圖5-16 LDA_MixLOOC_EIG 分類結果
圖5-17 LDA_MixLOOC_SVD 分類結果
圖5-18 LDA_MixLOOC_QZ 分類結果
圖5-19 LDA_RFE_EIG 分類結果
圖5-20 LDA_RFE_SVD 分類結果
圖5-21 LDA_RFE_QZ 分類結果
圖5-22 NWFE_PINV_EIG 分類結果
圖5-23 NWFE _PINV_SVD 分類結果
圖5-24 NWFE _PINV_QZ 分類結果
圖5-25 NWFE _ MECS_EIG 分類結果
圖5-26 NWFE _ MECS_SVD 分類結果
圖5-27 NWFE _ MECS_QZ 分類結果
圖5-28 NWFE _NLDA_EIG 分類結果
圖5-29 NWFE _NLDA_SVD 分類結果
圖5-30 NWFE _NLDA_QZ 分類結果
圖5-31 NWFE_RFE_EIG 分類結果
圖5-32 NWFE _RFE_SVD 分類結果
圖5-33 NWFE _RFE_QZ 分類結果
圖5-34 KNN 分類結果
圖5-35 SVM 分類結果
參考文獻
Chang, C.-C., and Lin, C.-J. (2004). LIBSVM: a library for support vector machines.
Software available at http://www.csie.ntu.edu.tw/ ~cjlin/libsvm.
Chen, G-S., Ko, L-W., Kuo, B-C., and Shih, S-C. (2004). A Two-stage Feature Extraction for Hyperspectral Image Data Classification. Proceedings of International Geoscience and Remote Sensing Symposiums, Sep 20-24.
Cristianini, N. and Shave-Taylor, J. (2000). Support Vector Machines and other kernel-based learning methods, Cambridge University Press.
Cover, T. M. and Hart, P. E. (1967). Nearest Neighbor Pattern Classification. IEEE Transactions on Information Theory, vol. 13, no. 1, pp.21-27.
Fisher, R. A. (1936). The use of multiple measures in taxonomic problems. Ann.
Eugenics, vol. 7, pp. 179-188.
Friedman, J. H. (1989). Regularized discriminant analysis. Journal of the American Statistical Association, 84, 165–175.
Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. San Diego, CA:Academic Press.
Greenbaum, A. and Dongarra, J. J. (1989). Experiments with QL/QR methods for the symmetric tridiagonal eigenproblem, Computer Science Dept. Technical Report CS-89-92, University of Tennessee, Knoxville, TN.
Golub, G. H. and Loan, C. F. V. (1996). Matrix Computations. Johns Hopkins University Press, Baltimore, 3rd edition.
Hastie, T. and Tibshirani, R. (1996). Discriminant Adaptive Nearest Neighbor
Classification. IEEE Transcation on Pattern Analysis and Machine Intelligence. vol.
18, no. 6, pp. 607-616.
Hong, Z-Q., and Yang, J-Y. (1991). Optimal discriminant plane for a small number of samples. Pattern Recognition, vol. 24, no.4, pp. 317-324.
Howland, P. and Park, H. (2004). An optimization criterion for generalized
discriminant analysis on undersampled problems. IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 26, No. 8, pp. 982-994.
Hsu, C.-W., Chang, C.-C., and Lin, C.-J. (2004). A Practical Guide to Support Vector Classification. Available at http://www.csie.ntu.edu.tw/ ~cjlin/libsvm
Hughes, G. F. (1968). On the mean accuracy of statistical pattern recognition. IEEE Trans. Inform. Theory, vol. 14, pp. 55 - 63, Jan.
Jolliffe, I. T. (1986). Principal Components Analysis. Springer-Verlag, New York.
Kuo, B-C., Chang, K-Y., Chang, C-H., and Hsieh, Y-C. (2004). Hyperspectral Image Data Classification Using Feature Extractions and Support Vector Machines.
CVGIP.
Kuo, B-C. and Landgrebe, D. A. (2004). Nonparametric Weighted Feature Extraction for Classification. IEEE Trans. on Geoscience and Remote Sensing, vol. 42, no. 5, pp. 1096-1105, May.
Kuo, B-C., Landgrebe, D.A., Ko, L-W., and Pai, C-H. (2003). Regularized Feature Extractions for Hyperspectral Data Classification. Proceedings of International Geoscience and Remote Sensing Symposium, Toulouse. France, July.
Kuo B.-C. and Landgrebe D. A. (2002). A covariance estimator for small sample size classification problems and its application to feature extraction. IEEE Trans.
Geosci. Remote Sens., vol. 40, no. 4, pp. 814–819, Apr.
Landgrebe, D. A. (2003). Signal Theory Methods in Multispectral Remote Sensing.
John Wiley and Sons, Hoboken, NJ: Chichester.
Lee, C. and Landgrebe, D. A. (1993). Feature Extraction Based On Decision
Boundaries. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
15, No. 4, April, pp 388-400.
Moler, C. B. and Stewart, G. W. (1973). An algorithm for generalized matrix eigenvalue problems. SIAM Journal on Numerical Aanlysis, vol 10, pp.241–256.
Moore, E. (1920). On the reciprocal of the general algebraic matrix, Bull. Amer.
Math. Soc. 26, 394-395.
Penrose, R. (1955). A generalized inverse for matrices, Proc. Camb. Philos. Soc. 51, 406-413.
Shah, C. A., Watanachaturaporn, P., Arora, M. K., and Varshney, P. K. (2003). Some Recent Results on Hyperspectral Image Classification. In IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data, NASA Goddard Spaceflight center, Greenbelt, October 27-28.
Tadjudin S. and Landgrebe D. A. (1998). Classification of High Dimensional data with Limited Training Samples, Purdue University, West Lafayetee, IN., TR-EE 98-8, April.
Thomaz, C. E., Gillies, D. F., and Feitosa, R. Q. (2004). A New Covariance Estimate for Bayesian Classifiers in Biometric Recognition. IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Image- and Video-Based Biometrics, vol. 14, no. 2, pp. 214-223, February.
Thomaz, C. E. and Gillies, D. F. (2004). A Maximum Uncertainty LDA-based
approach for Limited Sample Size problems - with application to Face Recognition.
Technical Report TR-2004-01, Department of Computing, Imperial College, London, UK, January.
Vapnik, V. (1995). The Nature of Statistical Learning Theory. New York, NY:
Springer-Verlag.
Watkins, D. S. (1999). Infinite eigenvalues and the QZ algorithm. Technische Universitaet Chemnitz, SFB 393, SFB393-Preprint 99-23. Available on-line http://www.mathematik.tu-chemnitz.de/preprint/1999/SFB393_23.html Ye J., Janardan R., Park C. H., and Park H. (2004). Generalizing discriminant
analysis using the generalized singular value decomposition. IEEE Transactions on Pattern Analysis & Machine Intelligence, vol 26, No. 8, pp. 995-1006.
Yu, H. and Yang, J. (2001). A direct LDA algorithm for high dimensional data – with application to face recognition. Pattern Recognition, vol. 34, pp. 2067-2070.