In order to obtain better performances in the classification of land cover types in hyperspectral images, a proper method of MCS is employed to reduce the curse of dimensionality and thus increase the classification accuracy. The core of the presented method, KDSM, is the proposed optimal kernel-based dynamic subspace method which applies both an optimal algorithm of selecting the proper RBF parameter and a dynamic subspace method in the subspace selection based MCS to improve its result of classification in high dimensional dataset. Some summary of this study was stated according to the chapter as follows.
In Chapter 2, ensemble methods, Support vector machine and optimal kernel methods are introduced which are the previous works for Chapters 3 and 4. After that, a new multiple classifier system named optimal kernel-based dynamic subspace method for constructing ensemble classifiers is proposed for mitigating the Hughes effect (small training sample problem) and improving high dimensional data classification performances in Chapter 3. Finally, after statement of experimental data and design are presented in Chapter 4 then some experimental results and findings were caught in this part.
According this structure of composition above, it is clearly that this robust study,
“combining ensemble technique of support vector machines with the optimal kernel method for high dimensional data classification”, was obtained through with this complete plan. Additionally, the experimental results display that the classification accuracies of KDSM invariably are the best among outcomes of all classifiers in each cases of hyperspectral image dataset. Moreover, these results show that comparing with DSM, the KDSM can not only obtain more accurate outcome of
classification but also economize on computer time.
The experimental results show that KDSM is a robust classification method with very competitive performance for high dimensional data. Based on the findings, there are several suggestions of future work are listed in the following.
1. Feature extraction is another good method for mitigating the Hughes effect.
Try to integrate feature extraction into multiple classifier system to scale new heights in classification result in the future.
2. In this kernel-based methods, focusing on the valued issue how to find an adaptive kernel function for the reference data to create a proper composite kernel function for each subspace dataset which combined by different features is one worth diligent work.
3. Integrating a spatial-contextual support vector machine concerns about both spectral and spatial contextual information with this MCS for increasing the information on the classifiers.
4. The technique of semi-supervised classification can be introduced in this method for increasing the number of training sample in each classifier.
5. Otherwise, the most direct method for obtaining the higher accuracy than all methods in this study is to attempt on getting the best union of all classifiers in KDSM.
6. Finally, in order to urge this method to be more comprehensive, we shall use other high dimensional data with huge dimensionality such as face recognition data to test the completeness and suitability of the multiple classifier systems.
APPENDIX A: THE TEST OF “SECTOR” UNIT
REFERENCES
Banfield, R. E., Hall, L. O., Bowyer, K. W., & Kegelmeyer, W. P. (2007). A Comparison of Decision Tree Ensemble Creation Techniques, IEEE Transaction on Pattern Analysis and Machine Intelligence, 29(1), 173-180.
Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classi_cation algorithms: Bagging, boosting and variants. Machine Learning, 36, 105-142.
Bellman, R. E. (1961). Adaptive Control Processes. Princeton, NJ: Princeton Univ.
Press.
Bernardo, J. M., & Smith, A. F. M. (1994). Bayesian theory. New York, NY: Wiley.
Bertoni, A., Folgieri, R., & Valentini, G. (2004a). Feature Selection Combined with Random Subspace Ensemble for Gene Expression based Diagnosis of Malignancies. Paper presented at the meeting of the 15th Italian Workshop on Neural Nets, Perugia, Italy.
Bertoni, A., Folgieri, R., & Valentini, G. (2004b). Random subspace ensembles for the bio-molecular diagnosis of tumors, in Proc. NETTAB Workshop on Models and Metaphors from Biology to Bioinformatics Tools, 2004.
Boser, B. E., Guyon, I. M. & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 144-152.
Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123-140.
Bruzzone, L. & Persello, C. (2009). A Novel Context-Sensitive Semisupervised SVM Classifier Robust to Mislabeled Training Samples, IEEE Trans. Geosci.
Remote Sens., 47(7), 2142-2154.
Buntine, W. (1990). A theory of learning classi_cation rules. Doctoral dissertation, School of Computing Science, University of Technology, Sydney, Australia.
Camps-Valls, G. & Bruzzone, L. (2005). Kernel-based methods for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens., 43 (6), 1351–1362.
Camps-Valls, G. & Bruzzone, L. (2009). Kernel Methods for Remote Sensing Data Analysis. John Wiley & Sons, Ltd.
Camps-Valls, G., Gómez-Chova, L., Calpe, J., Soria, E., Martín, J. D., Alonso, L. &
Moreno, J. (2004). Robust support vector method for hyperspectral data classification and knowledge discovery, IEEE Trans. Geosci. Remote Sens., 42 (7), 1530–1542.
Camps-Valls, G., Gomez-Chova, L., Munoz-Mari, J., Rojo-Alvarez, J. L. &
Martinez-Ramon, M. (2008). Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection, IEEE Trans. Geosci. Remote Sens., 46 (6), 1822–1835.
Chang, C.C. & Lin, C.J. (2001). LIBSVM: A Library for Support Vector Machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chen, B., Liu, H. & Bao, Z. (2008). Optimizing the Data-dependent Kernel under A Unified Kernel Optimization Framework, Pattern Recognition, 41(6), 2107-2119.
Chen, I.-L., Li, C.-H., Kuo, B.-C. & Huang, H.-Y. (2010). Applying Optimal Algorithm to Data-dependent Kernel for Hyperspectral Image Classification, Proceedings of International Geosciences and Remote Sensing Symposium, 2808-2811
Christopher, J. W. (2004). Hyperspectral image classification with limited trainingdata samples using feature subspaces, Proc. SPIE, 5425, 170–181.
Chuang, C.-H., Kuo, B.-C. & Wang, H.-P. (2008). Fuzzy fusion method for combining small number of classifiers in hyperspectral image classification, Proc. 8th Int. Conf. Intell. Syst. Des. Appl., 1, 26–28.
Cortes, C., & Vapnik, V. (1995). Support vector networks. Machine Learning, 20, 273–297.
Dietterich, T. G. (2000). Ensembles Methods in Machine Learning. Lecture Notes in Computer Science, 1857, 1-15.
Drucker, H., Cortes, C., Jackel, L. D., LeCun, Y., & Vapnik, V. (1994). Boosting and other machine learning algorithms. Proceedings of the Eleventh International Conference on Machine Learning, 53-61.
Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm.
Proceedings of the Thirteenth International Conference on Machine Learning, 148-156.
Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. (2nd ed.). San Diego, CA: Academic.
Ham, J., Chen, Y., Crawford, M. M. & Ghosh, J. (2005). Investigation of the random forest framework for classification of hyperspectral data, IEEE Trans. Geosci.
Remote Sens., 43(3), 492–501.
Ho, T. K. (1998a). The Random Subspace Method for Constructing Decision Forests.
IEEE Transaction on Pattern Analysis and Machine Intelligence, 20(8), 832-844.
Ho, T. K. (1998b). Nearest Neighbors in Random Subspaces. Proceedings of 2nd International Workshop Statistical Techniques in Pattern Recognition, 640-648.
Hughes, G. F. (1968). On the Mean Accuracy of Statistical Pattern Recognition.
IEEE Transaction on Information Theory, IT-14, 55-63.
John, S. T. & Nello, C. (2004). Kernel Methods for Pattern Analysis. Cambridge University Press.
Kong, E. B., & Dietterich, T. G. (1995). Error-correcting output coding corrects bias and variance. Proceedings of the Twelfth International Conference on Machine Learning, 313-321.
Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms.
Hoboken, NJ: Wiley & Sons.
Kuo, B.-C., Hsieh, Y.-C., Liu, H.-C., & Chao, R.-M. (2005). A Random Subspace Method with Automatic Dimensionality Selection for Hyperspectral Image Classification. Proceedings of International Geoscience and Remote Sensing Symposium, 25-29.
Kuo, B.-C., Pai, C.-H., Sheu, T.-W., & Chen, G.-S. (2004). Hyperspectral Data Classification using Classifier Overproduction and Fusion Strategies, Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 5, 2937-2940.
Landgrebe, D.A. (2003), Signal Theory Methods in Multispectral Remote Sensing, John Wiley and Sons, Hoboken, NJ: Chichester.
Li, C.-H., Lin, C.-T., Kuo, B.-C. & Chu, H.-S. (2010). An automatic method for selecting the parameter of the RBF kernel function to support vector machines, Proceedings of International Geosciences and Remote Sensing Symposium, 836-839.
Maclin, R., & Opitz, D. (1997). An empirical evaluation of bagging and boosting.
Proceedings of the Fourteenth National Conference on Arti_cial Intelligence.
Providence, RI: AAAI Press.
Melgani, F. & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans. Geosci. Remote Sens.,42 (8), 1778–1790.
Neal, R. M. (1993). Probabilistic inference using Markov chain Monte Carlo methods (Technical Report CRG-TR-93-1). Department of Computer Science, University of Toronto, Toronto, Canada.
Parzen, E. (1962). On Estimation of a Probability Density Function and Mode. Annals of Mathematical Statistics, 33(3), 1065-1076.
Quinlan, J. R. (1996). Bagging, boosting, and C4.5. Proceedings of the Thirteenth National Conference on Arti_cial Intelligence, 725-730.
Raudys, S. J. & Jain, A. K. (1991). Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners. IEEE Transaction on Pattern Recognition and Machine Intelligence, 13(3), 252-264.
Silverman, B. W. (1985). Density Estimation for Statistics and Data Analysis, London, UK: Chapman & Hall.
Skurichina, M. & Duin, R. P. W. (2001). Bagging and the Random Subspace Method for Redundant Feature Spaces. Proceeding of 2nd International Workshop Multiple Classifier Systems, 1-10.
Skurichina, M. & Duin, R. P. W. (2002). Bagging, Boosting and the Random Subspace Method for Linear Classifiers. Pattern Analysis and Applications, 5(2), 121-135.
Sun, S., Zhang, C., & Zhang, D. (2007). An Experimental Evaluation of Ensemble Methods for EEG Signal Classification. Pattern Recognition Letters, 28(15), 2157-2163.
Tao, D., Tang, X., Li, X., & Wu, X. (2006). Asymmetric Bagging and Random Subspace for Support Vector Machines-based Relevance Feedback in Image Retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1088-1099.
Tarabalka, Y., Benediktsson, J. A. & Chanussot, J. (2009). Spectral–Spatial Classification of Hyperspectral Imagery Based on Partitional Clustering Techniques. IEEE Trans. Geosci. Remote Sens., 47(8), 2973-2987.
Xiong, H. L., Swamy, M. N. S., Omair, M. & Ahmad. (2005). Optimizing the kernel in the empirical feature space, IEEE Trans. Neural Networks 16 (2) 460–474.
Yang, J-M., Kuo, B-C., Yu,P-T. & Chuang, C-H. (2010). A Dynamic Subspace Method for Hyperspectral Image Classification, IEEE Transaction on Geosciences and Remote Sensing, 48(7), 2840-2853.