國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER 5 CONCLUSION & FUTURE WORK
This study proposes a DSM for outlier detection problem with envelope module and moving window to adapt to concept drifting environment, especially as an unsupervised method. Since the side-effect of outlier, there is necessary to detect the suspicious patterns. Especially, in the changing environment, this outlier detection task has become more difficult. Hence, we adopt the incremental learning strategy to cope with changing environment problem. In contrast to the DSM proposed by Huang et al. (2014), we adopt not only resistant learning by envelope module but also incremental learning via moving window to handle with outlier detection in concept drifting environment. In our proposed DSM, both envelope module’s parameters and moving window’s type can be adjusted to the data nature or user requirement.
In this study, we also elaborate our proposed DSM in detail and develop the illustrative experiment for justifying and evaluating this DSM. The experiment results show a good performance. With the 95th simulation set result or other simulation sets’
results, we have confidence to adopt moving window mechanism to deal with concept drifting environment well. The result of detecting zero-day attack also shows a good performance.
The main contributions of this study are summarized below.
1. Efficiency aspect: Provide a DSM for decision maker to save lots of time.
- The experiment result shows that the amount of outlier candidates is approximately 12.4% of total instances. (2401 20000� )
- With this result, we are confident on providing a well performance DSM
62
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
for decision maker to work efficiently with outlier detection in concept drifting environment.
- Under some conditions, we can improve the decision support effectiveness with outlier candidate list and illustration figure. Provide visual evaluation method for straightforward comparison of the outlier candidates vs. non-outliers’ difference. With this visual evaluation method also can help decision maker observe the data’s trend easy to make decision intuitively.
2. Effectiveness aspect: Improve the decision support effectiveness with outlier candidate list.
- This proposed DSM performs well with this experiment data. We think this DSM can improve the effectiveness on outlier detection problem.
- In our experiment for this proposed DSM, there are 281 theoretical outliers, and 275 ones has been detected as outlier candidate. The proportion is about 98%.
- All of the zero-day attacks have been detected successfully in our experiment, and the proportion of zero-day attacks that been identified as outlier candidate is 45%.
3. Integrate resistant learning with incremental learning for dealing the outlier detection problem in concept drifting environment.
- Integrate the incremental learning strategy via implementing moving window can help us dealing the data expiration problem. Due to the data expiration problem, we can’t build the model with whole historical data.
63
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
For this issue, we set up the experiment where concept has changed to run this proposed DSM, and we also get a well performance about this issue in our experiment.
In this study, this work still has several research limitations and future goals as following:
1. Set up a real-world experiment.
- Due to this primitive justifying the proposed DSM, we have taken the total 100 simulation sets by geometric Brownian motion models. In addition, we have fixed some parameters’ value in our experiment. So when we set up a real-world experiment regarding the network security applications should adjust some parameters’ value, even the moving window’s type.
- Consider the impact on the side-effect from outliers for adjusting this DSM. In some cases, real outliers’ side-effect doesn’t make a harmful impact on us. Due to the specific feature, we can adjust the measurement about distinction between the outliers and non-outliers.
- In the ear of big data, outlier detection is considered to be the pre-processing step by all steps of data mining. Furthermore, it’s widespread belief that high dimensional data analysis has become an urgent need in this era.
2. Parallelize this DSM.
- Owing to the computation requirement in real-world experiment, it’s urgent need to parallelize it to improve the computational efficiency. Due
64
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
to the massive amount of data, it’s necessary to parallelize this DSM.
3. Apply to another type of concept drifting solution.
- During the various data nature, future research can build different types of concept drifting solution based on the data nature and applications.
4. Apply this DSM with other method as an ensemble.
- In some cases, this DSM can associate with other techniques as an ensemble to deal with other type concept drifting problems or solve some specific problems.
65
‧
1. Babcock, B., Datar, M., & Motwani, R. (2002). Sampling from a moving window over streaming data. In Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms Society for Industrial and Applied Mathematic, 633-634.
2. Babu, S., & Widom, J. (2001). Continuous queries over data streams. ACM Sigmod Record, 30(3), 109-120.
3. Banerjee, A. (2012). Density-based evolutionary outlier detection. In Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference companion, 651-652.
4. Barnett, V., & Lewis, T. (1994). Outliers in statistical data (Vol. 3), Wiley, New York.
5. Basu, S., & Meckesheimer, M. (2007). Automatic outlier detection for time series: an application to sensor data. Knowledge and Information Systems, 11(2), 137-154.
6. Bezdek, J. C. (1994). What is computational intelligence? , Computational Intelligence: Imitating Life, 1-12.
7. Bifet, A., Gama, J., Pechenizkiy, M., & Zliobaite, I. (2011). Handling concept drift: Importance, challenges and solutions. PAKDD-2011 Tutorial, Shenzhen, China.
8. Bilge, L., & Dumitras, T. (2012). Before we knew it: an empirical study of zero-day attacks in the real world. In Proceedings of the 2012 ACM conference on Computer and communications security, 833-844.
9. Buschermöhle, A., Schoenke, J., & Brockmann, W. (2012). Uncertainty and Trust
66
‧
Estimation in Incrementally Learning Function Approximation. In Advances on Computational Intelligence (pp. 32-41). Heidelberg: Springer Berlin.
10. Castelo-Fernández, C., De Rezende, P. J., Falcão, A. X., & Papa, J. P. (2010).
Improving the accuracy of the optimum-path forest supervised classifier for large datasets. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (pp. 467-475). Heidelberg: Springer Berlin.
11. Chen, C., & Liu, L. M. (1993). Forecasting time series with outliers. Journal of Forecasting, 12(1), 13-35.
12. Cook, R. D., & Weisberg, S. (1982). Residuals and influence in regression.
London: Chamman and Hall.
13. Crawford, K. D., & Wainwright, R. L. (1995). Applying Genetic Algorithms to Outlier Detection. In ICGA, 546-550.
14. Elwell, R., & Polikar, R. (2011). Incremental learning of concept drift in nonstationary environments. Neural Networks, IEEE Transactions on, 22(10), 1517-1531.
15. Ferdousi, Z., & Maeda, A. (2006). Unsupervised outlier detection in time series data. In Data Engineering Workshops, 2006. Proceedings. 22nd International Conference on IEEE, x121-x121.
16. Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys (CSUR), 46(4), 44.
17. Hawkins, D. M. (1980). Identification of outliers (Vol. 11), London: Chapman and Hall.
18. Hawkins, S., He, H., Williams, G., & Baxter, R. (2002), Outlier detection using replicator neural networks, Warehousing and Knowledge Discovery (pp.
170-180). Berlin Heidelberg: Springer.
67
‧
19. He, H. (2011). Self-adaptive systems for machine intelligence. John Wiley &
Sons.
20. Hodge, V. J., & Austin, J. (2004). A survey of outlier detection methodologies.
Artificial Intelligence Review, 22(2), 85-126.
21. Huang, S. Y., Yu, F., Tsaih, R. H., & Huang, Y. (2014). Resistant learning on the envelope bulk for identifying anomalous patterns. In Neural Networks (IJCNN), 2014 International Joint Conference on, 3303-3310.
22. Joo, D., Hong, T., & Han, I. (2003). The neural network models for IDS based on the asymmetric costs of false negative errors and false positive errors. Expert Systems with Applications, 25(1), 69-75.
23. Krawczyk, B., & Woźniak, M. (2014). One-class classifiers with incremental learning and forgetting for data streams with concept drift. Soft Computing, 1-14.
24. Lanquillon, C., & Renz, I. (1999). Adaptive information filtering: Detecting changes in text streams. In Proceedings of the eighth international conference on Information and knowledge management, 538-544.
25. Lin, H. C. (2013), ‘An Application of Streaming Data Analysis on TAIEX Futures’, Unpublished Master dissertation, Natioal Cheng-chi University, Taipet , TW.
26. Maggi, F., Robertson, W., Kruegel, C., & Vigna, G. (2009). Protecting a moving target: Addressing web application concept drift. In Recent Advances in Intrusion Detection (pp. 21-40). Springer Berlin Heidelberg.
27. Masud, M. M., Chen, Q., Khan, L., Aggarwal, C., Gao, J., Han, J., &
Thuraisingham, B. (2010). Addressing concept-evolution in concept-drifting data streams. In Data Mining (ICDM), 2010 IEEE 10th International Conference on, 929-934.
68
‧
Classification and novel class detection in concept-drifting data streams under time constraints. Knowledge and Data Engineering, IEEE Transactions on, 23(6), 859-874.
29. Navvab Kashani, M., Aminian, J., Shahhosseini, S., & Farrokhi, M. (2012).
Dynamic crude oil fouling prediction in industrial preheaters using optimized ANN based moving window technique. Chemical Engineering Research and Design, 90(7), 938-949.
30. Olson, D. L., & Shi, Y. (2007). Introduction to business data mining. Englewood Cliffs: McGraw-Hill/Irwin.
31. Rousseeuw, P. J., & Van Driessen, K. (2006). Computing LTS regression for large data sets. Data mining and knowledge discovery, 12(1), 29-45.
32. Sendhoff, B., Körner, E., Sporns, O., Ritter, H., & Doya, K. (Eds.). (2009).
Creating Brain-Like Intelligence: from basic principles to complex intelligent systems (Vol. 5436). Springer Science & Business Media.
33. Song, J., Takakura, H., & Kwon, Y. (2008). A generalized feature extraction scheme to detect 0-day attacks via IDS alerts. In Applications and the Internet, 2008. SAINT 2008. International Symposium on (pp. 55-61). IEEE.
34. Srinoy, S. (2007). Intrusion detection model based on particle swarm optimization and support vector machine. In Computational Intelligence in Security and Defense Applications, 2007. CISDA 2007. IEEE Symposium on , 186-192.
35. Stanley, K. O. (2003). Learning concept drift with a committee of decision trees.
Informe técnico: UT-AI-TR-03-302, Department of Computer Sciences, University of Texas at Austin, USA.
69
‧
36. Storkey, A. (2009). When training and test sets are different: characterizing learning transfer. Dataset shift in machine learning, 3-28.
37. Sykacek, P. (1997). Equivalent error bars for neural network classifiers trained by Bayesian inference. In ESANN.
38. Tolvi, J. U. S. S. I. (2002). Outliers and Predictability in Monthly Stock Market Index Returns. Liiketaloudellinen aikakauskirja, 369-380.
39. Tsaih, R. H., & Cheng, T. C. (2009). A resistant learning procedure for coping with outliers. Annals of Mathematics and Artificial Intelligence, 57(2), 161-180.
40. Tsay, R. S. (2014). An Introduction to Analysis of Financial Data with R., Wiely.
41. Tsymbal, A. (2004). 'The problem of concept drift: definitions and related work'.
Computer Science Department, Trinity College Dublin.
42. Wang, H., Fan, W., Yu, P. S., & Han, J. (2003). Mining concept-drifting data streams using ensemble classifiers. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, 226-235.
43. Warren S. (1983), Cubic Clustering Criterion, SAS Technical Report, A-108, SAS Institute Inc., Wiley.
44. Widmer, G., & Kubat, M. (1996). Learning in the presence of concept drift and hidden contexts. Machine learning, 23(1), 69-101.
45. Windham, M. P. (1995). Robustifying model fitting. Journal of the Royal Statistical Society. Series B (Methodological), 599-609.
46. Wrótniak, K., & Woźniak, M. (2013). Combined Bayesian Classifiers Applied to Spam Filtering Problem. In International Joint Conference CISIS’12-ICEUTE´
12-SOCO´ 12 Special Sessions (pp. 253-260). Springer Berlin Heidelberg.
47. Yoon, K. A., Kwon, O. S., & Bae, D. H. (2007). An approach to outlier detection of software measurement data using the k-means clustering method. In Empirical
70
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Software Engineering and Measurement, 2007. ESEM 2007. First International Symposiu, 443-445.
48. Zimek, A., Campello, R. J., & Sander, J. (2014). Ensembles for unsupervised outlier detection: challenges and research questions a position paper. ACM SIGKDD Explorations Newsletter, 15(1), 11-22.
71