Performance of DDoS Detection Evaluation

Figure 4.4 depicts the false positive ratios for the four periods in a day. The result indicates that the false positive ratio ranges from 1.2% (bed time) to 2.4% (morning). In [10], the false positive ratio ranges from 1% to 8% depending on the background traffic.

However, it is not clear the amount of attack traffic in [10]. In D-WARD [13], the false positive ratio (which is called false alarm in [13]) is about 2%. However, it is clear about the amount of attack and normal traffic in [13]

62

Figure 4.4 False Positive Ratio in DDoS detection

Figure 4.5, Figure 4.6 and Figure 4.7 depict the false negative ratios of TCP SYN flooding, ICMP flooding, and UDP flooding, separately. Because the sending rates in the ICMP flooding attack and the UDP flooding attack are the same, the results in ICMP and UDP flooding attacks are similar. When the attack rate is 150 packets per second (note that in the training phase the attack rate is 150 packets per second), the false negative ratio ranges from 5% to 10% for UDP and ICMP flooding. The false negative ratio is 2% ~ 3% for the TCP SYN flooding. Because we didn’t detect the UDP\ICMP flooding attack by “# of incoming UDP packet counts” and “# of incoming ICMP packet counts” and also the attack rate for UDP\ICMP flooding is the same in test\training data, therefore the results for UDP and ICMP are look similar.

0

TCP SYN (250 pkts/sec) TCP SYN (150 pkts/sec) TCP SYN (70 pkts/sec)

Figure 4.5 False negative ratio of TCP SYN flooding attack

63

Figure 4.6 False negative ratio of ICMP flooding attack

0

Figure 4.7 False negative ratio of UDP flooding attack

Figure 4.8, Figure 4.9 and Figure 4.10 depict the results in the false classification ratios. The results show that the false classification ratio for the TCP SYN attacks is lower than that for ICMP and UDP flooding attacks. Nearly 40% to 50% of ICMP attacks may be mistaken as UDP attacks. Similarly, nearly 40% to 50% of UDP attacks may be mistaken as ICMP attacks. On the other hand, TCP SYN attacks are seldom mis-classified.

Figure 4.8 False classification ratio of TCP SYN flooding attack

64

Figure 4.9 False classification ratio of ICMP flooding attack

0

Figure 4.10 False classification ratio of UDP flooding attack

Another important issue in DDoS detection is the detection latency, that is, how soon the system will claim an attack after the attack traffic reaches the victim. In our system, time is sliced into 1-minute slots. According to the results in Figure 4.11, Figure 4.12 and Figure 4.13, our system could claim an attack within 1 to 1.4 minutes under different attack rates.

TCP SYN (250 pkts/sec) TCP SYN (150 pkts/sec) TCP SYN (70 pkts/sec)

Figure 4.11 Detection latency of TCP SYN attack

65

Figure 4.12 Detection latency of UDP attack

0

Figure 4.13 Detection latency of ICMP attack

4.2.2.2 Performance of Attacker Traceback

When reconstructing the attack paths, it is possible to mistake an edge that is not on the attack path as an attack edge and vice versa. Figure 5.14 and Figure 5.15 show MNER and MAER with different observed windows and different trend-pattern thresholds. Remember the observed window is the amount of time the sentinels collect traffic data after an attack is claimed. Figure 4.15 shows that MAER is almost a constant while Ttrend < 0.9 regardless of the observed window, which means that the attack path shows high trend value (> 0.8) and above the Ttrend. Also MNER has burst increasing while the Ttrend > 0.9 due to the too high Ttrend and the upstream sentinels wouldn’t forward the traceback command because the computed trend-pattern value is less than Ttrend. The results also verify that the grey relational analysis is suitable for small sample space (size of sample space < 30).

When we keep MAER low (that is, the Ttrend < 0.9), the lowest MNER is around

66

12% - 18.5% (from Figure 4.14). Furthermore we enforce an observed window for at least 3 minutes, MNER falls between 12% and 14%. In [38], MNER is 17% ~ 19% in old iTrace model and 6% in new proposed model under different network traffic.

MNER in [37] is less than 10% but that system makes use of a modified probability packet marking mechanism, which involves many other issues.

0

Figure 4.14 Misidentified normal edge ratio in traceback

0

Figure 4.15 Misidentified attack edge ratio in traceback

67

Chapter 5 CONCLUSIONS

In this thesis, we propose a DDoS defense system, which includes attack detection by decision tree and attacker traceback with traffic-pattern match. Our system is based on the observation that the network traffic under DDoS attack would differ from the traffic in normal situation. We apply the decision tree (C4.5) generating algorithm to construct the classification model and detect abnormal traffic flow. In traceback phase, we use a novel traffic pattern matching procedure with grey relational analysis to identify the traffic flow that is similar to the attack flow and, based on this similarity, to trace back the origin of an attack. The attack path reconstruction is then accomplished by the protection agent and the sentinels. We conduct our experiment on the DETER system.

According to our experiment results, our system could detect the DDoS attack with the false positive ratio about 1.2% ~ 2.4%, false negative ratio about 5% ~ 10% with different kind of attack, attack sending rate and find the attack path in traceback with the misidentified attack edge ratio about 8%~12% and misidentified normal edge ratio about 12%~14%. The result indicates that our proposed system is capable of detecting the attacks and tracing them back with high accuracy.

In the future we will improve our system according to the following list.

1. In our current experiment, there are only flooding-based attacks being tested.

In the future, there are other attacks such as ping of death, smurf attack, should being tested and constructed a more comprehensive attack detection system.

2. The attributes that our system provided might not be sufficient to describe

68

the attack in the future, therefore the more detailed and preciously attributes for describing the whole network environment should be provided to deal with the novel attack created in the future.

3. The large amount of packet loss that the DDoS attack generates could result in the loss of traceback command and results in the false negative. The message passing mechanism could be refined and improved in the future.

69

References

[1] Martyn Williams,CNN, ”'Immense' network assault takes down Yahoo“, February 8, 2000 Web posted,

http://archives.cnn.com/2000/TECH/computing/02/08/yahoo.a ssault.idg/index.html

[2] David Kleinbard, CNNMoney,“ eBay, Buy.com, CNN.com and

Amazon come under attack; FBI probes Yahoo! incident ”, February 8, 2000, http://money.cnn.com/2000/02/08/technology/yahoo/

[3] Puneet Zaroo,“A Survey of DDoS attacks and some DDoS defense mechanisms”,Advanced Information Assurance (CS 626)

[4] Jelena Mirkovic, Peter Reiher, “A Taxonomy of DDoS Attack and DDoS Defense Mechanisms”, ACM SIGCOMM Computer Communications Review, Apr.2004, Vol. 34, No. 2, pp. 39-54.

[5] Christos Douligeris , Aikaterini Mitrokotsa, “DDoS attacks and defense mechanisms: classification and state-of-the-art”, Computer Networks:

The International Journal of Computer and Telecommunications Networking, Volume 44, Issue 5 (April 2004), Pages: 643 - 666 [6] CERT/TW,“DDoS 與 DoS 的發展與分類”,

http://www.cert.org.tw/document/column/show.php?key=88 [7] Wikipedia, “Denail-of-services attack ",

http://en.wikipedia.org/wiki/Denial-of-service_attack [8] Wikipedia, http://en.wikipedia.org/wiki/LAND

[9] Sanguk Noh, Cheolho Lee, Kyunghee Choi, Gihyun Jung, “Detecting Distributed Denial of Service (DDoS) Attacks Through Inductive Learning”, Springer Berlin / Heidelberg publisher, Volume 2690/2003 [10] Stefan Seufert, Darragh O’Brien, “Machine Learning for Automatic

Defence against Distributed Denial of Service Attacks”,

IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings

[11] BoonPing Lim, Md. Safi Uddin, “ Statistical-based SYN-flooding Detection Using Programmable Network Processor ”,

Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05)

70

[12] Haining Wang, Danlu Zhang, and Kang G. Shin, “Change-Point Monitoring for Detection of DoS Attacks”, in IEEE Transactions on Dependable and Secure Computing, Vol. 1, No. 4, December 2004.

[13] Jelena Mirkovi´c, Gregory Prier, Peter Reiher, “Source-End DDoS Defense”, Proceedings of the Second IEEE International Symposium on Network Computing and Applications (NCA’03)

[14] Mirkovic. J, Reiher. P, “D-WARD: a source-end defense against Flooding denial-of-service attacks”, IEEE Transactions on Dependable and Secure Computing, vol. 2, no. 3, July-Sept. 2005, pp. 216-232.

[15] Stefan Savage, David Wetherall, Anna Karlin, Tom Anderson,

“Practical Network Support for IP Traceback”, August 2000, SIGCOMM '00: Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication

[16] R. Stone,“CenterTrack: An IP Overlay Network for Tracking DoS Floods”, In to appear in Proceedings of thje 2000, USENIX Security Symposium, Denver, CO, July 2000.

[17] H. Burch,“Tracing Anonymous Packets to Their Approximate

Source”, December 2000, LISA '00: Proceedings of the 14th USENIX conference on System administration

[18] T. Baba and S. Matsuda, “Tracing network attacks to their sources,”

IEEE Internet Computing, vol. 6, pp. 20-26, Mar. 2002.

[19] S. M. Bellovin, “Internet Draft: ICMP Traceback”, Mar. 2000.

[20] Tcpdump/libpcap, http://www.tcpdump.org/

[21] Burton H. Bloom, “Space/time trade-offs in hash coding with allowable errors.”, Communications of the ACM, Volume 13, Issue 7 (July 1970), Pages: 422 - 426

[22] Xiongmin Li, Christine W Chan, “Applying A Machine Intelligence Algorithm for Prediction”, Computational Intelligence and Security, 2006 International Conference, Volume: 1, On page(s): 793-796

[23] Lior Rokach, Oded Maimon, “Top-down Induction of Decision Trees Classifier – A survey”, Systems, Man, and Cybernetics, Part C:

Applications and Reviews, IEEE Transactions on, Nov. 2005 Vol: 35, page(s): 476- 487

[24] Ben Krose, Patrick van der Smagt, “An Introduction to Neural Network”, University of Amsterdam, 1996

http://www.librecours.org/documents/33/3396.pdf

71

[25] Yacine Bouzida, Fr’ed’eric Cuppens, “Neural networks vs. decision trees for intrusion detection”, IEEE / IST Workshop on Monitoring, Attack Detection and Mitigation (MonAM). Tuebingen, Germany, 28-29 September, 2006.

[26] J.R Quinlan, “Induction of Decision Trees”, Machine Learning 1:

81-106, 1986

[27] J.R Quinlan, “C4.5 : Programs for Machine Learning”, Morgan Kaufmann Publishers, San Mateo, CA, 1993.

[28] RULEQUEST RESEARCH, data mining tools, http://www.rulequest.com/index.html

[29] Yongjin Kim, Ahmed Helmy, “SWAT: Small World-based Attacker Traceback in Ad-hoc Networks”, Mobile and Ubiquitous Systems:

Networking and Services, 2005. MobiQuitous 2005. The Second Annual International Conference on, 17-21 July 2005 Page(s):85 – 96 [30] J.L. Deng, “Introduction to Grey system theory”, November 1989, The

Journal of Grey System, Volume 1 Issue 1

[31] Yi Lin, Sifeng Liu, “A historical introduction to grey systems theory” Systems, Man and Cybernetics, 2004 IEEE International Conference on,

10-13 Oct. 2004, Volume: 3, On page(s): 2403- 2408 vol.3 [32] K.H. Hsia, M.Y. Chen, M.C. Chang, “Comments on data

pre-processing for grey relational analysis” Journal of Grey System, vol.

7, no. 1, pp.15-20, 2004.

[33] T. Benzel, et al, “Experience with DETER: A Testbed for Security Research”, Second IEEE Conf. on Testbeds and Research

Infrastructures for the Development of Networks and Communities (TridentCom2006).

[34] Stephen Schwab, Brett Wilson, Calvin Ko, Alefiya Hussain, “SEER:

A Security Experimentation EnviRonment for DETER”, August 2007 DETER: Proceedings of the DETER Community Workshop on Cyber Security Experimentation and Test on DETER Community Workshop on Cyber Security Experimentation and Test 2007

[35] Bernard M. Waxman, “Routing of multipoint connections”, Selected Areas in Communications, IEEE Journal on , vol.6, no.9, pp.1617-1622,

Dec 1988

[36] Joel Sommers, Hyungsuk Kim, Paul Barford, “Harpoon: a flow-level traffic generator for router and network tests”, June 2004, ACM SIGMETRICS Performance Evaluation Review, Volume 32, Issue 1 [37] Liming Lu., Mun Choon Chan, Ee-Chien Chang, 2008 , “A general

72

model of probabilistic packet marking for IP traceback”, Conference on Computer and Communications Security, Proceedings of the 2008 ACM symposium on Information, computer and communications security, SESSION: Network security (II), On page(s): 179-188 pp.

[38] Fu-Yuan Lee, May. 2005, “Designing Protection Mechanisms against DDoS Attacks”, doctor’s thesis, Department of Computer Science and Information Engineering, National Chiao-Tung University, Taiwan.

[39] Izaddoost, A. Othman, M. Rasid, M.F.A., 18-21 Dec. 2007, “Accurate ICMP TraceBack Model under DoS/DDoS Attack”, Advanced

Computing and Communications, 2007. ADCOM 2007. International Conference on, On page(s): 441-446 pp.