Adaptive Sequential Hypothesis Testing

Our proposed adaptive sequential hypothesis testing provides estimates of θ0 and θ₁ adaptively based on observations of the outcomes of FCC requests.

We will consider only the estimation procedure of θ₀. The estimation procedure for θ₁ is similar.

The basic idea of our proposed estimation procedure is as follows. An estimate of θ₀, denoted by θ , is generated when the total number of remote ^ˆ₀ hosts that are detected as benign is greater than or equal to K, where K is a design parameter. Let S_i and F_i represent, respectively, the numbers of successful and failed FCC requests sent by r_i when it is detected as benign. Furthermore, remote host makes an FCC request to a local destination, its likelihood ratio is

updated according to the outcome, i.e., success or fail, of the FCC. If the FCC request is classified as success, S of _i ^{Hash r( )} is increased by one, where

Hash r( ) represents the hash result of IP address r. On the contrary, if the FCC request is classified as fail, F_{i of ( )}Hash r is increased by one.

Table 3: Data structure of the adaptive sequential hypothesis testing algorithm.

Hash r( ) Λ X

( )

611 5.545177 0 2

849 6.415920 3 4

965 -4.674434 3 0

1540 -5.361835 7 2

… … … …

The remote host r is detected as benign if its likelihood ratio is lower than threshold η₀. On the other hand, if its likelihood ratio is higher than threshold

η1, the remote host r is declared as malicious. Once remote host r is decided as benign or malicious, the corresponding S_{i and} F_i values are added to the data structure shown in Table 4.

Table 4: Data structure for updating θ and ^ˆ₀ θ . ^ˆ₁ benign or malicious. However, it achieves better accuracy and thus is

worthwhile to sacrifice the decision time. In our design, θˆ₀ and θˆ₁ are updated for the first time when a total of K remote hosts are decided as benign or malicious, respectively. Based on ordered statistics [10], for a group of benign remote hosts which issue FCC requests randomly to local hosts, the first few hosts that are detected as benign tend to have zero or very few failed FCC requests. Similarly, the first few malicious remote hosts that are detected as malicious tend to have zero or very few successful FCC requests. Consequently, the estimates may largely deviate from the real values if we set K = . In general, a large value of 1

K provides better accuracy but longer detection time. We select K =10 in our experiments presented in the next chapter.

Chapter 5.

Experimental Results

In this chapter, we present simulation results for the TRW algorithm (with known θ and unknown θ ) and our proposed adaptive sequential hypothesis testing algorithm. The desired false positive and false negative probabilities are both set to 0.01. In other words, we choose α =0.01 and β =0.99 in our experiments. Simulations are performed for 900 benign hosts and 100 malicious hosts. The probabilities of success for an FCC request generated by a benign host or a malicious host are equal to θ₀ and θ₁, respectively. We performed simulations for different values of θ₀ and θ₁.

Figure 1 and Figure 2 compare, respectively, the false positive and false negative probabilities of the TRW algorithm with or without knowing θ₀ and θ₁ and our proposed adaptive algorithm, for various values of θ₀ and θ₁. We assume that θ₀ =0.8 and θ₁=0.2 are used for the TRW algorithm without knowing θ₀ and θ₁. As one can see, the false positive and false negative probabilities are very low for the TRW algorithm with perfect knowledge of θ₀

and θ₁. However, without knowing the real values of θ₀ and θ₁, its false positive and false negative probabilities of TRW could be much greater than the desired values when θ₀ is small and θ₁ is large (say, θ₀ =0.6 and θ₁=0.4).

The reason is that the step size of moving upward using θˆ₀ =0.8 and θˆ₁=0.2 is significantly larger than the step size of moving upward using θ₀ =0.6 and

1 0.4

θ = . Using our proposed scheme (i.e., Adaptive SHT), the false positive and false negative probabilities are almost lower than 5% for all cases (except for

0 0.55

θ = and θ₁=0.45). The results are close to the desired values because the estimates of θ₀ and θ₁ in our proposed scheme are quite accurate (as Table 5 shows). Note that in our proposed scheme, the false positive probabilities are larger than false negative probabilities when θ₀ is large and θ₁ is small. This is because the number of benign hosts is much larger than the number of malicious hosts. As a result, θˆ₀ is updated much earlier than θˆ₁. As mentioned before, the earlier detected benign hosts tend to have many more successful FCC requests than failed ones. This implies θˆ₀ tends to be larger than the real value which makes it easier to detect a remote host as benign.

Figure 1: Comparison of false positive probabilities.

Figure 2: Comparison of false negative probabilities.

Table 5: Estimates of θ₀ and θ₁ for the proposed adaptive algorithm.

Table 6 and Table 7 show, respectively, the average number of FCC requests sent by a remote host to be detected as benign or malicious. The TRW algorithm with unknown θ₀ and θ₁ is fast in making a decision because the large step sizes. Unfortunately, as illustrated in Figures 1 and 2, its false positive and false negative probabilities are not satisfactory. The average number of FCC requests for our proposed adaptive algorithm are comparable to those for the TRW

algorithm with known θ₀ and. Let ˆθ _{and ˆ '}θ be two successive estimates of θ . One can stop updating ˆθ if ˆθ θ'− ˆ < for a given ε ε to speed up the detection time. In other words, the time spent to obtain a stable estimate of θ can be regarded as the period of training. Of course, to adapt to a changing

environment, the training procedure should be reactivated once in a while.

Table 6: The average number of FCC requests to detect a remote host as benign.

θ0 95% 90% 85% 80% 75% 70% 65% 60% 55%

θ1 5% 10% 15% 20% 25% 30% 35% 40% 45%

TRW (known

θ0 and θ₁)

2.21 3.74 4.24 6.61 9.92 14.81 26.32 59.14 225.61

TRW (unknown

θ0 and θ₁)

4.45 5.00 5.70 6.63 7.80 9.36 11.26 13.40 15.23

Adaptive_SHT 15.82 17.66 20.01 23.03 27.35 33.97 46.15 77.19 304.51

Table 7: The average number of FCC requests to detect a remote host as malicious.

θ0 95% 90% 85% 80% 75% 70% 65% 60% 55%

θ1 5% 10% 15% 20% 25% 30% 35% 40% 45%

TRW (known

θ0 and θ₁)

2.21 3.74 4.23 6.63 9.91 14.78 26.42 59.15 224.71

TRW (unknown

θ0 and θ₁)

4.45 5.01 5.71 6.63 7.80 9.36 11.24 13.38 15.24

Adaptive_SHT 15.33 17.13 19.33 22.11 25.98 31.75 41.43 61.81 139.21

Chapter 6.

Conclusion

We have presented in this paper an adaptive sequential hypothesis testing algorithm for accurate detection of scanning worms. Numerical results show that our proposed adaptive algorithm provides accurate estimates of θ₀ and θ₁ and thus achieves false positive and false negative probabilities close to the desired values. The proposed adaptive estimation procedure for θ₀ and θ₁ is an important enhancement of the sequential hypothesis testing algorithm because it makes the algorithm much more robust to variation of θ₀ and θ₁. The

proposed adaptive detection algorithm is only suitable for scanning worms.

How to effectively detect other types of worms remains to be further studied.

Bibliography

[1] J. Jung, V. Paxson, A. W. Berger, and H. Balakrishnan, “Fast Portscan Detection Using Sequential Hypothesis Testing,” In Proceedings of the IEEE Symposium on Security and Privacy, May 9-12 2004.

[2] S. E. Schechter, J. Jung, and A. W. Berger, “Fast Detection of Scanning Worms Infections,” In Proceedings of the 7th International Symposium on Recent Advances in Intrusion Detection (RAID 2004), September 15-17 2004.

[3] N. Weaver, S. E. Schechter, V. Paxson, “Very Fast Containment of Scanning Worms,” In Proceedings of the 13th USENIX Security Symposium, August 9-13 2004.

[4] M. Roesch, “Snort: Lightweight Intrusion Detection for Networks,” In Proceedings of the 13^th Conference on Systems Administration (LISA-99), pages 229–238, Berkeley, CA, Nov.

7–12 1999. USENIX Association.

[5] L. T. Heberlein, G. V. Dias, K. N. Levitt, B. Mukherjee, J. Wood, and D. Wolber, “A Network Security Monitor,” In Proceedings of IEEE Symposium on Research in Security and Privacy, pages 296–304, 1990.

[6] D. Moore, C. Shannon, and J. Brown, “Code-Red: a case study on the spread and victims of an Internet worm,” in Proc. ACM/USENIX Internet Measurement Workshop, France, Nov. 2002.

http://www.caida.org/dynamic/analysis/security/nimda.

[8] D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, and N. Weaver, “Inside the Slammer Worm,” IEEE Magazine of Security and Privacy, 1(4): 33-39, July 2003.

[9] A. Wald, Sequential Analysis, J. Wiley & Sons, New York, 1947.

[10] R. Hogg and A. Craig, Introduction to Mathematical Statistics, The Macmillan Company, 1970.

[11] C. C. Zou, D. Towsley, W. Gong, and S. Cai. “Routing Worms: A Fast, Selective Attack

Worm based on IP Address Information.” In Proceedings of the 19^th Workshop on Principles of Advanced and Distributed Simulation (PADS’05), June 2005.

[12] N. Weaver, V. Paxson, S. Staniford ,and R. Cunningham. “A Taxonomy of computer worms.” In Proceedings of the 2003 ACM Workshop on Rapid Malcode, pages 11–18. ACM Press, October 27, 2003.

[13] Type I and Type II errors. From Wikipedia, the free encyclopedia,

http://en.wikipedia.org/wiki/Type_I_and_type_II_errors