Chapter 3 Proposed Behavior-based Botnet Detection Algorithm in Cloud
3.1 Problem statement
3.1.2 The sub-problems
Traffic reduction
Since there are some bot-unrelated data in the collected input traces, we may filter out these data to speed up botnet detection. Bot behaviors always involve specific network operations. Therefore, we may retain packets related to these operations.
Traffic partitioning
Since we want to reduce the total execution time of the proposed algorithm, we use a cloud computing platform to speed up botnet detection. Traffic partitioning divides an input trace into pieces. Then, available server
9
instances in the cloud can operate on these pieces concurrently using the proposed botnet detection algorithm.
Feature extraction
In our observation, bots always contain specific behaviors that are different from normal user’s behaviors. Therefore, we want to detect bots by features extracted from bot behaviors.
DNS phase
The DNS phase focuses on the bot DNS query and response packets since bot activities often start with DNS traffic. We use fuzzy pattern recognition with max membership principle to identify some bot behaviors from DNS traffic in the DNS phase.
TCP phase
A bot master may send and update bot binaries programs to bots. The TCP phase focuses on the TCP request and response packets. We use fuzzy pattern recognition with max membership principle to identify bot behaviors from TCP related traffic in the TCP phase.
3.2 Design of a behavior-based botnet detection algorithm
The proposed behavior-based botnet detection in cloud computing environments (BBDC) algorithm is shown in Figure 2. There are five stages in the algorithm:
traffic reduction, feature extraction, traffic partitioning, DNS phase and TCP phase.
First, input traffic is passed to the traffic reduction stage by removing bot-unrelated traffic. Second, the feature extraction stage will extract bot features from the reduced input traffic. The packets’ related information will be recorded into the database by bot features that we observed. Third, the reduced input traffic will be
10
divided into several pieces according to available server instances in the cloud.
Each server instance in the cloud then runs the proposed two phase bot detection algorithm to detect bots from the received piece. Fourth, in the DNS phase, the detection algorithm checks if any DNS related packets are bot traffic or not based on bot features extracted in the second phase. If there is no bot found in the DNS phase, the piece will be fed into the TCP phase to determine whether any TCP related packets are bot traffic or not. Both the DNS and TCP phases use the proposed fuzzy pattern recognition with max membership principle to match bot behavior states. If the membership value falls into any bot’s state, the input trace is identified as a bot.
Once an input trace is identified as a bot, we will record the bot related information in the database, such as DNs and IP addresses. Then the other hosts can be informed of these bots via the alert system.
Figure 2. Proposed behavior-based botnet detection in a cloud computing environment.
11
3.3 Traffic reduction
It is true that a good traffic reduction filter can reduce the data needed to be processed and also increase classification accuracy [1]. That is, by removing these unrelated data, both bot detection time and bot detection accuracy can be improved.
Figure 3 shows the procedure in traffic reduction stage. An input trace usually involves several network protocols. In our observations, a bot master may register many domain names and let bots to inquire IP addresses of these domain names.
Therefore, bots often send DNS queries to domain name servers frequently to retrieve IP addresses. Then, bots will establish TCP connections to these IP addresses. That is, bot behaviors always involve DNS query/response and TCP request/response packets.
Based on these observations, we can filter out packets that are not related to DNS or TCP protocols. The retained packets will then be stored in a database (DB) for feeding into the feature extraction stage.
Figure 3. The procedure in the traffic reduction stage.
12
3.4 Feature extraction
Since bots always contain specific behaviors that are different from normal users’
behaviors, we can detect bots by extracted features from bot behaviors.
3.4.1 Feature extraction from DNS packets
We observed that bots send DNS queries periodically in a time period since bot behaviors always involve DNS queries. For example, Figure 4 shows the distribution of botnet DNS query packets. We may use some features that we observed from bots specific behaviors to detect bots, such as total number of DNS packets between query and response packets, total times that a node used the same IP addresses, etc. Those packets that are related to bot DNS features will be stored in the database.
Figure 4. The distribution of botnet DNS query packets. [x axis: seconds; y axis:
number of DNS query packets].
3.4.2 Feature extraction from TCP connection packets
Figure 5 shows the distribution of botnet TCP request packets. It illustrates that the TCP request packets distribution of botnet traffic is periodical. We use some features that we observed from bots specific behaviors to detect bots, such as packets count per second, bytes count per packet, etc. Those packets that are related to bot TCP features will be stored in the database.
13
Figure 5. The distribution of botnet TCP request packets. [x axis: seconds; y axis:
number of TCP request packets].
3.5 Fuzzy pattern recognition for DNS and TCP phases
As mentioned before, bot behaviors can be obfuscated to evade a string signature-based detection system. Therefore, we propose a behavior-based botnet detection algorithm in cloud computing environments (BBDC) that uses fuzzy pattern recognition to identify bots, including obfuscated bots. In the proposed BBDC algorithm, the fuzzy pattern recognition has two phases: DNS phase and TCP phase.
In the DNS phase, we determine if it is a bot based on DNS features that were collected from the feature extraction stage, as shown in Figure 6. If it is not a bot, then the input trace is passed to the TCP phase. Oterwise, the input trace is identified as a bot. In the TCP phase, we detect a bot based on TCP features that were collected from the feature extraction stage, as shown in Figure 7. Both phases use fuzzy pattern recognition to classify bot and non-bot behaviors based on the max membership principle. Each membership function corresponds to a state and has a membership value, which will be described in the DNS and TCP phases later. The BBDC algorithm will find a max membership value, and the bot trace or normal trace is in
14
the state associated with this max membership value.
3.5.1 DNS phase
In the DNS phase, we define a packet features vector x = (α, β, γ, λ). α is a set of time intervals {αi | 1 ≦ i ≦ n} between DNS query and response packets, where αi is the length of time interval i between DNS query and
Figure 7. The TCP phase of the proposed botnet detection algorithm.
Figure 6. The DNS phase of the proposed botnet detection algorithm.
15
response packets, and n is the total number of DNS queries; β is a set of total number of DNS query packets {βi | 1 ≦ i ≦ n} in contrast to α, where βi is and their associated membership functions, as described in the following.
Bot trace states
(a) Normalized abnormal variance of total number of DNS packets between query and response packets
An active malicious DNS traffic usually has a large packet count in a time period. More DNS packets in a time period lead to a higher membership value. We define a membership function for calculating the normalized abnormal variance of total number of DNS packets between query and
Figure 8. Fuzzy max membership principle in DNS phase.
16
query packets in time interval at the jth second, and is the threshold of being abnormal variance of DNS packets.
(b) Normalized abnormal total times that a node used the same IP addresses
Bots may contact specific IP addresses many times in their execution period.
Therefore, we calculate the contact times per IP address to identify the ith IP address used by this node, and is the threshold of the abnormal contact times per IP address.
(c) Normalized abnormal total number of DNS query and response packets per second
Bots may send DNS query packets many times in their execution time.
Therefore, we calculate the times of DNS queries per second to identify abnormal DNS queries. We define a membership function for calculating normalized abnormal total number of DNS query and response packets per second.
where N is the duration of an input trace in seconds, is the total number of DNS query and response packets in the ith second, and is the threshold of the total number of DNS query and response packets per
17
second.
Normal trace state
We define a membership function for calculating the probability of being a normal trace.
(4)
3.5.2 TCP phase
In the TCP phase, we define a packet features vector x = (α, β, γ, λ). α is a set of time intervals {αi | 1 ≦ i ≦ n} between TCP request and response packets, where αi is the length of time interval i between TCP request and response packets, and n is the total number of TCP request packets; β is a set of total number of TCP packets {βi | 1 ≦ i ≦ n} in contrast to α, where βi is the total number of TCP packets in contrast to αi, and n is the total number of TCP requests; γ is a set of total number of bytes {γi | 1 ≦ i ≦ n} in contrast to α, where γi is the total number of bytes in contrast to αi, and n is the total number of TCP requests. λ is total number of TCP request and response packets per second.
Figure 9 shows the fuzzy pattern recognition with max membership principle in the TCP phase. In this phase, we define six states and their associated membership functions, as follows:
Figure 9. Fuzzy max membership principle in TCP phase.
18
Bot trace states
(a) Normalized abnormal packets count per second
If a TCP connection sent too many requests in a second, the TCP packets count per second would reflect the abnormal behavior. We define a membership function for calculating the normalized abnormal packets count per second. duration of an input trace in seconds, and is the threshold for abnormal packets count per second.
(b) Normalized abnormal bytes count per packet
If a bot master wants to send commands to other bots, the bytes count per TCP packet will reflect the abnormal behavior. We define a membership function for calculating the normalized abnormal bytes count per packet.
(c) Normalized abnormal variance of total number of TCP packets between request and response packets
An active malicious TCP traffic usually has a large packet count in a time period. More TCP packets in a time period lead to a higher membership
19
value. We define a membership function for calculating the normalized abnormal variance of total number of TCP packets.
threshold of the variance of total number of TCP packets.
(d) Normalized abnormal variance of total number of bytes
An active malicious TCP traffic usually has a large byte count of TCP packets in a time period. More bytes in a time period lead to a higher membership value. We define a membership function for calculating the normalized abnormal variance of total number of bytes.
variance of abnormal total number of bytes.
(e) Normalized abnormal total number of TCP request and response packets per second
Bots may send TCP request and response packets many times in their execution periods. Therefore, we calculate the total number of TCP request and response packets per second to identify abnormal behaviors. We define a membership function for calculating the normalized abnormal total number of TCP request and response packets per second.
20
where N is the duration of an input trace in seconds, is the total number of TCP request and response packets in the ith second, and is the threshold of the total number of TCP packets per second.
Normal trace state
We define a membership function for calculating the probability of being a normal trace.
(10)
3.5.3 Observation
We observed 50 bots and 50 normal traces. Each trace lasts for two hours.
In these experiments, we found that there are different behaviors between bots and normal traces. Figure 10 shows some observations of six bot characteristics. In Figure 10(a) and 10(b) show the statistics about the variance of packet inter arrival time and the variance of bytes per packet, respectively. In Figure 10(c) and 10(d) show the statistics about the number of packets between request and response packets and the contact times per IP address. In Figure 10(e) and 10(f) show the statistics about the average number of packets per second and the average number of bytes per packet.
Each characteristic reflects different behaviors of some bots. To reduce the false-positive rate, we choose more bot’s behavior characteristics that can characterize botnets. To reflect the behaviors of IRC bots, the contact times that a node used the same IP addresses and the total number of DNS query and response packets per second can be used to detect IRC bots. To
21
reflect the behaviors of HTTP bots, the packet count per second and the byte count per packet can be used to detect HTTP bots.
22
(a) (b)
(c) (d)
(e) (f)
Figure 10. Some observations of six bot characteristics.
23
Chapter 4
Performance Evaluation
4.1 Traces collection
To collect real botnet traces, we installed an unpatched Windows XP SP3 in a virtual machine using Ubuntu and Virtualbox, and executed 250 real bot samples inside a Windows environment (HoneyTrap). We used a share folder to manage honeytraps (1 to N), as shown in Figure 10. Both input and output network traffic of the virtual machine were recorded by a Recorder and stored in an Apache database via DBInserter. Among 250 bots, only 240 bots had network traces. The rest of bots were not executable. The botnet traces were recorded for 2 hours via Recorder. Both the packet header and complete packet payload were stored in the database for further botnet analysis.
Figure 10. Experimental environment for botnet traces collection.
24
4.2 Test results of botnet traces
We used real botnet traces to evaluate our BBDC algorithm. Statistics of the botnet traces and the false negative rate are shown in Table 1. The evaluation result shows that BBDC a low false negative rate (FNR), 4.17%.
Table 1. Botnet traces statistics and false negative rate.
Number traffic reduction rate and the false positive rate (FPR). T1 traces were collected from the National Chiao Tung University’s campus beta site. We collected 695 traces in T1 for 2 hours. T2 traces were obtained from our MAL laboratory. We collected 5 traces in T2 from laboratory’s members. These traces contain various types of benign applications using IRC, HTTP, and P2P, etc. In Table 2, we found that the BBDC achieves high reduction rates and low FPRs.
Table 2. Normal traces statistics and false positive rates.
Test site
25
(SIs) to evaluate the total execution time in our proposed system. In the experiments, we evaluate both host-based and cloud-based execution time. Experimental results show that the total execution time can be reduced in proportion to number of SIs used.
The cloud-based system is 4.73 times faster than the host-based system in terms of total execution time with 5 SIs used.
Figure 11. Total execution time (sec) for 950 traces using various number of server instances (SIs) in Windows Azure cloud.
Figure 12 shows the false positive bots distribution among bot behaviors in the DNS phase. Figure 13 shows the false positive bots distribution among bot behaviors in the TCP phase. Since our proposed botnet detection system may have false positive bots, we want to find out the causes of such bot features. Both figures show the bot features that cause normal traces being identified as bots. By these statistics, we can adjust membership functions to reduce the proposed system’s FPR in the future.
26
Figure 12. False positive bots distribution with respect to bot features in DNS phase.
Figure 13. False positive bots distribution with respect to bot features in TCP phase.
.
27
Figure 14 shows the false negative bots distribution for each bot type, IRC or HTTP bots. Since bots may evade and obfuscate the botnet detection system, our proposed system may have false negative bots. By observing the bots’ traces which cause false negative (FN), we may modify or add bot features to reduce FN bots.
Figure 14. False negative bots distribution for different bot types.
We also evaluate the execution time in each stage in our botnet detection system.
The traffic reduction phase spent 10% of time. The DNS phase spent about 37% of time. The TCP phase spent about 53% of time. Since the DNS and TCP phases spent most total execution time, we port the proposed system into a cloud environment in order to reduce the botnet detection system’s total execution time.
In Table 3, we compare the proposed BBDC and the other three existing botnet detection methods. Unlike these three existing methods, BBDC used 250 real bot samples to emulate real botnet traffic. Except Yu [5], the proposed BBDC performs better than Park [4] and Lin [1] in terms of false positive rate and false negative rate.
Note that the botnet traces of Yu [5] only contain four fabricated bots, not real bots.
28
Table 3. Comparison of different behavior-based botnet detection methods.
Approach
IRC+HTTP IRC+HTTP IRC IRC+HTTP
Total execution
29
Chapter 5 Conclusion
5.1 Concluding remarks
In this thesis, we have presented a behavior-based botnet detection algorithm using a cloud computing environment to speed up botnet detection. Our algorithm can reduce false positive and false negative rates compared with other behavior-based detection systems. We reduce the total execution time by traffic reduction and cloud computing. Since the proposed algorithm identifies different protocols used in an input trace and detect bots based on bot behaviors in each protocol, it can have a high detection rate (true positive rate). We use a fuzzy pattern recognition with max-membership principle to perform bot behaviors matching in the DNS and TCP phases. We have used real bots to generate real botnet traces for evaluating the proposed BBDC algorithm. As a result, the proposed botnet detection system can be feasible in a real world. Experimental results show that our proposed system can achieve a high true positive rate of 95.83% and a low false positive rate of 3.429% in the Windows Azure cloud computing platform. Furthermore, the proposed cloud-based system with five server instances is 4.73 times faster than a host-based system.
5.2 Future work
In the future, we will extend our work to detect P2P botnets by including P2P bots features in the proposed botnet detection system. Moreover, we want to reduce the FPR and FNR of the proposed botnet detection system by extracting more bot’s behavior features that can characterize botnets.
30
References
[1] K. Wang, C. Huang, S. Lin, Y. Lin, “A fuzzy pattern-based filtering algorithm for botnet detection,” Computer Networks, 2011 (accepted for publication).
[2] “Snort,” [Online]. Available: http://www.snort.org/.
[3] B. AsSadhan, José M. F. Moura and D. Lapsley, “Periodic behavior in botnet command and control channels traffic,” in Proc. IEEE Global Telecommunication Conf., 2009, pp. 2157-2162. CUSUM,” in Proc.IEEE Wireless Communications and Trusted Computing Conf., 2009.
[7] W. Lu, M. Tavallaee, G. Rammidi, A.A. Ghorbani, “BotCop: An online botnet traffic classifier,” in Proc.IEEE Communication Networks and Services Research Conf., 2009.
[8] F. Alserhani, M. Akhlaq, I.U. Awan, A.J. Cullen, “Detection of coordinated attacks using alert correlation model,” in Proc.IEEE Progress in Informatics and Computing Conf., 2010.
[9] M. Szymczyk, “Detecting botnets in computer networks using multi-agent technology,” in Proc.IEEE Dependability of Computer Systems Conf., 2009.
[10] L. Braun, G. Munz, G. Carle,” Packet sampling for worm and botnet detection in
31
TCP connections”, in Proc. Network Operations and Management Symposium (NOMS), 2010.
[11] “Windows azure,” [Online]. Available:
http://msdn.microsoft.com/zh-tw/windowsazure/cc947856.
http://msdn.microsoft.com/zh-tw/windowsazure/cc947856.