Below is the content of following chapters:
In Chapter 1, we will discuss the DDoS attacks and the current methods for defense. In Chapter 2, we will introduce the internet-based intrusion detection system that becomes more and more emphasized with Snort example as the illustration. In Chapter 3, we will illustrate the IP status database that build upon the basis of attacking behavior and it’s application in records analyzing to reduce the Snort overwhelmed records problem. In Chapter 4, we will introduce the practical system and analyze the result of the experiments. In Chapter 5, we will make a conclusion for the researches of this thesis.
CHAPTER 2
BASIC CONCEPT OF DoS AND IDS
2.1 Denial of Service (DoS) & Distributed Denial of Service (DDoS)
2.1.1 The DoS attacks
DoS (Denial of Service) is just like that you build a store for customer shopping. The attackers interrupt customer shopping at your store by occupying your store or line up at the entrance. When it happens, the store cannot provide the service in time. For example, web provider cannot provide the website service; ftp server cannot provide files uploading and downloading; email service cannot able to be sent and received.
DoS attacks were often used while sending mass of fault IP address packets at the same time so that the server being attacked cannot afford ( i.e.
over the capacity ) or over the maximum linked quantity to paralyze the internet or server so that end user cannot get the service of the server.
2.1.2 The DDoS attacks
Because DoS attacks are via sending faulty mass packet from one point, we can find the attacker from a simple flow surveillance system (i.e. MRTG that often used by TANET). To avoid the disadvantage and get better effect of DoS attacks, the hackers change the original pure DoS attacks to multi-tier and develop them as multi-points ( as figure 2-1 ). Therefore, by screening the statistics of the flow surveillance system, we can only narrow down the attacked areas to a limited IP addresses range. However, when we find these IP address,
most of them are only a relay. The real hacker may possibly behind many relays.
Moreover, as the packet IP address is faulty, it’s very difficult to trace back the attackers from IP address.
Figure 2-1 DDoS Layer Control1 Attacker; Master: Control Side
Daemon: Program Side; Victim: The host which is attacked
DDoS (Distributed Denial of Service) attacks are via a main terminal to control many distributed hosts those have been attacked. While receiving the command of attacking, it will send mass packets to target computers. The target computers cannot work when the flow is overwhelming. The so-called
“Distributed” Denial of Service is because it attacks distributed computers. The attacking flow is as below:
A. Hackers intrude hosts then install software which is with specific function (ex. Enable to send mass faulty packets or scan the weak points of the internet automatically); like figure 2-2.
Figure 2-2 DDoS Attacking-Stage 12 ( Figure Source [11] )
B. The hackers can intrude more hosts through the attacked hosts; like Figure 2-3.
Figure 2-3 DDoS Attacking-Stage 23 ( Figure Source [11] )
C. To start the attacking program from far away main control host to make the attacked hosts intrude specific websites at the same time; like Figure 2-4.
Figure 2-4 DDoS Attacking-Stage 34 ( Figure source [11] )
2.2 Intrusion Detection System (IDS)
Intrusion Detection System (IDS) [1] [9], can detect immediately abnormal behaviors (misuse, abuse, attack) or act against a user from its own rule of security policy. It’s different from so-called Firewall; the main function of the Firewall is to execute Access Control. As long as it is not obey the rules, the packet cannot be passed. If it follows the rules or the data doesn’t through the system, it cannot guarantee to provide security protection. For example, internal attacking which is behind the firewall or the accounts used by the one who is not authorized.
The IDS can be divided into two major types:
Host-based IDS: It is mainly designed for one host. It can monitor the higher level data of Protocol Stack, like operating system or
application programs. As Figure 2-5.
Figure 2-5 Host-based IDS5
The Network-based IDS: The IDS of this thesis is NIDS. It usually supervises the data transmitted via internet. Its advantages are as below:
A NIDS can monitor several hosts.
IDS deployment is as figure 2-6 showed. Unlike the firewall monitors all the data in and out, it just monitors the whole internet transmissions. The firewall usually impacts the network transmitting efficiency; moreover, it becomes the bottle neck of the internet. However, IDS only monitors and doesn’t impact the transmitting efficiency.
Figure 2-6 IDS deployment diagram6
Data updated (ex. renew the new rules): It is only to be done for one time, no need to update each machine one by one.
It cannot only see the higher level data in the Protocol Stack, but also the lower packet data. As figure 2-7.
Figure 2-7 Networks-based IDS7
It can detect the intrusion activities immediately; no time wasted as waiting the packets entered the machine. The function of the NIDS is not really to execute protection. The main mission is to detect the intrusion events. Usually, it detects the intrusion activities by comparing between analyzing packet flow quantity, and the noted attacking pattern.
2.3 IDS Snort
Snort [10] is mainly designed by Martin Roesch design. The main characteristics are as below.
Lightweight intrusion detection system
Instant data analyzing and logging the packets on internet
Able to execute analysis of communication protocol, content searching and comparing
Use libpcap to catch internet packet which the program written by C language
Adopting Rule-based to detect intrusion; able to describe the packet to be detected with a simple language. Able to download the latest Rule database from Snort website at any time
Adaptable to small and low flow network
It is free software of GNU (GENERAL PUBLIC LICENSE ), and provides the source code.
Its structure is such as figure 2-6. Making use of the Libpcap to start BPF (Berkeley Packet Filter) to catch the packet on internet and compare the string (pattern matching) with the Rule base that set up in advance.
When intrusion activities are detected, it will produce log, alert or pass.
Figure 2-8 Snort Structure8
2.3.1 Snort Rules
The rule matching of Snort is its key technique. We will introduce its applications with the rule first.
※Construction of Rule:
Rule header + Rule options
For example: alert tcp !10.1.1.0/24 any -> 10.1.1.0/24 any (flags: SF; msg: “SYN-FIN Scan”;)
|---header---||---options---|
※Rule header：
Action：
Alert: Produce a warning such as writing them to syslog or flick a message Windows on the monitor of the appointed machine
Log: To record the packets.
Pass: Allow the packet pass without doing anything
Protocol：
Including three kinds of protocol: TCP, UDP, and ICMP
IP address:
Use “any” for random IP
Appointed IP address: ex.140.112.8.164.
CIDR format: ex. 140.112.8/24.
Able to use"!" to represent "not" what kind of the IP address, as!
140.112.8/24, indicates not a sub-network of 140.112.8. x
Port Numbers：
. Use “any” for random Port . Appointed port: ex. 80,23 …etc.
. A scope: ex.
¾ 1:2048, indicates port 1 to port 2048.
¾ :1024, indicates less or equal to port 1024.
¾ 512:, indicates bigger or equal port 512
.Can use"!" to indicate "Not" what kind of the port.
Direction：
source_ip_port -> destination_ip_port：Source IP address is in the left;
destination IP address is in the right.
<>：Means it doesn’t tell them from source IP address or destination IP address.
Rule
※ options：
Msg: The message that let log or alert together to be recorded.
Language msg:”<message text>”;
Logto: Point out the specific file which is to be save the records.
Language logto: “<filename>;
IP TTL: Set a condition that matches a TTL (time to live).
Language ttl:”<number>”;
IP ID: A setting that matches a IP Fragment ID Language id:”<number>”;
Dsize: Check the size of the packet Payload.
Language dsize: [>|<]<mumber>;
Content: Searching a specific stream in the packets. If it is binary data, it will be with “|”
For example, content: "|90C8 C0FF FFFF|/bin/sh";
Language content:”<content string>”;
Offset: Comparing the streams from a specific start point.
Language offset:”<content string>”;
Depth: Only comparing the stream for some specific length of the data from packets.
Language depth:<number>;
Nocase: Ignore the whether it is capital letter or lowercase when comparing Language nocase;
Seq: The setting of the Sequence Number which matches TCP heading Language seq:<number>;
Figure 2-9 TCP Headings9 ( Figure Source [8] )
Flags: The setting of the Flag which matches TCP heading. There are eight of Flag as below.
F - FIN S - SYN R - RST P - PSH
A - ACK U - URG
2 - Reserved bit 2 1 - Reserved bit 1
Language flags:<flag values>;
Ack: The setting of the acknowledgement number which matches TCP heading.
Language ack:<number>;
Itype: The setting that matches ICMP TYPE Language itype:<number>;
Icode: The setting that matches ICMP VALUE.
Language icode:<number>;
[Please refer to appendix 1 for ICMP TYPE and VALUE]
Session: Retrieve all the data of User’s TCP Session; especially in telnet, rlogin, ftp and web session. There are two kinds as below:
Printable：Only record the data that can be read directly; like text.
All：Transfer the data that cannot be read directly to HEX.
Language session:<printable|all>;
Icmp_id: The setting of ICMP ID Number that matches ICMP ECHO packets.
Language icmp_id:<number>;
Icmp_seq: The setting of ICMP Sequence Number that matches ICMP ECHO packets.
Language icmp_seq:<number>;
Ipoption: Provided the function of searching some specific option rr - Record route
eol - End of list nop - No op
ts - Time Stamp
Rpc: Searching the application, procedure, and program version when using RPC.
Language rpc:<application number, [procedure number|*], [program version number|*]>
Resp: Providing flexible response. There are seven kinds of resp_modifier as below. They can be used at the same time.
rst_snd - send TCP-RST packets to the sending socket rst_rcv - send TCP-RST packets to the receiving socket rst_all - send TCP_RST packets in both directions
icmp_net - send a ICMP_NET_UNREACH to the sender icmp_host - send a ICMP_HOST_UNREACH to the sender icmp_port - send a ICMP_PORT_UNREACH to the sender icmp_all - send all above ICMP packets to the sender
Language resp:<resp_modifier[,resp_modifier…]>;
2.3.2 Rule Tree
The Snort rule has more than 5000s about currently. It can detect various kinds of intrusion. The rule is very huge. To increase the speed of detecting, it needs to take some actions to the Rule. Snort stores Rules with the Tree pattern.
Firs, taking Rule header as a horizontal string row, the Rule option is each Rule header under of lengthways string row, and make all Rule headers be the only one. If the repetition of the Rule header appears, then adding the string of Rule
option row directly underneath, as figure 2-10. The advantage is that the same rule headers are only being checked once and raise the speed of comparing.
Although it doesn’t eliminate comparing the rule Option, it doesn’t matter as the repetition is not frequent.
Figure 2-10 Snort Rule Tree Structure10
2.3.3 Preprocessors
The Preprocessor makes the users be able to join their own module easily, and it execute before the packets enter the detection engine of Snort.
Language preprocessor <name>:<options>
2.3.4 Variables
Usually, we will edit snort.lib before using Snort. There are many variables need to be defined in advance; like your network, and IP address of
DNS.
Language <name>:<value>
2.3.5 Weakness
Generally speaking, the common problem of the pattern-based- IDS is the record is excessive; Snort also faces the same situation.
2.3.6 Snortlog analysis tool
※ Snort_stat.pl:
It is a snort log statistics program written with perl. It can produce the statistics data as follows:
Number of attack from same host to same destination using same method
# of attack from to With
26 140.110.21.99 140.112.10.100 PING-ICMP Time Exceeded
1 140.110.20.81 140.112.18.163 FTP-bad-login
3 140.112.10.24 140.112.18.99 IDS159-PING Microsoft Windows
… ….. …. ……….
Percentage and number of attacks from a host to a destination
% # of attacks from To
24.46 32 140.109.20.100 140.112.18.164
12.23 16 140.109.20.80 140.112.8.153
5.35 7 140.112.12.34 140.112.8.98
… … … …
Percentage and number of attacks from one host to any with same method
% # of attacks from Type
17.95 14 140.109.30.10 ICMP Destination Unreachable
15.38 15 140.109.30.80 ICMP Destination Unreachable
12.89 11 140.112.12.34 PING-ICMP Time Exceeded
… … … …
CHAPTER 3
IP STATUS DATABASE
3.1 Intrusion Behavior Analysis
An intrusion event occurred usually isn’t by chance. The hacker wants to intrude a system is just like a commando permeate into a castle. It must have a detailed and complete plan. The complication of the plan depends on the safety level of the system. First, it collect information and overview its target, because blindly trial reveals the sign of intrusion very easily and the one intruded alerts to guard and increase the difficulty of intrusion. Therefore, once we have a specific target, we can decrease many unnecessary trial and error. For example, there are different attacking ways to different systems, and servers. Below, we will divide intrusions into four stages.
※ Stage One: Information Collecting
The information collection let the attackers get more useful information of the attacked one. For example, the safety measures (whether if it sets up firewall or not, IDS...etc.), the allocation of intranet IP, stir to answer the phone number (some of the string network can be hided from the checking of firewall), and branch (possibly the authorized IP which passed from the IP-Based access control …etc. Making use of some current tooling and technology, the attackers can get some information automatically. For example:
Finger, whois: The command to obtain the user information.
Traceroute: The path tracking program which acknowledge the routers passed by, or the possible existent firewall. However, currently there are many units have already closed traceroute.
Teleport Pro: It can duplicate the contents of the target website automatically, the find out useful information from them. Some websites establish a program that can create a great deal of faulty data to prevent being duplicated.
※ Stage two: Network Scanning
An entrance of a normal house is its front door, but for the sneaks, it can be windows, back door, and chimney…etc. Network Scanning is to find out the host of the attack target and the services it provides. It makes use of the channels and possible safety leak. The so-called portscan( scanning conjunction) is exactly the hacker often use for diction beforehand. By using portscan, the hackers can find out a network operation in active and each port that can provide services of all machines. There are two types of portscan:
Scanning the entire network: It usually scan the IP address in the order from small (big) to the big (small) IP one after another. It mainly is to know to how many active machines are.
Scanning all the connectors of each machine: Scanning from small (big) connectors to big (small) ones one by one. It mainly is to know the services that provided by the hosts.
※ Stage Three: The weakness attack
After understanding the services and operation system of the target host, the hackers can launch the intrusion attack to the related weakness. For example, a server opens port 80 or 8080, so that it maybe a web server, then the intruder can find out the web program and version used, and taking use of related web attacking program to start intrusion.
※ Stage four: Obtain the Right of Usage
The final purpose that attacker makes series of intrusion activities is to obtain the right of usage (general user), control root or install particular program, such as backdoor program, DDoS master or daemon …etc. With these rights means that the hacker can control the host completely, it can set up false account and obtain confidential data in the host…etc.
3.2 Dividing the Rules into Urgency Levels
In the Snort intrusion detection rules, in fact it is separated by different safety urgency. For example, ftp-bad-login is to record ftp on-line register errors.
Such event is not necessarily a very urgent intrusion event. Moreover, it often includes typo from normal users. However, some events like the overflows, usually may cause the system crash, the program breaks off abnormally or even jumped to the system control mode...etc... Above emergency events possibly need immediately taking actions on.
For Snort rule, we divided it into three levels:
High: The events those needs to be handled immediately; like it may obtain the root legal power or cause the system crash.
Medium: Although it doesn't immediately cause damages, it may follow with DDoS attack.
Low: "Possible" attacking events, like MISC-PCAnywhere Attempted Administrator Login.
As different administrators may have different definitions for each emergency degree of various rules, so the rule emergency level can be defined by users. We have already set up the rule grade as appendix two.
3.3 Build IP Status Database
According to the harmfulness degree of the intrusion records, we can give IP a different status according to the behavior of different stage. We divide it into three kinds of grades:
Hostile: IP possibly is attacking.
Threaten: IP may be potential offensive
Suspicious: IP with suspicious behavior.
For example, the activity of finger, scan, and the ping …etc. is generally the behavior that belongs to intrusion preliminary, the IP with records like this is included in "Suspicious" grade. When this IP is the records those belong to Medium grade, then it is upgrade to "Threatening" grade. At the end, if they are the events those belong to High, it will be raised as the highest grade
"Hostile". We illustrate this with status figure 3-1.
Figure 3-1 BAD IP finite state11
H: High even log
When "Low" of a new IP records number is bigger than a certain minimum value, then it will get into "Buffer" status of the database. At the moment, it still doesn't belong to any grade. However, if the number is bigger than the minimum value of "Suspicious", then it directly gets into "Suspicious" status.
When IP is at "Suspicious" status, if there is "Low" events happened, then it will stay in the original status. Once "Medium" events occurred, it will be transferred to "Threatening" status. If there is "High" event, then it directly turns into "Hostile" status.
When IP is at "Threatening" status, if there is "Low" or “Medium”
happened then it will stay the original status. Once "High" event occurs, it turns into "Hostile" status.
When IP is at "Hostile" status, if there is any record occurs, then it stays original status.
Each IP may still stay in the condition of original status for different events, but its scores will have different scores increment with the amount of events occurred.
Each IP will gradually decrease its scores everyday. Different status may have different decreasing value till to zero, because the attacking to a host usually is in a short time like one month or even one week, two or three days. The attack will not take actions in a long time interval.
IP and status database can choose not using day as a unit, for example, every half day, or every hour to be processed once. As long as you adjust the parameter of the transformation ratio between the times and the scores, and the value of gradually decreased per day.
The high or low scores of each IP represent different possible damage degree. The parameter s that user can define by itself are as below:
The high or low scores of each IP represent different possible damage degree. The parameter s that user can define by itself are as below: