There are three issues that make it very difficult to compare events between the live traffic and replayed traffic. We need to solve the issues on network behaviors, replayed network traffic trace, and traffic identification. They are discussed as below.
The issue on network behaviors: network
behavior of and affect the comparison of events between them. Network behaviors are packet loss, duplicate packets and packets out of order. By comparing the events from a live connection and reproduced in a replayed connection. If packets within these connections are affected by the behavior of the network then it is likely that the comparison of events is incorrect. Thus, affected connections by network behavior are not usable to apply equation (1).The issue on replayed traffic trace: before capturing, the traffic traces to be
replayed by replay tools must not be processed by a DUT. Processed traces may have the modified packets or lack blocked packets. Therefore, processed traces cannot reproduce the same events on replay scenarios. Traffic traces used by a replay tool should be able to reproduce the same events occurred on .The issue of traffic identification: Each event on live traffic and replayed traffic
requires an identifier, especially when live traffic is different from replayed traffic.The differences between live traffic and replayed traffic can be of IP addresses and port numbers. Identifiers are used for comparing events in live traffic with those in replayed traffic.
4.2. Solution to Measuring the Effectiveness of Replayed Traffic
Figure 4 shows the solution process into four phases. The goal of the all the
16
To explain how to capture traffic, we illustrated the traffic flows in Figure 5.
There are two traffic flows and . Each flow is divided into two traffic flows.
Flow is divided into flow and . And flow is divided into flow and .
1) Capturing traffic process are captured. And from live traffic and from replayed traffic are recorded.
Steps to obtain the traffic trace for replay tools. First, capture the live traffic flows, and , and record . Second, split the traffic flow into and . Similarly, the traffic flow is separated into and . Third, merge
and . The merged traffic flow is a traffic trace without being processed by DUT, and will be used for replay testing. On the replay test bed, the first and second steps were applied to the replay traffic traces, and the traffic flows , , , and were obtained for further event reproduction analysis.
2) Processing live traffic and replayed traffic. Each traffic trace is divided into connections. In other words, network traces , , , , , , and are further processed into sets of connections, , , ,
, , , and . The connections are identified by 5-tuple (Src IP, Dst IP, Src Port, Dst Port, Proto), and the packets within each connection is identified by IP identification number, TCP sequence number, and packet payload.
Traffic used to create live events and replay events. Live events are created from and , while replay event are created from and . The packets
𝑇𝑐𝐿 𝛽
𝑇𝑐𝐿 𝛼 𝑇𝑠𝐿 𝛽
𝑇𝑠𝐿 𝛼
𝑇𝑐𝐿 𝑇𝑠𝐿
17
within these connections haven’t been modified by the DUT or lack blocked packets by the DUT. Therefore, we can compare all the events produced by live traffic and by replayed traffic.
Algorithm 1 shows the process to create packet events. The variable Connection is defined as an element from one of
,
,
and
.6 Directory ← source directory
7 Dest_File ← Connection
8 Packet ← Read tcpdump network packet from Directory [Connection]
9 Open Dest_File for writing
10 Else-If Traffic direction = destination
11 Then
12 Directory ←destination directory
13 Dest_File ← Connection
14 Packet ← Read tcpdump network packet from Directory [Connection]
15 Open Dest_File for writing
16 Else
17 Then exit (error)
18 EndIf
19 For each Packet X to end of Packets
20 Do Decode Packet X
21 If Packet.protocol = TCP
22 Then print to Dest_File "IP_ident, tcp_seq, payload, event_value"
23 Else
24 Then print to Dest_File "IP_ident, payload, event_value"
25 EndIf
26 EndFor
27 Close Dest_File
The description of Algorithm 1 is as follows. Function Decode_connection accepts an element from one of , ,
and
(line 1). The variableevent_value and the structure Packet [] are initialized (line 2 to 3). The variable traffic_direction stores the traffic direction of the element Connection and decides
what block is executed depending on the stored value in traffic_direction (Line 4, 10).The values of Connection that are classified as server side of the traffic, are categorized with traffic direction as “destination” and the values of Connection that are classified the client side of the traffic, are categorized with traffic direction as
“source”. The PCAP file directory of the connection, Directory, and the destination directory for the output, Dest_File, are initialized (line 6 to 7, line 12 to 13). A Perl function of Tcpdump decodes each packet of Connection to store it in the structure
18
Packet [] (Line 8, 14). A file Dest_File is allowed to use it for writing (line 9, 15). A
loop read all the values in the structure of Packet [] to extract the fields of IP identification number, TCP sequence number, and the payload (line 19-27). Each packet in the structure Packet [] is filtered (line 21). Additionally the extracted values are stored in an output file Dest_File (line 24).Live traffic with logs produces , and replayed traffic with logs yields . The pair of connections ( , ) are used to create a series of live events and . The pair of connections ( , ) are used to create a series of replay events and . Packets within the above pairs of connections are treated as packet events. Logs were compared to define the connection events. Each entry on and was assigned to a connection event, associated with its corresponding connection. Connections not registered in or were taken as events with non-specific attribute, i.e. non-logged attribute.
3) Comparing packets within pairs of connections ( , ), ( , ), ( , ) and ( , ), to identify blocked packets and modified packets. The comparison was done using a modified open source program [12] that can compare two PCAP files. The results of the comparison are the blocked packet and modified packets. This information is used to assign modifying event and blocking event. The packets, which are no in this result, are treated as passing events.
4) Comparing events , , , , and to calculate the event reproduction ratios and the effectiveness of replayed traffic. The event orders in the sets of must be the same order as that in the set of . The order of the event sets ensures the correctness of event comparison between live traffic and corresponding replayed traffic. For packet event each event will be ordered based on two fields: the TCP sequence number and the IP identification number. Thus, the two set of events are going to have the same order. Connection events do not need to order for the calculation of the reproduction of events because they are compared on the basis of information of the 5-tuple.
19
Algorithm 2 shows the procedure of event comparison. The variable EventA is defined as one of , and . The variable EventB is defined as one of .
Algorithm 2: Event comparisons
1 Event_comparison(EventA[], EventB[]) 2 TN, TP, FP, FN, i ←
3 While TRUE 4 Do i ← +
5 Last unless end of EventA or EventB 6 If EventA[i] = 0 and EventB[i] = 0
7 Then TN++
8 Else-If EventA[i] = 1 and EventB[i] = 1
9 Then TP++
10 Else-If EventA[i] = 0 and EventB[i] = 1
11 Then FP++
12 Else-If EventA[i] = 1 and EventB[i] = 0
13 Then FN++
14 EndWhile
15 Print “TN, TP, FP, FN, i”
The description of Algorithm 2 is as follows. The function Event_comparison analyzes two set of events for input (line 1). All variables are set to zero (line 2). The loop read all the elements from both set of events to compare the values in each of them (line 3 to 13). The statement last_unless checks if there are elements in the event sets, otherwise it breaks the loop (line 5). The print statement gives the results (line 15).
If the replayed traffic is different from the live traffic, we use identifiers while comparing events. These identifiers link the live traffic and the replay traffic. They showed the changes in the fields of live traffic fields, such as IP address and port numbers.
20