In this chapter, we will evaluate the functionality, performance and scalability of ProxyReplay with HTTP protocol. The evaluation of functionality focus on two proxy behaviors: web-caching and content filtering. The performance focus on the throughput of the two replay strategies: preprocessing and replay, and concurrent replay. Third, the evaluation of scalability replay focus on how large the input file can be supported by ProxyReplay. Last, we will compare different replay tools using different types of DUT.
5.1 Test Environment for ProxyReplay
As shown in Figure 18, we use two computers to do the experiment including a proxy server and a replay machine. Both of the computers have two interfaces which are connected to each other. We use squid3 as a proxy daemon running on the HTTP proxy. Next, we use Bind9 as a DNS server and run the ProxyReplay on the replay machine.
Figure 18: The test environment for ProxyReplay
Before starting the experiment, it is important to make sure the completeness of the input PCAP file. For HTTP application replay, we need to find a corresponding response message for each request since an HTTP connection should include at least one pair of request and response. Therefore, we should remove the traces which
cannot be identified since its application header was lost. Table 7 gives a statistics of the input pcap file which is used in this experiment. The PCAP file records 2.6 giga bytes HTTP network flow in 20 minutes. The number of connections is 10964 including 363 HTTP clients and 916 HTTP servers. The number of requests and response is 26825 and 27420 accordingly. Last, the request and responds we can replay is 21166 pairs.
Table 7: Statistics of the input PCAP file
File type HTTP PCAP File Number of responses will be cached 6074 Number of responses will not be cached 15092 Number of requests will be filtered (.jpg) 7134 Number of requests will not be filtered 14032
5.2 Functional Evaluation
In this section, we evaluate the functionality of ProxyReplay. In order to make sure the correctness of replay, we make the comparison between predicted value and replay result for each proxy behavior. The predicted value is calculated by analyzing the packet trace.
5.2.1 Web-Caching
Before replay, we predict the number of cacheable responses according to the web caching rule which is shown in Table 8. The response will be cached if it satisfied the 5 factors including request method, status code, Cache-control header field, Expiration and Dynamic content.
Table 8: Web caching rules
Cacheable Non-cachable
Request method GET, HEAD POST, PUT, DELETE,
OPTIONS, TRACE Status code 200, 203, 206, 300, 301, 410 Other status
Cache-control header field public, private, max-age>0 no-cache, max-age=0, no-store Expiration and validation Date < Expires Date >= Expires
Dynamic Content Others Query terms
(“?” mark in URL)
Figure 19 shows the results of replay with and without web caching. Without caching means that we replay to a web caching proxy which does not have any cache data. On the contrast, with caching means that the proxy we have replayed once and the proxy stores some cache data. In Figure 19, the number of messages between client and proxy is the same ether with caching or without caching since simulated client will replay all requests and all correspond responses should be received.
Without web caching, the number of messages between proxy and server decrease a little bit because some requests with the same URL were cached. With web caching, we found that about 16263 requests did not be cached which is similar to the non-cacheable number of requests 15092. Due to each proxy daemon has their own caching rules, the number of non-cacheable requests will not the same with our estimation but they will be similar.
Figure 19: Results of ProxyReplay with and without web caching 5.2.2 Content Filtering
In order to evaluate the functionality of content filtering replay, we configure the
21166 21166 21166 2116621166 20952 20952 2116621166 16263 16263 21166
0
ACL (access control list) rule of the proxy server. The types we can filter out by ACL including Source/Destination IP address, Source/Destination Domain, Words in the requested URL (Ex: .jpg, .gif, .exe), Current day/time, Destination Port and HTTP Method.
In this experiment, we configure the proxy to filter out all .jpg requests and the total request will not be filtered out is 14032 as shown in Table 7. Figure 20 shows the result of with content filtering and without content filtering. We found that the sum of the number of requests from proxy to server 13845 and the number of cached response 187 which is the same with our predict number 14032.
Figure 20: Results of ProxyReplay with and without content filtering
5.3 Performance Evaluation
In this section, we compare the performance between our two replay strategies including “preprocessing and replay” and “concurrent replay”. Figure 21 shows the result of the performance evaluation. We recorded the bandwidth of replay in each second. The result of two replay strategies is shown as preprocessing and concurrent.
In Figure 21, we found that the throughput of preprocessing strategy is higher than concurrent strategy since parser will decrease the performance of replay. In order to achieve application stateful replay, parser needs to construct the application data structure, modify payloads and update DNS message. Therefore, parser would be the bottleneck since replayer should wait for the parser until the application data structure
21166 21166 21166 2116621166 20952 20952 2116621166 13845 13845 21166
0
is constructed. On the other hand, the parse time of concurrent strategy is shorter than preprocessing strategy since parser and replayer starts concurrently.
Figure 21: Performance of two replay strategies
Figure 22 shows the request rate of two replay strategies using the 2.6GB HTTP PCAP file as input. For preprocessing strategy, the average request rate can achieve 1200 requests per second. For concurrent strategy, the average request rate about 200 requests per second which is limited by the performance of parser. However, when parser finished parsing, the average request rate can achieve 1000 requests per second.
Figure 22: The request rate of two replay strategies
Table 9 shows the throughput of ProxyReplay tool in different size of responses.
0
We configure the size of each response to 1.5 KB, 15 KB and 50 KB. The average request rate of ProxyReplay tool is 1300 requests per second. We found that the throughput of ProxyReplay depends on the size of the response since the speed of data transfer is faster than establish connections. If the size of response is larger, ProxyReplay tool will spend more time on data transfer.
Table 9: Throughput of ProxyReplay tool in different size of responses
Size of response 1.5 KB 15 KB 50 KB
Throughput (Mbps) 28.18 106.74 244.60
5.4 Scalability Evaluation
between the two strategies. For “preprocessing and replay”, the average replay speed can achieve 200 Mb/s but the parse time is long and the input file size is limited by memory size. For “concurrent replay”, the average replay speed is only 103.1Mb/s but the parse time only needs 2 second for initial parsing and there is no limitation for the input file size.Table 10: Comparison between two strategies
Preprocessing and Replay Concurrent Replay Parse time Depending on the file size 2 sec (initial parsing)
Average replay speed 200 Mb/s 103.1 Mb/s
Input file size Depending on the memory size No limited Test Case
Cons 1. Longer parse time 2. Limited be memory size
1. Low replay speed (parser is bottleneck)
5.5 Replay Tools Comparison Evaluation
In this section, we make a comparison between different existed replay tools. As shown in Table 11, the replay tools including TCPReplay, NATReplay, SocketReplay and ProxyReplay. We configure the computers which are listed in Figure 18 to NAT, transparent, router and proxy mode. As shown bellow, TCPReplay supports transparent mode and router mode device and the throughput is 363Mb/s and 288Mb/s accordingly. NATReplay can support NAT devises and the throughput is about 320Mb/s. SocketReplay supports TCP/IP stateful replay on transparent and router devices and the throughput is 40Mb/s and 12Mb/s separately. The last, ProxyReplay is the tool supports proxy mode devices and the throughput can achieve 200 Mb/s.
Table 11: Comparison between replay tools
TCPReplay NATReplay SocketReplay ProxyReplay
Transparent (363Mbs) X (40Mbs) X
Router (288Mbs) X (12Mbs) X
NAT X (320Mbs) X X
Proxy X X X (200Mbs)