Embedded TaintTracker is evaluated on effectiveness and performance. In effectiveness evaluation, we reproduced a return address smashing attack against a vulnerable echo server. For the performance evaluation, we took the most widely used web server, Apache, as the testing target and evaluated the impact on latency, throughput and the sustainable number of requests per second.
4.1 Effectiveness
A buffer overflow attack has to first inject malicious code into victim’s memory space and then corrupt code pointers of different types including return address, function pointer, longjmp buffer and GOT. Many solutions for buffer overflow defense were proposed to prohibit code pointer corruption, so their effectiveness should be evaluated for the enumerated code pointer types. However, our system, based on taint tracking technique, does not prevent code pointer corruption but avoid malicious code execution since any type of corrupt code pointers is finally targeted to execute malicious code. Thus we only need to verify whether it blocks malicious code execution to demonstrate that it is able to defend against buffer overflow attacks.
Our test program was an echo server [23] with a synthetic vulnerability that copies the string received from the client into local buffer without bound checking, and sends it back to client. The vulnerability is exploited when the copied string exceeds the size of the local buffer to inject malicious code and overflow return address, where the malicious code adds a new account for the attacker. Fig. 10 shows the system log after the attack was lunched. As shown in the figure, Embedded TaintTracker successfully identified the attack and logged what and where the system
call was invoked. Besides, the log also recorded the value of instruction pointer to the last function invoked for bug fixing and dumped the memory near the address of system call invocation for signature generation.
Attack Identification
Bug Fixing Signature Generation
Fig. 10. The log from Embedded TaintTracker after detecting an attack.
4.2 Performance
We measured the performance degradation of Embedded TaintTracker on an Apache web server [24] which is the most widely used server on the Web [25]. There are three key criteria for performance evaluation including latency, throughput and sustainable number of requests per second. Our evaluation was performed on a system with an Intel Core 2 Duo T5600 (1.86 GHz) CPU and 2 GB of RAM, running Ubuntu 7.10 on Linux kernel 2.6.22.
In order to compare with previous work, TaintCheck, and profile the source of overhead, we additionally measured the performance of Apache with the kernel component of Embedded TaintTracker, Valgrind Nullgrind [26] and MemCheck [27], denoted as Embedded TaintTracker - Kernel, Nullgrind, and Memcheck, respectively, in Fig. 11. The kernel component of Embedded TaintTracker is used to measure the performance overhead when the mechanism that examination of execution on taint data is alone enabled. Nullgrind and Memcheck, like TaintCheck, are extensions of
the emulator Valgrind. These extensions have diverse degrees of instrumentation that can represent two primary sources of overhead from TaintCheck. Nullgrind does not instrument any additional instructions, implying that the extra execution time is caused by the emulator Valgrind itself. MemCheck is a substitute for TaintCheck since the source code of TaintCheck is unavailable. MemCheck looks for memory leak and illegal memory access by means of the same data structure as TaintCheck to trace the status of memory and instrumentation on all memory operations. Its performance is better than TaintCheck because TaintCheck requires extra interception of each jump-instruction, and the author of TaintCheck also had demonstrated that the performance of TaintCheck is worse than that of MemCheck [6].
In the evaluation of latency, we requested different size of web pages from 1 KB to 10 MB, and timed how long it took to connect, send the request, receive the response, and disconnect to server. In order to prevent resource contention in the test bed, the server was connected to another machine running the testing program. The testing program was executed five rounds, and for each round, we requested the same page 60 times. The result is the average median in each testing round.
0.5
Fig. 11. Experimental result of performance evaluation: a) latency in different page sizes requested, and Y-axis is the slowdown factor which is latency of mechanism-enabled divided by native execution time;
b) and c) are degradation on throughput and requests per second for different numbers of clients. The native results are listed below X-axis with parentheses.
Fig. 11 (a) shows the result of latency with a slowdown factor which is execution time of measured target divided by execution time of Apache. The slowdown factor decreases as the size of the request grows because the server becomes less CPU-bound and more I/O bound. Embedded TaintTracker generates a 1.37 slowdown in case that 1 KB page is requested and almost no overhead when the size of the accessed page exceeds 100 KB. The performance of MemCheck is much worse than our system especially when the page size is less than 100 KB.
The throughput and sustainable number of requests per second for different numbers of clients were evaluated with WebBench, as shown in Fig. 11 (b) and (c).
On average, our system imposes only 9.3% (73.48 KB/sec) performance degradation which outperforms, by eight folds, the 75.2% (592.08 KB/sec) performance degradation caused by MemCheck. Running Apache under Valgrind
already brings a great overhead of 60% (358.78 KB/sec) in the degradation of MemCheck. Dynamic instrumentation of all memory access operations and memory information maintenance contributes the remaining 40% (233.3 KB/sec) overhead, which would increase in proportion to the number of instrumented operations and result in the overhead of TaintCheck to be more acute. For example, when there are twenty clients, memory access and jump represent 31% and 8% of operations respectively. TaintCheck would impose extra overhead from instrumentation of the additional 8% jump operations for checking malicious execution.
Fig. 11 also shows that the kernel component of Embedded TaintTracker influences slightly performance, meaning that the major overhead of our system is not from the examination of execution on taint data but from maintenance of taint information. Thus we further profiled time consumed in subroutines for maintaining taint information table. When 1 KB pages are requested 1000 times, 61% of the extra time is spent on the bit-copy subroutine which is used to copy taint status from one bit to another, and another 36% is spent on address translation for page tables. The overheads from these subroutines may be reduced as we designed them to reduce performance overhead rather than memory usage. For example, the time for address translation can be diminished by changing the structure of taint information table to bitmap.