Memory corruption detection aims at finding out invalid modification to specific memory locations in a sound and complete manner, so it can be treated as an application of memory watchpoint. This technique is usually used to combat attacks on critical targets we defined in Table 1.
The detector must have the knowledge about whether the modification is valid or not. For example, stack smashing modifies the saved frame pointer, and after function epilogue, the location of the saved frame pointer of the last frame will be reused and modified, so we have to differentiate an attack from normal operations.
StackGuard [17] uses a canary approach. A canary is a word which has a special value placed in front of the return address on the stack. The canary will be overwritten before the saved registers are modified. Prior to
function return, the canary is verified to see if the saved registers are safe.
Modern version of GCC has integrated the stack protector into itself and provide an option to enable it, and the canary is place before the saved registers. However, if the overwrite action is not sequential, the canary can be bypassed easily.
Backup-and-verify is another approach to justify the correctness of saved registers. When the registers are pushed onto stack, their value is also saved to another safe region. Before the function returns, the saved register’s value is compared to the backup value. Stack Shield [18] uses such method to detect memory corruption.
2.2.1 Detection and Diagnosis of Control Interception (Beagle) Our previous work Beagle also uses the backup-and-verify method to detect the memory corruption errors. The difference is that Beagle uses dynamic instrumentation and inserts checker calls at the end of each basic block of the dynamic instrumentor’s intermediate representations. In this way we can provide more accuracy with respect to the actual location of the memory corruption site. It is useful for debugging because finding out that the saved registers are corrupted at the end of a long function may be too far away from the bug which causes the corruption.
2.2.2 Automated Security Debugging Using Program Structural Constraints (CBones)
CBones is similar to Beagle, and the modification to memory targets is a subset of CBones’ program structural constraints [19]. It implements a two-stage approach: at the first level, insert the constraint checkers before
function return like Stack Shield to reduce overhead. If any constraint vio-lation is found, then re-run the program and instrument checkers after each write operation to provide more accurate debugging information. Although the coarse-to-fine manner helps on increasing the performance, if the pro-gram input is from outside, like network propro-grams, and not repeatable, then it can only report the error spot at the end of the function where the checker is. Also, CBones does not cover all the memory targets we defined in Table 1.
2.3 Combination
By combining data watchpoint optimization and memory corruption detection together, we can build a robust and cost-effective detector for se-curity or debugging purpose.
3 Method
3.1 Memory Write Notification via Page Protection by mprotect
The mprotect() function is used to change the protection status of a designated memory region. Most machine architectures provide the memory protection mechanism on page level. A memory page can be set to read-only, write-only, read and write capable, or no access. When a page is read only, any modification to it is treated as an invalid operation, and the hardware’s memory management unit (MMU) will generate an exception to inform the operating system. The kernel will send out a SIGSEGV signal to the access violating process. If we install a SIGSEGV handler, then we can monitor the memory modification to specific regions by setting the regions as read only.
3.1.1 Steps
We generalize our idea as the following steps:
1. In the malloc() function wrapper, use mprotect() to set the allocated chunk’s page to read-only.
2. Write to the read-only page will cause a segmentation fault.
3. Verify the correctness of the metadata in the SIGSEGV signal handler.
4. In the signal handler:
- If the metadata is corrupted, halt the program execution and print backtrace message for debugging.
- If the metadata is correct, pass the memory modification.
3.1.2 Two Stage Signal Handling
When fault happens, typically the reason of the fault should be fixed in the related signal handler. After the signal handler returns, the fault instruction will be restarted. In the SIGSEGV handler, if we do not unprotect the page, then the fault instruction will continue to generate a new fault, which forms an endless loop. So we have to unprotect the page to make it writable. But after the write instruction is executed, the following write instructions to the same page will not generate faults anymore because the page is now writable, and the asynchronous mechanism simply stops working.
Apparently the page should be again read only immediately after the write operation is done. A way to achieve that is to put the CPU into single-step mode. Single-single-step mode is usually used for debugging purpose. Many CPU architectures support this feature. In x86 architecture, there is one bit in the EFLAGS register indicating the on / off status of single-step mode called TRAP flag. When TRAP flag is set, CPU will issue a SIGTRAP signal after each instruction is executed. We can use this feature to reprotect the page in the SIGTRAP handler, and then clear the TRAP flag to leave single-step mode.
Two stage signal handling consists of the first stage’s SIGSEGV han-dling, and the second stage’s SIGTRAP handling. Figure 3 shows the flow chart of this method. Very similar method was proposed in VAX DEBUG by Beander et al. [12] back in 1983, but it has not been popular and widely implemented on mainstream debuggers until this decade when computing power is faster enough to make the overhead from signal handling compara-tively lower. For example, not until the release of Mac OS X 10.1 in 2001, the
Figure 3: Working Flow of Method 3.1
page protection watchpoint has been implemented on Mac OS X’s gdb [20].
3.1.3 Pros and Cons
In the method proposed above, there are some good perspectives with respect to other methods. First of all, we do not have to do read / write redirection. In EDDI’s partial instrument, it creates shadow pages and in-struments read / write operations to be redirected to access the shadow area.
And the memory usage overhead is twice. In contrast, our method do not allocate extra memory space. Also, we can implement this method by using
only function wrappers to replace the original malloc-related functions, and we do not have to deal with the dirty assembly instrumentation job. So the solution is pretty portable.
Though this method looks promising, it has several severe disadvantages.
The overhead in signal handling is too high, because one write operation triggers two signals, and four context switches happen. Not to mention that mprotect() is used twice in the signal handlers. Second, Valgrind just ignores the trap flag set in the CPU’s context, so it is not easy for us to integrate this method into our existing Beagle framework.