Rollback system recovery and VM-based log analyzer are very popular in recent years.
With zero day attack raise, computer security faces an unprecedented challenge. Virtual machine can provide an isolated environment for analyzing malware. It is also used for standalone system debugging as honeypot. Following are related works on replay system state:
2.1. Jockey
Jockey[12] is a record and replay tool for single process debugging in Linux. It do not need to change the target binary and any programming language or API. It can record the non-deterministic data by pre-loading as module. Jockey segregates resource of target program for avoiding being compromised. But it cannot record hardware level non-deterministic event such as memory access races, thread scheduling interrupt and interrupts. Jockey only focus on one simply process debugging but cumbersome whole-system debugging.
2.2. Flashback
Flashback[13] provides rollback and replay ability for software debugging. It forks a new process as shadow processes for checkpoint. Then it captures the memory state at specific execution point and interaction between processes. Nevertheless, Flashback cannot replay thread dependency correctly because it is hard to trap the interrupt of thread scheduling.
Without the correctly context switching, the dependency of thread would be change at re-execution. Thus, even Flashback is a lightweight replay for software debugging; it cannot
6
debug some problems with hardware interrupts.
2.3. Revirt
Revirt[7] logs enough information for replay even in long-term system. Because of based on virtual-machine monitor (VMM), it needs to modify the kernels of guest OS. Revirt consist of two parties to monitor, one is guest user, and another is guest kernel. Both of them are building on host system as processes. By delivering signal SIGUSR1, the guest kernel can trap the system called by guest user. Additionally, it records non-deterministic events to follow a set pattern by using SIGIO and SIGSEGV. Revirt also replace some instruction can return non-deterministic results. Specifically, the rdtsc (read timestamp counter) and rdpmc (read performance monitoring counter) get CPU's information directly. It replaces that functionality by using other time-related system call. Thus the environment of whole-system would be some differences from real. Moreover, Revirt only replays specified guest system and have some restricts for guest OS. This feature makes Revirt losing generality for many applications. A useful replay system must achieve a transparent and general replaying for general purpose.
2.4. XenLR
XenLR[8] is achieved on a lightweight VM (Mini OS) replay. It causes a little time and space overhead to log the keystroke and time updating on Mini OS. XenLR do not think about file system and process interaction because of Xen, a vitalization VM so that many non-deterministic events cannot be capture. But in real system, the file system and threads are usually an essential part of the system.
7
2.5. BugNet
BugNet[14] focuses on replaying application‟s execution and sequence of memory access, not a whole-system replay. It collects the execution information of program before crash. It uses the First-Load Log (FLL) to record the load instruction return value. They also record synchronization information by Memory Race Logs (MRL) so that it can replay race condition of memory.
Even BugNet can replay at least tens of millions of instructions with low overhead, but it only cares about memory access. There are still many events it cannot replay such as achieve faithful replay, all of external inputs in cache need to be recorded on each processor.
It can provide approximately 1 second replay because of the size limitation of record buffer.
However, FDR is hard to be applied widely because its short replay interval of system and hardware support. In our system, we use VM to make replay system more flexible to have long period replay.
2.7. ExecRecorder
Based on Bochs, ExecRecorder[5] perform hardware interrupts and whole-system replay.
It can replay the executions of entire system by checkpoints and logs of non-deterministic
8
events. A checkpoint is a duplicate of Bochs VM process via the fork system call. When replaying the system, ExecRecorder invokes the suspended child process by SIGUSR1. Same as our system, the implementation of ExecRecorder does not address DMA and multiprocessors. But Bochs has heavy computation overhead in emulation so that it is hard to be applied in large-scale analysis.
2.8. Summary
All of above replay system cannot guarantee that the instruction order is absolutely identical with previous run. Ignoring the instruction dependency does not impact the analysis result of execution because they only care about race condition between multi-threads. Mostly, there are many system issues in virtue of instruction dependency such as cause of program crash and activity of malware. By only synchronizing resource of processes, which erases much information for instruction level analysis. In our system, we focus on instruction dependency and reproduce the same execution order to ensure the faithful analysis result.
9
Name
Record Hardware Interrupts Record External I/O Determinism Hardware Support
Application Level Replay
ExecRecorder Yes Yes Multi-thread
dependency No
FDR Yes Yes Multi-processor Yes
Table 2.1 Related work comparison
10