RELATED WORK

2.1 Cloudburst attack by Virtunoid – attacking the hypervisor

Virtunoid is a special malicious program that runs inside a VM and can take over a host. We use a VM in response to this attack and allow the VM to acquire host privileges after the attack. From monitoring the VM’s run-time status, by especially utilizing virtualized hareware devices of VM by qemu-KVM, VIS can determine that there has been a ‘unexpected’ change, since usually the VM owner is not permitted to modify virutalized hardware. VIS can use this information to determine if the VM should be migrated to a sandboxed host or shutdown to prevent similar attacks. This kind of attack attack is called cloudburst.

When simulating the cloudburst attack we use the virtunoid exploit, first published at BlackHat/DEFCON 2011 by Nelson Elhang [ 8]. We installed

Ubuntu

14.04 o

2.2 Malicious Behavior Detection

A running VM employs virtualized hardware and one of the VMs on the host. It is a process is executed by KVM, where we use another physical host as a “SandBoxed-Host” [ 29] and execute the malicious program in a VM. In this way we decide to use “strace” and “qemu-monitor” to collect the dynamic data from VM and compare a VM’s malicious behavior to models. Using “strace” this can be viewed as “black-box testing”, as the vector from which the VM can request the needed system call from the host . This system call then can be collected from the running VM for analysis. On the other hand, qemu-monitor can observe the integrity of virtualized hardware and determine if it is normal or being used inappropriately. The execution of a malicious program may come in several phases, such as vulnerability scanning, executing the attack, or embedding a backdoor. We separate these periods and record the data for analysis, much like TTAnalyze [ 15]. On the other hand, we classify the data collected by VIS from strace by period and role. Borrowing from I. Santos et al. [ 30], we use static analysis to disassemble the program and expose some suspicious “opcode”, in order to use several different kinds of analysis models to analyze the malicious software, as well as already known and classified normal software. Next, unknown software is put into the analysis model. The analysis model

14

then determines if the unknown software is malicious or normal. We also use this method to do a dynamic analysis: VIS uses the statistical result from data collected by “strace” to find out the status of the VM during the execution of a malicious program, as well as the the system call used at that time.

Simultaneously we use these sequence system calls to reanalyze and classify again by doing a similarity analysis as a reference basis of a malicious program.

There are two ways to extract software behavior models that could provide alternative solutions for analyzing a virtual machine’s behavior model. One is to retrieve properties of data values [ 5] [ 23] such as constraints on legal values in the form of a Boolean expression. The second way would be to examine the properties of interaction patterns such as possible interaction sequences in the form of final state machines [ 9]. However, neither of these models account for their mutual interplay.

Even when one takes both analysis models into consideration, it is hard to see their intricacies clearly.

Thus, in the future we will adopt a new analysis method, GK-trail, which would produce models in the form of extended final state machine [ 19]. These models were designed to represent the properties of the data values, interaction sequences, and provide insight into the interplay between the two. The extended final state machine shows itself to be a reliable model because it does not depend on the size of the analyzed systems, but rather the complexity of the interplay within the system's components. This makes the GK-trail quite flexible in analyzing different types of systems effectively.

However, as this model is still being tested (at least at the time of the paper), there may be uncertainties that come with it, and thus we have yet to incorporate it.

2.3 Virtualization Introspection

There are many research papers on the cloud environment working on defending from malicious programs or hackers such as the security risk evaluation of the cloud computing presented in detail by Enisa [ 7] . In Siebenlist [ 33], a number of security issues are discussed. There are also many interesting and worthwhile surveys about cloud security presented by Armburst [ 10]. However, all of these papers are discussing primarily attacks coming from guest machine users who turned said guest machine in the cloud computing environment into a malicious machine.

15

A cloud attacker may use a virtualization environment’s vulnerabilities to break into another VM when they detect the target machine in the cloud environment. In order to guard against the attack method stated above, most approaches use Virtual Machine Monitor (VMM) isolation properties to secure VMs by leveraging different levels of virtual introspection [ 16]. Virtual introspection [ 2] allows user to observe a VM’s state through this process. In previous papers surveyed, there are some useful approaches catching our attention. For example, SecVisor [ 32] Lares [ 26] and KVM-L4 [ 27], CUDACS [ 21], among others, leverage virtualization to monitor the integrity of a guest kernel code from a privileged virtual machine or from the VMM, or a hypervisor. One research Security Analysis of Cloud Management interfaces [ 34] focuses on the XSS security on Amazon Cloud’s cloud management interfaces which can control VM instance. but we use another commodity open source virtualization management interface called virsh. Advanced Cloud Protection System (ACPS) [ 22] is one mean of improving the security of cloud nodes. ACPS is an extension of the KvmSec [ 20] and KvmSma [ 22]

which are also known as extensions of the Linux KVM. ACPS is a protection system that is totally transparent to cloud environments and can also monitor the cloud component, both locally and via remote, to protect the whole cloud. However, none of these papers mentioned how to protect cloud computing platforms from cloudburst attacks and have no mechanism, like migration, in this paper.

The component of Virtualization is a native tool for monitoring a VM, such as QEMU-monitor and VMM. VIS is using the QEMU-monitor to monitor the hardware components of a VM. EagleEye [ 36] changed the QEMU device in the hypervisor instead of using QEMU-KVM or QEMU-monitor, and EagleEye is only for Xen hypervisor which is only for para-virtualization. Most of the virtualization monitoring mechanisms needed to modify the hypervisor or VMM, which makes monitoring mechanisms more difficult to deploy in the real cloud ecosystem, prove more difficult to fix a bug if it is not a mainstream cloud computing platform which is supported by companies or large open source users.

Another work about botnet profiling [ 14] and detection, uses VMM directly to collect information without installing an agent in the guest OS, and analyzes the behavior patterns of bots from the API calls. It has both passive detection and active detection. Passive detection is based on a modified QEMU hypervisor (VMM) to examine data by a bot to check bot behavior profiles. Active detection is the active fingerprinting on bots to detect different bots performs it’s malicious jobs, by utilizing specific stimuli to examine if bots perform expected behavior, through bot profiling,

16

fingerprinting and distinguishing bot families. One upside is that it can profile bot families and use the specific stimuli to examine known bots. The downside, however, is that it needs to modify the Virtual Machine Monitor (VMM) as a bots detection system. On the on hand, passive detection can only observe particular bot activities and then trigger the security alarm. It can only detect the bot or backdoor behavior. This work has no intrusion prevention mechanism, however, after the bots are detected, the hacker has already successfully hacked the system.

Nitro [ 28] is a VMI-based system for system call tracking and monitoring supports in Linux, Windows 64/32bit system, but only works on Intel x86 hardware architecture. It also utilizes machine learning to classify the system calls or API calls (actions) of malicious process in the guest OS, as some sandboxing environments (honeypot environments) will be monitoring the particular system calls or API calls. Nitro also needs to modify the KVM by adding new commands called Nitro Commands and send those new commands to the KVM kernel module and the I/O control interface. The outputs are then sent to the proc filesystem by creating a node in the proc filesystem. The strength about Nitro is that it has a Hardware-based tracking system that performs well while simultaneously providing transparency that guest OS will not award itself in being monitored. On the other hand, Nitro only works on specific hardware and is not easy to replace current popular cloud hypervisor drivers, such as VMware Workstation/Player, VirtualBox and OpenVZ. Nitro only supports KVM.

There is another related work about virtualization inspection by using periodically integrity check on the state of VMs. The key advantages of VMI IDS (Livewire) [ 11] include: being an isolated VM and using a modified VMM for monitoring purpose. Given no VM can avoid being monitored by VMM, Livewire is able to conduct inspection on a VM by using VMM.

Livewire periodically compares the integrity of the state of a VM via a modified VMM of VMWare. When monitoring a VM, Livewire needs to copy the state of a VM first which interrupts the monitored VM. Because of this, Livewire is a costlier monitoring system, and easier to be discovered by intruders of the malicious VM.

Comparing to VIS, Livewire only checks the integrity of VMs, while VIS has several more features as following. VIS doesn’t need to interrupt monitored VMs while monitoring them, so it is less

17

possible to be discovered by hackers or intruders. Also, VIS doesn’t need to modify the VMM, and is able to support different cloud hypervisors other than VMWare, which is good for the deployment in the cloud system. It also has an online migration feature. Moreover, VIS can distinguish differences between different malicious behaviors and further recover compromised VM by using snapshots of VM images.

18