用虛擬機內省技術強化OpenStack雲端安全機制 - 政大學術集成

全文

(1)國⽴立政治⼤大學資訊管理學系碩⼠士學位論⽂文指導教授：蔡瑞煌博⼠士、︑､郁⽅方博⼠士. 立. 政治大. ‧ 國. 學. ‧. ⽤用虛擬機內省技術強化 OpenStack 雲端安全機制. sit. y. Nat. Enhancing OpenStack Cloud Security with. n. er. io. Virtual Machine Introspection al v i n Ch engchi U. 研究⽣生：李彥亨中華民國 104 年 7 ⽉月.

(2) Abstract Today, we attributes it to virtualization technology that the application of cloud computing is so well-developed that the world-wide famous company can make use of this technique to reap the profits, just likes Google and Amazon etc. While cloud service bringing kinds of benefit to system vendors and cloud tenants, cloud security is exposed to many threats. Traditionally, two main kinds of intrusion detection system (IDS) are host-based IDS (HIDS) and network-based IDS (NIDS). With. 政治大 superior to HIDS and NIDS both on isolation and visibility properties as far as cloud 立. virtualization technology development, virtual machine monitor (VMM) based IDS is. security concerned.. ‧ 國. 學. We address a cloud security protection framework, called Virtualization. ‧. Introspection System for OpenStack (VISO), to strengthen OpenStack security. y. Nat. defensive mechanism. VISO has some following characteristics. (1) VMI based. er. io. sit. monitoring mechanism (2) behavior-based analysis (3) elastic to expand system functionality and easy to operate (4) all apparatuses in VISO are free on Internet that. n. al. is why we also choose the deploying cloud environment.. ni C h famous private Ucloud most engchi. v. solution, OpenStack, to. About our experiment method, we using supervised and unsupervised artificial technology algorithm to analyze behaviors monitored in a sandbox environment. All malwares are downloaded from OWL Taiwan official malware knowledge base and labeled by anti-virus scanner. The purpose is to see how effective the features of behaviors collected by VISO can recognize the same family malwares. Detecting unknown malware variants previously not recognized by commercial anti-virus software by training the same family known malware samples.. i .

(3) Contents Abstract ........................................................................................................................... i Contents .........................................................................................................................ii List of Figures .............................................................................................................. iii List of Tables ................................................................................................................ iv List of Appendixes ......................................................................................................... v Chapter 1. Introduction ................................................................................................ 1. 1.1. Background and Motivation ...................................................................... 1. 1.2. Approach: Open Source Cloud Security Solution Based on VMI ............. 2. 1.3. Contribution ............................................................................................... 4. 1.4. Content Organization ................................................................................. 5. 2.1. Hardware Virtualization, Virtual Machine Monitor and OpenStack ......... 6. 學. ‧ 國. Chapter 2. 政治大 Related work .............................................................................................. 6 立 Virtual Machine Introspection ................................................................... 9. 2.3. Artificial Intelligence ............................................................................... 10. 2.4. Malware Detection ................................................................................... 12. 2.5. Clustering and classifying malwares ....................................................... 13. 2.6. Uniqueness of VISO ................................................................................ 14. 3.3. y. sit. er. al. v i n C h of VISO Framework Components and profiles e n g c h i U ....................................... 19 Detail of Operation in Training and Testing phase .................................. 21 Architecture and Deployment .................................................................. 16. n. 3.2. Methodology ............................................................................................ 16. io. 3.1. Nat. Chapter 3. ‧. 2.2. 3.3.1. Training phase.............................................................................. 22. 3.3.2. Testing phase ............................................................................... 28. Chapter 4. Experiment ............................................................................................... 35. Chapter 5. Conclusion ............................................................................................... 45. Reference ..................................................................................................................... 46 Appendix ...................................................................................................................... 52. ii .

(4) List of Figures Figure 1.. Deployment of OpenStack and VISO ...................................................... 17. Figure 2.. Profile of compute-node .......................................................................... 17. Figure 3.. Illustration of VISO host online operation............................................... 18. Figure 4.. Components of VISO............................................................................... 18. Figure 5.. Operation of training phase ...................................................................... 21. Figure 6.. Operation of testing phase ....................................................................... 21. Figure 7.. Example of DLL list ................................................................................ 31. Figure 8.. Example of psxview before remarks........................................................ 31. Figure 9.. Example of psxview after remarks .......................................................... 32. Figure 11. Figure 12.. 政治大 Example of VM level perspective vectors ........................................... 33 立 Example of attribute list ....................................................................... 33 Example of process level perspective vectors ..................................... 34. 學. ‧ 國. Figure 10.. Example of hidden process detection (1) ............................................. 38. Figure 14.. Example of hidden process detection (2) ............................................. 38. Figure 15.. Example of hidden process detection (3) ............................................. 38. Figure 16.. Example of hidden process detection (4) ............................................. 39. ‧. Figure 13.. n. er. io. sit. y. Nat. al. Ch. engchi. iii . i n U. v.

(5) List of Tables Table 1.. Hidden process checking routine ............................................................. 34. Table 2.. Class list used as label.............................................................................. 35. Table 3.. Class list used as label.............................................................................. 37. Table 4.. Example of malware list with label ......................................................... 37. Table 5.. Result of test1 for rule2............................................................................ 41. Table 6.. Result of test1 for rule3............................................................................ 41. Table 7.. Result of test2 for rule2............................................................................ 42. Table 8.. Result of test2 for rule3............................................................................ 42. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. iv . i n U. v.

(6) List of Appendixes Appendix 1.. Rule2 GHSOM predicting model in test1............................................ 54. Appendix 2.. Rule3 GHSOM predicting model in test1............................................ 79. Appendix 3.. Rule2 GHSOM predicting model for test2 .......................................... 81. Appendix 4.. Rule3 GHSOM predicting model for test2 .......................................... 92. Appendix 5.. Attributes of vectors of ldrmodules raw files .................................... 110. Appendix 6.. Decisive attributes of each class in rule2 in test1 .............................. 112. Appendix 7.. Decisive attributes of each class in rule3 in test1 .............................. 122. Appendix 8.. Decisive attributes of each class in rule2 for test2 ............................. 124. Appendix 9.. Decisive attributes of each class in rule3 for test2 ............................. 134. 政治大. Example of resolving memory snapshot ........................................ 135. 立. 學 ‧. ‧ 國 io. sit. y. Nat. n. al. er. Appendix 10.. Ch. engchi. v . i n U. v.

(7) Chapter 1 Introduction 1.1 Background and Motivation The essential reason of the prosperity of cloud computing is that virtualization technology makes it cost effective that system vendors can lend their cloud computing resource to VM tenants by sharing hardware, such as CPUs, memory, disk … etc. This prominent development achieves an innovation that people regard the cloud. 政治大 on their kernel products and rent the computing power paid on demand so that they 立 computing resource as utilities. On one hand, small and medium enterprises can focus. can reduce expenditure of managing hardware. On the other hand, cloud service. ‧ 國. 學. vendors, who may provide IaaS, PaaS or SaaS cloud services, can pay full attention to. ‧. manage their cloud datacenter, like scalability, reliability and varieties of cost issues,. y. Nat. to offer the highest qualities for their co-operative clients. All of them can benefit. er. io. sit. from this kind of collaborative business model. There are so many advantages and merits about cloud service that we can’t enumerate the whole reasons for developing. n. al. cloud technology.. Ch. engchi. i n U. v. However, like 2009_Cloud_Computing_Security_Risk_Assessment saying [9], cloud’s economies of scale and flexibility are both a friend and a foe from a security point of view. The massive concentrations of resources and data present a more attractive target to attackers. If certain hackers, who know cloud architecture well, have malicious motivation to breach the cloud service, they can pretend a normal VM tenant and do something bad in cloud environment. On account of cloud environment, one small mistake or vulnerability can cause a chain reactions so that one company may severely lose people’s confidence and reputation of a firm. Both computing resources providers and end users may face the serious threats of their private data or 1 .

(8) enterprise operation process. Furthermore, in cloud using virtualization technique, the unit of resources is VM. Due to analogy between VM and physical host, cloud may not only need to come up against conventional host-based or network-based attacks but new cloud-based trick as well, to say nothing of various and unknown attacks changing with each passing day. That is why securing a cloud environment is the most important factor that has majorly affected the growth of cloud computing. The real event is that the big breach took place twice in Sony enterprise on May 13th, 2011 and November 26th, 2014. It is unbelievable to hear such crazy news in. 政治大 negative image. It also tells us a truth that the importance of cloud security cannot be 立 an internationally famous company. This seriously make cloud security an undeniably. overstated [27].. ‧ 國. 學 sit. y. Nat. on VMI. ‧. 1.2 Approach: Open Source Cloud Security Solution Based. n. al. er. io. Facing such kinds of potential threats, we need more robust cloud-based defenses. i n U. v. instead of traditionally host-based or network-based defensive systems anticipating. Ch. engchi. that it can recognize existing attacks via given specified patterns created from malware process and even learn varieties rule patterns of brand-new hacker tricks by designated training task. VISO was born for guiding cloud security. It offers the online VM monitoring mechanism based on VMM technology so that it can observe VM behavior with isolation and visibility. According to Tal Garfinkel Mendel Rosenblum’s “A Virtual Machine Introspection Based Architecture for Intrusion Detection,” [12] the properties, Isolation and visibility, are necessary norms for assessing a defensive system. Those are inherited from VMM which is the software responsible for virtualizing the hardware of a single physical machine and partitioning 2 .

(9) it into logically separate virtual machines. Isolation means VISO cannot be accessed from virtualization level. Malicious VM has no idea of the sign of monitoring, so VISO can do the best to collect VM behavior data instead of being compromised like conventional in-host antivirus system. Visibility signifies transparent observation. VISO can view many kind of information related to VM, such as VM address space, disk files, network sockets, etc. In simple terms, whatever we want to get from VM, it never has ability to dismiss. In other words, VMM is the mechanism that facilitates the construction of a VMI IDS to make programmer capable to inspect the VM from. 政治大 innate properties in VMM, isolation and visibility, which are not only indispensable 立 the outside to analysis the software running inside. VISO completely has the two. characteristics but the core spirits of VMI IDS as well [12, 41].. ‧ 國. 學. Therefore, VMI technique has become the mainstream detection mechanism,. ‧. which is known as virtualization introspection monitoring VM behaviors from. y. Nat. the angle of hypervisor through VMM by permission. We also can find out so many. er. io. sit. implements of cloud security based on VMI technique like our system in this article [4, 15, 21, 33, 46]. With the expanding of VMI technology development, many. al. n. v i n researchers spend much time onC exploiting the field ofU h e n g c h i virtualization introspection, but. there is still a big problem obstructing the growth of that. We call this subtle issue semantic gap, which presents a disconnection between the low-level state that the hypervisor sees and its semantics within the guest kernel. To this day, computer science is more mature than before. People take advantage it on business for a long time and seem to more considerably rely on it, too. Certainly, security problem has already existed prior to cloud concept. Most threats like Virus, Worm, Trojan horse, Rootkit, Backdoor, Botnet, Spyware, Adware, etc. often referred to a malware, have been bullying around the world since many years ago. In order to perceive hacker’s bad actions and abate the damages caused by malwares, we need to 3 .

(10) know what malwares are and what negative effects they offer, and then react to it. Fortunately, we have permission to access the OWL malware database, which is devoted by predecessors. In this database, we can get hundreds and thousands malware source code that is trapped into honeypot set by security governments in Taiwan. In other words, we have the real malwares coming from corners throughout the world supplied to us for research. In our experiments, many kinds of defensive system in market such as antivirus may have inferior isolation, but when we use it to statically scan download source programs from OWL database, they could be our best. 政治大 references. So, in this paper, we propose a defensive system called VISO to show how 立. helper. In such case, the information they advised will be one of our comparable. to use our methods, such as some unsupervised or supervised artificial intelligent. ‧ 國. 學. algorithm, to solve the semantic gap, which is a big problem everyone in this field. ‧. wants to overcome by their own methodology, and try to extract the behavior of. y. Nat. current live VM. The behavior mean what information patterns we collected from VM,. er. io. sit. and we all know a VM, analogous to a host, comprise a lot of processes running in themselves. The mission for VISO is judging whether there has a malware processing. al. n. v i n in VM or not by means of ourCown techniques. OfU h e n g c h i course, our own techniques to. judge a VM will be introduced in latter part of this paper. Finally, we check the correctness of VISO by comparing the results to existing defensive systems in market, like Avira Antivirus, and check whether VISO achieves our expectation.. 1.3 Contribution VISO is a framework for fully automatic and systematic analysis of malware behaviors using intelligent learning. We hope to substitute for the traditional IDS and effectively use it in current cloud environment, especially OpenStack, where we. 4 .

(11) cannot ask all tenants to install specific monitor program in their rental VM. The VISO framework can derive effective detection rules on attacks for enhancing cloud system security and can be deployed on any Linux OS host with KVM virtualization platform. VISO consists of two characteristics. (1) VMI based monitoring mechanism (2) behavior-based analysis (3) elastic to expand system functionality and easy to operate (4) all apparatuses in VISO are free on Internet that is why we also choose the most famous private cloud solution, OpenStack, to deploying cloud environment. In order to completely make VISO cooperate with the centralized property of OpenStack,. 政治大 design is dedicated to immense corporate cloud using cloud technique to solving 立 we offers host-crossing online centralized monitoring service. Although all detailed. every potential problem, we present an uncomplicated operation stage for cloud. ‧. ‧ 國. 學. administrators.. 1.4 Content Organization. sit. y. Nat. al. er. io. In chapter 2, we introduce technical background knowledge you should know to. v. n. completely understand our research, and we compare the difference and the same with. Ch. engchi. i n U. related research. In chapter 3, we go through the detail of methodology in our research, including how to implement offline training, online testing and the way of tests to present our research result. In chapter 4, we show the result of our research and discuss meaning of this result. Chapter 5 is the conclusion.. 5 .

(12) Chapter 2 Related work In this chapter, we are going to specify technical background knowledge, shown in 2.1 ~ 2.2, within VISO and append the researches which are using similar technology in related domain. Then we point out the uniqueness of VISO in last section.. 2.1 Hardware Virtualization, Virtual Machine Monitor and OpenStack. 政治大 Virtualization technology 立 help us costly handle multipurpose work on different. ‧ 國. 學. OS (Operation System) by sharing the same host. Hardware manager can manage resources more efficiently and conveniently by virtualizing hardware. Hardware. ‧. virtualization or platform virtualization refers to the creation of a VM that acts like a. sit. y. Nat. real computer with an operating system. Software executed on these VMs is separated. al. er. io. from the underlying hardware resources. For example, a computer that is running. v. n. Microsoft Windows may host a VM that looks like a computer with the Ubuntu Linux. Ch. engchi. i n U. operating system; Ubuntu-based software can be run on the VM. In hardware virtualization, the host machine is the actual machine on which the virtualization takes place, and the guest machine is the VM. The words host and guest are used to distinguish the software that runs on the physical machine from the software that runs on the VM. The software or firmware that creates a VM on the host hardware is called a hypervisor or Virtual Machine Manager. Different types of hardware virtualization include: [45]. •. Full virtualization. 6 .

(13) A complete simulation, not emulation, of a real hardware to allow software can run on VM with no modification. •. Partial virtualization Not complete simulation for a VM, so some guest programs may need. modifications to run in this virtual environment. •. Paravirtualization The guest programs could be executed as though they are running in their own. isolated domains, a separate system, because the hardware environment does not be. 政治大. simulated. Guest programs need to be specifically modified for running in the. 立. environment.. ‧ 國. 學. A hypervisor, so called virtual machine monitor (VMM), is just a piece of computer software, firmware or hardware that creates and runs VMs. A computer on. ‧. which a hypervisor is running some of VMs is defined as a host machine, and VMs is. y. Nat. sit. called a guest machine, or called guest OS. The hypervisor presents the guest OS with. n. al. er. io. a virtual operating platform and manages the execution of the guest OS. Multiple. i n U. v. instances of a variety of OS may share the virtualized hardware resources, the. Ch. engchi. hardware virtualized by hypervisor. There are roughly two types of hypervisor: [16]. •. Type-1: native or bare-metal hypervisors These hypervisors run directly on the host's hardware to control the hardware. and to manage guest operating systems. That is why they have the other name called bare metal hypervisors. A guest operating system runs as a process on the host and it is usually faster than on hosted hypervisors. •. Type-2: hosted hypervisors These hypervisors run on a conventional OS just like other computer programs. do. Type-2 hypervisors abstract guest operating systems from the host operating 7 .

(14) system. KVM, VMware Workstation and VirtualBox are examples of type-2 hypervisors.. In general, people use VirtualBox or VMWare etc. For a simple instance, after we developed an application program on Windows-NT environment, we want to test its performance on Linux kernel. Most people may install VirtualBox or VMWare in their PC or notebook and create a virtual environment by VirtualBox or VMWare, so called hypervisor. It seems that we can use another new computer with impending or. 政治大 the hypervisor and control it to create a VM through the VMM panel. That is a basic 立 short demand, and then delete it when I don’t need it. All you need to do is to install. use when it comes to VM. So, cloud business can put it to use in their large number of. ‧ 國. 學. computing source like thousands or millions of server level computers. The vendor of. ‧. computing power farm out computing source by a unit of VM and gain reputation. y. Nat. from the scalability and reliability. How do they manage such a great quantity of VMs?. er. io. sit. Maybe there are kinds of methods in each business, but there is a one open source way for a general people in a corporation does not have enough capital. The solution. n. al. is OpenStack which is a. i n free,Cwell-document and U open hengchi. v. source cloud computing. platform with widely adopted commercial/industrial cases. Users primarily deploy it as an infrastructure as a service (IaaS) solution [30]. There also has a VMM of OpenStack called horizon. Horizon makes you think a whole cloud sources like a very big computer because OpenStack already chain all the computing sources in the cloud together. OpenStack still has additional function like monitoring performance of tenant’s VM or offering a lot of API for administrator to manage VMs on the cloud [29]. For example, we can create, delete, freeze or copy VM for many purposes. In our case, These APIs is our helper to control the malicious VM when VISO found it.. 8 .

(15) 2.2 Virtual Machine Introspection Virtual Machine Introspection, VMI for short, is a technique refers to VM’s self-criticism. Due to virtualization technology, we can regard VM’s hardware as a software. In virtualization environment, VM is created by hypervisor, so we can access VM’s memory, hard disk, CPU registers and so on. Further, we make use of information collected from target guest OS by accessing VM’s hardware, only if we have administration permission. But, there are a big challenge in VMI research field, so called semantic gap [6,. 政治大. 11, 17, 40]. Fortunately, accumulating people’s efforts, there has some VMI tools to. 立. help successor easily bridge this gap for their needs like LibVMI, which is a virtual. ‧ 國. 學. machine introspection library. This means that it helps you access the memory of a running VM. LibVMI provides primitives for accessing this memory using physical. ‧. or virtual addresses and kernel symbols. LibVMI also supports accessing memory. y. Nat. sit. from a physical memory snapshot, which is helpful for debugging or forensic analysis.. n. al. er. io. In addition to memory access, LibVMI supports memory events. Events provide. i n U. v. notifications when registered regions of memory are executed, written to, or read, but. Ch. engchi. so far they are only available with Xen virtualization platform currently [22, 32]. Another memory forensics tool is well-known Volatility. It not only can be used in a dependent host, but also associated with LibVMI for introspecting [21, 22, 32, 41]. Volatility offers a lot of convenient and flexible API for user and is an open source python based extensible framework that assists investigators whether they be forensic examiners or malware analysts. The frameworks assists them in the examination of physical memory dumps, crash dumps and hibernation files. Like Michael Sevilla says, they can find hidden process using Volatility [41]. Volatility. 9 .

(16) currently has support for many OS environments such as Windows-NT or Linux-based memory images [23, 31]. VMI technique has many advantages that traditional IDS or antivirus doesn’t has. According to B. D. Payne, “Secure and flexible monitoring of virtual machines,” [34] such as no superfluous modifications to the VMM, ability to monitor any data on target OS and target OS cannot tamper with monitors, total 6 VMI norms mentioned in paper, and that why VMI becomes the mainstream of the malware detection [5, 15, 18, 28, 35, 42, 46]. VISO certainly do best to apply all those norms.. 治政 2.3 Artificial Intelligence 大立 ‧ 國. 學. Artificial intelligence (AI) is the intelligence exhibited by machines or software. It is an academic field of study which studies the goal of creating intelligence. There. ‧. are many subfield under AI like machine learning, natural language processing,. sit. y. Nat. motion and manipulation and so on. In this paper, we principally access the machine. al. er. io. learning [24] field which has support vector machines (SVMs) [43] and some of other. v. n. neural network algorithms like growing hierarchical self-organizing maps (GHSOMs). Ch. engchi. i n U. [13]. Machine learning algorithms are applied to a lot of aspects due to their powerful classification, clustering or regression. In generally, we can divide all machine learning algorithms into three categories, supervised, unsupervised and reinforcement. Roughly speaking, if someone say it use machine learning algorithm to classify the data, we can infer that it use some kind of supervised learning algorithm. Additionally, when it comes to regression like statistics usually using, it also is a kind of supervised learning algorithm like SVMs using in this paper. Only if his application wants to accomplish the object of clustering, we can ascertain that he will use an unsupervised learning algorithm like GHSOMs using in our research.. 10 .

(17) •. Supervised Learning Algorithm The supervised means we need to give computer a “teacher” so that it can learn. from what the teacher teaches. In above phase, we usually call training. After training, we will give him the real problems, and expect it can answer correctly. In this phase, if you still give it the known problem on purpose of checking what the learning efficiency it attained, this process is testing. Once you already have believed that the computer is ready for predicting the unknown problem, we call this phase predicting. In a word, supervised learning is presented with example inputs and their desired. 政治大. outputs, given by a teacher, and the goal is to learn a general rule that maps inputs to outputs.. 立. Unsupervised Learning Algorithm. 學. ‧ 國. •. A set of inputs is to be divided into groups. Unlike in classification, the groups. ‧. are not known beforehand, making this typically an unsupervised task. In detail, all. y. Nat. we need to do is just setting up some of configurations. The case in GHSOMs, we set. er. io. sit. the two main parameters, T1 and T2, to give it a reference whether growing horizontally or vertically when processing. We didn’t know how many sizes and. al. n. v i n levels in the end result. On the C other hand, in the supervised h e n g c h i U algorithm, we must need to give the number of groups and each label in advance, and it returns the output which must also in this scope. That is the biggest difference between a supervised and unsupervised learning algorithm. •. Reinforcement Learning Algorithm A computer program interacts with a dynamic environment in which it must. perform a certain goal such as driving a vehicle, without a teacher explicitly telling it whether it has come close to its goal or not. Another example is learning to play a game by playing against an opponent. This type of machine learning algorithm didn’t. 11 .

(18) use in our experiment, but it can be taken into account in the feature. We will spend time on being familiar with this type of algorithm.. 2.4 Malware Detection Today, the major threat to Internet is various malware like Virus, Worm, Trojan-horse, Rootkit, Backdoor, Botnet, Spyware, Adware and so on. These kinds of program running on the host or VM result in accessing user’s private data or breach of system crash [8]. General person who is not familiar with security conscience or. 政治大 is so popular. People depend 立on a trusted software that can help them monitor system. computer science may be a target victim of malware authors, so that is why antivirus. ‧ 國. 學. states and hinder malicious program wreck their important system or data. In cloud ecology, we usually cannot ask VM tenant to install monitor agent in their rental VM.. ‧. Of course no one want to be monitored, and IDS inside host or VM has higher risk of. sit. y. Nat. being compromised by malware, even though it has better visibility. Although there is. al. er. io. type of IDS so called network-based IDS and has good isolation, it is poor at visibility,. v. n. too. A new malware detection solution, VMI, gradually become a substitute for. Ch. engchi. i n U. conventional IDS. When it comes to malware detection, we means using a variety of method to extract malware feature and recognize it in the future. There are two type of detection way depending on whether the malicious program run on the OS or not. •. Static analysis Analyzing malware and extracting several useful features from the executable in. inspection without executing it. The feature extracted from executable binary code often depends on its semantics, treated as signature. If hacker changes its code semantic, maybe the malware can evade monitoring of the static analysis detector. So, about static analysis, the main challenge is to create a key pattern that makes it. 12 .

(19) difficult for an advanced, semantics-based malware detector to properly determine the effect of a piece of code [1]. Most of modern antivirus products also have robust static analysis functionality but for the reason mentioned above it is why an antivirus always need to update and upgrade after a period of time for new discovered unknown malicious software. No malware expects to be detected before doing something [6, 25].. •. Dynamic analysis. 政治大 run-time. One ideal dynamic analysis manner is expected to discover a malicious 立 In contrast to static techniques, dynamic techniques analyze the code during. program when it executes an unwarranted instruction. That means you find out the. ‧ 國. 學. criminal caught red-handed [6]. Comparing to static analysis, if you define. ‧. unwarranted instruction as a strong feature, you will regard all program with this. y. Nat. specific instruction as a malware. In such situation, you may cause a lot of erroneous. er. io. sit. judge which is a serious effects on clients in cloud service. We usually call this type of miscarriage type error in statistics. There are a variety of example of dynamic. al. n. v i n C hbe found in previous malware analysis related work can achievement of predecessors, engchi U just like collecting system call [44], using VMI technique for kinds of information,. especially Dynamic link library (DLL) used by malware in VM [7] in this paper, and others [3, 4, 14, 15, 19, 21, 33, 36, 46].. 2.5 Clustering and classifying malwares Many machine learning approaches, like Decision Tree, Support Vector Machines (SVMs), Growing Hierarchical Self-Organizing Maps (GHSOMs), K-centers and so on, have been proposed for malware detection field, because of their. 13 .

(20) prominent clustering and classification functionality [1, 2, 10, 20, 26, 37-39, 47, 48]. Clustering means identifying novel classes of malware with similar behavior, that is features extracted from malware, and classification refer to assigning unknown malware to these previously known classes. Based on clustering and classification, we can create an incremental approach for malware behavior-based analysis. Like Konrad Rieck, “Automatic analysis of malware behavior using machine learning,” saying [31]. Comparing to static analysis, due to novel malwares, behaviors observed during run time are more complicated and difficult to recognize characteristics. In. 政治大 machine learning algorithm to cluster and classify malware behaviors which can’t be 立. such circumstance, It may get unpredictable results using powerful sophisticated. distinguished with naked eye [31][36].. ‧ 國. 學 ‧. 2.6 Uniqueness of VISO. sit. y. Nat. The experiment of the most related researches has its specific prerequisites. They. al. er. io. have a little consideration of the situation of introducing the methodology to business.. v. n. We need a solution which can be used as cloud dataset to be our experiment. Ch. engchi. i n U. environment, so we can be familiar to the integration between the academic circle and industry. We also take the real cloud circumstance, which must consist of a lot of computers and need to take a centralized management into account, so VISO implement the cross-host monitoring and data collecting. The cloud solution we chose is OpenStack. Its kinds of advantage are mentioned previous section. The researchs on OpenStack security are all about the process of account management or key token protection. The hacker attacking on VM is also an essential issue in OpenStack cloud, though it always happened in many cloud. 14 .

(21) platforms, but OpenStack nothing to do it. So, one of our contributions is to show that VMI technology also can be deployed on business private cloud. On the aspect of behavior analysis, we use Volatility VMI tools to offer more general method, which are formatted with easy-handled vectors and roughly solve part of semantic gap problem. For the remaining semantic gap problem, we provide the elastic framework to easily integrate kinds of machine learning algorithm, like GHSOM or SVM. We also use some of popular or unpopular data collected from VM to show the practice of our idea, like DLL information extracted from ldrmodules. 政治大 VISO cloud security framework has a lot of potential to develop more effective 立. plugin.. method to against a variety of malwares. Because of malware’s obfuscation, we really. ‧ 國. 學. need a mechanism of growing generating detection rule having good expandability so. ‧. that it can resist the variant malwares depending on learning experience.. n. er. io. sit. y. Nat. al. Ch. engchi. 15 . i n U. v.

(22) Chapter 3 Methodology 3.1 Architecture and Deployment In this work, we present a new framework called Virtualization Introspection System for OpenStack (VISO) that can monitor dynamic status of lived guest VMs to detect and prevent the cloud ecosystem from potential malicious attacks. VISO has an aggressive, real-time monitoring mechanism with the aim of achieving efficiency, precision and transparency.. 政治大. In order to offer an open-source solution for cloud security, we leverage. 立. OpenStack to build our KVM-based cloud environment and use VISO to monitor. ‧ 國. 學. VMs dynamically and derive detection rule. The deployment of OpenStack and VISO is shown as below in Figure 1. OpenStack has two necessary components called. ‧. Controller node and Compute node [29]. Compute node is the major resource of. y. Nat. sit. OpenStack. It means the VMs targeted by VISO are launched on compute node, so as. n. al. er. io. hypervisor. Therefore, we install some another plugin on compute node and collect. i n U. v. VM behavior from that. VISO is a software on another host or maybe you also can. Ch. engchi. install on the same host as controller node for centralizing data collection and behavior introspection. The duty of VISO is to monitor the virtual machine based on KVM hypervisor. It is convenient that VISO uses OpenStack NOVA command line or restful API, which is the service exported on the controller node, to interact with the VMs. That is to say, we can write scripts to automate our defense rules when VIS detects abnormal VMs. For example we can stop malicious VMs when detect attacks such as cloudburst attack or migrate victim-VMs when they are compromised.. 16 .

(23) 政治大 Figure 立 1. Deployment of OpenStack and VISO ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Figure 2. Profile of compute-node. 17 .

(24) 政治大. Figure 3. Illustration of VISO host online operation. 立. ‧. ‧ 國. 學 er. io. sit. y. Nat. n. aFigure iv l C 4. Components of VISO n he gchi U The operation in VISO has two n phases: training and testing. The profile of controller node is the same as installation in OpenStack install guide. The profile of compute-node is shown in Figure 2. All of VISO’s components are shown in Figure 4. The training phase consists of three steps, shown in Figure 5, in this order: data collection, behavior analysis and rule derivation while the testing phase consists of two steps, shown in Figure 6, in this order: runtime observation, behavior detection and defense reaction. We will discuss them mentioned above in more detail later in this chapter.. 18 .

(25) 3.2 Components and profiles of VISO Framework In this section, we will exhibit the profile of each node in VISO framework and explain the duty of each component. Figure 2 shows profiles in compute node, in which Ubuntu is the bottommost element, KVM hypervisor next. Kernel-based Virtual Machine (KVM), a full virtualization hypervisor for Linux, is installed in compute node for running multiple virtual machines. VISO can see what virtual machines doing in terms of Virtual Machine Monitor, VMM for short. This technique is called Virtual Machine Introspection (VMI). Well, what is the term "monitor". 政治大. means? It means that VISO can observe the behavior of a virtual machine and. 立. response special action to VM through OpenStack built-in manage API when it is. ‧ 國. 學. suspicious of an attacker or invader. When it comes to VMI, like Figure 62, the related VMI plugin is installed on Virtual Machine Manager (VMM). In this figure,. ‧. VMM means the control panel of hypervisor. It offer API for programmer of. y. Nat. sit. administrator. One well-known cross-hypervisor Virtual Machine Manager is Libvirt,. n. al. er. io. which is collection of software that provides a convenient way to manage virtual. i n U. v. machines. These software pieces include an API library, a daemon (Libvirtd), and a. Ch. engchi. command line utility (Virsh). Due to support of many kinds of hypervisor like KVM, VMWare Xen, etc. OpenStack takes advantage of this asset to achieve the integration of multi-hypervisor cloud ecosystem and just manage them in one manner. LibVMI is a VMI tool. It also read VM related information from hypervisor through Libvirt. The functionality of LibVMI is so many. It not only can get data from memory and disk of target VM, but also network and other hardware. In our article, we focus on memory forensics, we have another tool to cooperate with LibVMI for helping us bridge some part of semantic gap. The tool name is Volatility, one of the well-known memory forensic kit. Volatility usually use in dependent PC host, or you can give it a memory 19 .

(26) image file. In our system, the target memory or the memory image file is replaced with VM memory address space, which is offered by LibVMI. That is so-called the association between LibVMI and Volatility. So, if you prepare all of environment settings mentioned above in this section. We can read related data we care about, just like DLL library loaded by each process in VM memory address space through calling command exported by Volatility, so we also can write script to automate these works. On the host VISO side, we can roughly divide VISO into two main part, training. 政治大 worker, called detection worker, compares VM behavior got from detection manager 立 phase and testing phase. When data is collected from VM on compute node, the. to rule table for checking whether VM is normal or malicious. The result of matching. ‧ 國. 學. will be temporarily stored in event object. When event object created, they are going. ‧. to be sent to reactor. Reactor check the content of event object and decide to do which. y. Nat. the routine defined in event table, like stop, delete, suspend VM through OpenStack. er. io. sit. API. This phase is called Predicting, but in experiment stage we call it “Testing” and it is doing online, see Figure 3. On the other hand, we need to derive rules for testing,. al. n. v i n so the phase which purpose is C to derive detection rules h e n g c h i U is called “Training”. We use. well-known AI algorithm to create rules, so training phase is doing offline. Both. training phase and testing phase has its sub-components. We are going to clarify each of these in detail in next section.. 20 .

(27) Figure 5. Operation of training phase. 立. 政治大. ‧. ‧ 國. 學 sit. y. Nat. n. al. er. io. Figure 6. Operation of testing phase. i n U. Ch. v. i Testing phase e n g c hand 3.3 Detail of Operation in Training The operation in whole VISO system is mainly divided into two stage, training phase and testing phase. The purpose of training phase is to create the detection rules which is necessary to monitoring in testing phase. In this article, we use AI algorithm to create our detection rules and the data collected from VMs is the information about DLL library of target VM. After having the detection rules, we create a table of rules and use it to filter the malicious behavior of VM which is judged by rules. According the behavior of VM, you can define the reaction of the event. Comparing results of testing phase with our known VM behavior, we can know whether our rules is 21 .

(28) effective enough and then adjusting it in training phase. So, above is the main work in testing phase. We expect to create an impeccable rules through training and testing repeatedly.. 3.3.1. Training phase. There are three sub-components in training phase, the order of flow path is data collection, behavior analysis and rule derivation, shown in Picture 5. Training phase is an offline task and doing on the VISO host. We are going to explain details of each step in the following section. •. 立. Data Collection. 政治大. ‧ 國. 學. VMs in cloud are scattered over different hosts. The way to implement host-crossing data collection is based on SSH protocol. Because every host has. ‧. hypervisor to manage instance on them, we can search for getting the host name. sit. y. Nat. which the specific instance run on with two OpenStack API, nova hypervisor-list and. al. er. io. nova list –host {hostname}. We install VISO on trusted host which exchanges SSH. v. n. key with all computenode so that VISO can access VM by script without. Ch. engchi. i n U. administrator typing command in front of terminal. In order to avoid the lag caused by SSH network transmission, we add “UseDNS no” argument to SSH daemon configuration file. On the computenode, we install something like Figure 2 saying. So, we can request DLL list from VMM host OS by volatility API “ldrmodules” plugin. The data of DLL list is like Figure 77, called ldrmodules raw file. It show the state of VM in a moment about DLL library used by each process in that moment. The column “InLoad”, “InInit” and “InMem” mean the DLL library which source is found at path shown in “MappedPath” column. There are also kinds of information we can get from. 22 .

(29) VM like process list offered by volatility “psxview” plugin scanning for helping you find hidden process, like Figure 8, 9. Figure 9 is the result of remarks of Figure 8, and we will explain that later. We call the file of Figure 9 psxview raw file. It offers seven data structure in one Windows-NT guest OS, which of them can record the processes existed in one process. The column names are alias names given by Volatility for seven data structures. By cross-reference between these seven data structures, we can say some suspicious process intend to do something malicious. We automate the collection task in batch with python script. The script logic. 政治大 of files, including ldrmodules or psxview. Each of them represents the memory state 立. algorithm shown in experiment chapter. After once collection task, we can have lots. of target VM in a moment, because the Libvmi freezes the memory of target VM and. ‧ 國. 學. copy the memory image to Volatility, then Volatility extract the information based on. ‧. your given plugin, like ldrmodules or psxview, and output into files, like ldrmodules. y. Nat. raw file or psxview raw file. Why Volatility memory forensics tool can accurately. er. io. sit. extract expected information from such complicated OS data structure of target memory without error? It is because that we give the complete profile information. al. n. v i n C hOS version and target about target OS including OS type, VM hardware architecture, engchi U like Win7SP1x86. The example of resolving memory snapshot to extract DLL library used by a process is shown in the appendix10.. •. Behavior Analysis After collecting log data from VM, we program python script to parse log file. with regular expression to dig out what we care about. In our case of ldrmodules raw file, one of each file means the state of all DLL or EXE used by all process in the target VM at a moment. After a period of time of collecting, we have lots samples of ldrmodules raw file, so we extract the “mapped path” column in the file and gather 23 .

(30) statistics of them to know the final DLL attribute list as Figure 10, supposed from a1 to an. In accordance with this list, on the term of VM level, we can count the accumulation of times that DLL is used by a VM at a moment. So we call the result of count is DLL distribution of one VM at a moment, like Figure 11. On the other hand, we can switch our perspective to process level. One ldrmodules raw file also can be parsed into several vectors as Figure 12. Each vector means what DLL used by one process in target VM. So, we got the two set of vectors from last step. Both of them are gotten from ldrmodules raw file. The difference is the level of perspective. One is. 政治大 “DLL distribution for entire VM”, and the latter is “DLL list for a process”. If they 立. on VM level perspective, the other is on process level perspective. The former is. are extracted from the same dataset, they will have the same attribute list a1 to an.. ‧ 國. 學. There is also a difference in range of their value of each attribute. The range of value. ‧. of attribute of vector of DLL distribution for entire VM is a natural number which. y. Nat. means the number of processes using this DLL library. On the other hand, the range. whether the process is using this DLL library.. al. er. io. sit. of value of attribute of vector of DLL list for a process is a binary which means. n. v i n C hthe seven kinds ofUdata structure that can maintain About psxview raw file, show engchi. process list. The column is the seven kinds of method used to find process list in each kind of data structure. We use cross-reference of different source of process listing to help us judge whether process is suspect of hidden process. The brief explanation of. these seven data structure is following, the detail of Windows OS data structure can be found in “The art of memory forensics” [23]. As aforementioned psxview plugin, volatility framework offer seven different data structures information for you to conclude whether a process existing or not. Pslist: using symbol to locate the begining address of _EPROCESS and then traverse ActiveProcessLinks data structure within _EPROCESS . Psscan: the pool-scanning approach to find _EPROCESS. Thrdproc: 24 .

(31) because every process must have one main thread, you can scan for _ETHREAD objects and then map them back to their owning process. Csrss: csrss.exe is involved in the creation of every process and thread (with the exception of itself and the processes that started before it). Thus, you can walk this process’ handle table, and identify all _EPROCESS objects that way. PspCid: using symbol to find out the address of PspCid table that stores a reference to all active process and thread objects. Session: the SessionProcessLinks member of _EPROCESS associates all processes that belong to a particular user’s logon session. Deskthrd: these structures store a list. 政治大 thread back to its owning process. 立. of all threads (tagTHREADINFO) attached to each desktop, and you can easily map a. While collecting ldrmodules raw files for a target VM, we completely retain. ‧ 國. 學. whole information about the target VM and what malware sample runs in it. So, when. ‧. converting log files into vectors, we can easily label each vector with class number.. y. Nat. Of course, we also have the table to maintain each class number symbols which is. er. io. sit. parsed from anti-virus scanning reports. Because the malwares are download from OWL web and there is no related malware metadata, we use anti-virus in the recent. al. n. v i n market to do static analysis on C the malwares we test U h e n g c h i on experiment. That is why we. know each malware class and label it, such as “W32”, “TR” and “WORM” etc. Anti-Virus usually use signature-based detection to do pattern match, so the scanning result always be trusty. But, unfortunately, because of malware variability, some of malware downloaded from OWL database cannot be recognized by commercial anti-virus, we give it a special label named “Unknown”. The reasons we chose DLL library as our behavior feature is that DLL library is a shared library which offers general function in an OS [7]. When a malware want to do something malicious, it usually need to use some DLL library which is responsible for accessing some system service or hardware interface. Some malwares may infect 25 .

(32) or pretend some important DLL library which is frequently used by other normal processes. So, we believe DLL library is a special feature which can represent a malware behavior to a certain extent. The experiment in this research will prove this hypothesis. On the other hand, it is very intuitional to doubt a VM is malicious when we find any hidden process in a VM. That is why we use psxview to check. And, there is ever never a malware can hide from all seven data structure at the same time without any error when its purpose is that it want to be portable between many Windows NT OS, said by “The art of memory forensics” [23].. •. Rule Derivation. 立. 政治大. The purpose on this step is to use the result offered by last step to create your. ‧ 國. 學. own rules for known malware. Like many research, one rule to detect malware is finding whether there is hidden process. See figure 8, the data tell us where the. ‧. process is recorded. The string “False” indicate that the process is missing. So, if a. y. Nat. sit. process is missing at column “pslist” but it can be found at all other 6 locations, it. n. al. er. io. means correspond process has intention to hide itself. Before using this rule, we. i n U. v. should do a remark job which adjusts some exception normal status. Processes that. Ch. engchi. start before csrss.exe (including System, smss.exe and csrss.exe itself) are not in the CSRSS handle table. Processes that start before smss.exe (including System and smss.exe) are not in the session process or desktop thread lists. Processes that have exited will not be found by any of the methods except process object scanning and thread scanning (if _EPROCESS or _ETHREAD still happens to be memory resident). After remark job, we will get the file like Figure 9. String “Okey” means normal status, although “False” found at first. So, we put Table 1 logic on Figure 8 file and then we can clearly get the result about whether a VM has malicious hidden processes. 26 .

(33) Another rule we are going to introduce is created by some of artificial intelligence algorithms. After preparing two set of vectors, DLL distribution of an entire VM and DLL list of a process, we can convert each set of data into AI algorithm input format file to do clustering or classifying. In this article, we chose unsupervised neuron network algorithm, GHSOM, to respectively cluster two set of data. GHSOM clustering result is a tree data structure which node is a neuron and sibling nodes are a map data structure with h * w neurons (h is the length of height of map, w is the length of width of map). A neuron has a weight vector. That is an. 政治大 with several vectors is. important point of our rule. The whole tree is our predicting model, so as our rule. A neuron object has a property. 立. a cluster. In the ideal. circumstance, there will have several cluster with the same label on each member in. ‧ 國. 學. the same cluster. That is mean the behavior of the malwares or normal processes can. ‧. be featured by the DLL it has loaded or used. We use vectors parsed from ldrmodules. y. Nat. raw file to run GHSOM clustering and try to find pure malware cluster. But, in fact,. er. io. sit. there usually some impurity in target cluster that members in this cluster are almost labeled as malware. So, our rule is that if the number of malware m1 in a cluster is x. al. n. v i n C h (like malwareUm2, malware m3 etc.) is y and and the number of the other malwares engchi the percentage of x in x+y is 50%, we can say this cluster has a good feature of malware m1 and mark a property in this neuron object. But, if any normal process in a. cluster, the cluster is marked as “Normal”. In a special case, we might find out a cluster with kinds of malwares but no normal process in it and no one has a 50% number. We can say this cluster has feature of kinds of malwares. So we mark this cluster as “Malware” label that means we suppose this cluster represent a malware character but we cannot clearly judge which malware class it belonging to. The strategy of creating predicting model is strict for “Normal”, because there is a lot of criticism if you has false predicting on real normal one in business. After rule 27 .

(34) derivation step, we got the GHSOM predicting model as our detection rules. The examples of GHSOM predicting model are shown in appendixes. The method we use models to detect VM unknown behavior vector will specify in Testing phase section.. 3.3.2. Testing phase. Testing phase is online task. It continuously sampling raw file and detect behavior vector to check whether a live VM is malicious. In the experiment, we prepare a set of testing data to check the accuracy of our rules and then we define the. 政治大 behavior detection, and defense 立 reaction.. reaction for each situation. The flow path of testing phase is runtime observation,. Runtime Observation. ‧ 國. 學. •. Like data collection step in training phase, we collect ldrmodules and psxview. ‧. raw file by the same way. The little difference is that we extract vector or vectors. y. sit. al. er. Behavior Detection. io. •. Nat. when every single raw file got. Then, do the behavior detection step instantly.. v. n. In this step, we use a message queue framework called Celery to do analysis for. Ch. engchi. i n U. each rule on one VM. Celery provide python API to launch worker working for task we assign. The default number of worker is the number of cores the analysis cluster can offer, and a task is defined by VM, rule, and timestamp. The VM is defined by related information of VM, such as VM id, VM host name of its position in cloud cluster, VM guest OS type and so on. We clearly record the all information about VM, because not every rules can be eligible for every VM, for every malware, or even for every host OS, like system call collected by strace OS process monitoring command. For each rule we also can defined the time shift between two detections.. 28 .

(35) Rule1 is for hidden process. If a process found at all data structures except column “pslist” (_EPROCESS), we can roughly suppose this process intend to hide itself and detection worker would throw the hidden process found event. The completely logic is shown in Table 1. The result of Table 1 algorithm is the content of the event object. Rule2 is using DLL distribution for entire VM vectors, we use GHSOM output tree model with our special mark handled in the training phase rule derivation step. For each vector extracted from raw file, we compute the distance with each neuron’s. 政治大 neuron. So, we can output the mark of this neuron as our event. 立. weight to find the closest neuron, and then we can know this vector belongs to the. Rule3 is to match the DLL list for a process vectors extracted from a raw file. It. ‧ 國. 學. will have its own GHSOM tree model and the matching method is as same as rule2.. ‧. The only difference is that the event output will not be normal if any vector in the. y. Nat. same raw file has abnormal mark found.. er. io. sit. In our experiment, we will have test1 and tes2 for each rule2 and rule3. In test1 and tes2, for each sample, we dump 6 raw files and we have a set of vectors extracted. al. n. v i n C hvector extracted from from each raw file. For rule2, one one file. For rule3, multiple engchi U vectors extracted from one file, because one VM has multiple processes in it. We. compute the distance with each neuron’s weight to find the closest neuron, and then we can know this vector belongs to the neuron and output its mark as predicting result of this vector. So, we can know all predicting results of one sample. If all predicting results of one malware has any normal mark, we say this sample is normal. If has any not normal mark, we do voting mechanism to see which not normal mark is more possible. Therefore, for each sample, we can get final predicting result. Checking the final predicting result with label scanned by anti-virus. For malware samples, we can know how much malwares we can recognized. For normal sample, we can know how 29 .

(36) much normal VM vectors we have a mistake on it. The result of tests will be presented in experiment chapter. •. Defense Reaction Defense reactor is an agent responsible for catching event. Then, it take the event. information to compare with the event table that records the response for each event type prepared in advance. The responses are implemented by OpenStack API, command line or REST style or you can decide the special routine response by yourself.. The OpenStack API includes freeze, suspend, stop, or delete and so on.. 政治大 Both rule2 and rule3 are using GHSOM model to do predicting job. They must 立. ALL of API can be found on the OpenStack official website [29, 30].. involve a question. How degree of accuracy they display? About this question, we has. ‧ 國. 學. a lot of experiment of these method. And the final real data, OWL malware database,. ‧. experiment results are shown in the experiment chapter.. n. er. io. sit. y. Nat. al. Ch. engchi. 30 . i n U. v.

(37) Figure 7. Example of DLL list. 學. ‧ 國. 立. 政治大. ‧. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Figure 8. Example of psxview before remarks. 31 .

(38) 立. 政治大. ‧ 國. 學. Figure 9. Example of psxview after remarks. ‧. n. er. io. sit. y. Nat. al. Ch. engchi. 32 . i n U. v.

(39) 立. 政治大. ‧. ‧ 國. 學 er. io. sit. y. Nat. al. n. v i n C h Example of attribute Figure 10. list engchi U. Figure 11.. Example of VM level perspective vectors. 33 .

(40) Figure 12.. Example of process level perspective vectors. step1.. function remarks(). step2.. foreach process do {. 立. 政治大. if (pslist == False) { go to Step3 }. ‧ 國. 學. }. return OK. ‧. if ((pspcid && csrss && session && deskthrd) == False) {. y. Nat. io. } else { return suspect }. n. al. sit. pass. er. step3.. i n U. v. Table 1. Hidden process checking routine. Ch. engchi. 34 .

(41) Chapter 4 Experiment In our experiment environment, we use OpenStack based on hypervisor named KVM as virtualization environment. The guest OS is Windows7, service pack 1, and x64 hardware architecture. The malwares is obtained from OWL database which is maintained by official government deploying many honey pot around Taiwan (http://owl.nchc.org.tw/km/). We download the all exported malware binary code form OWL database and check all of them to filter out the damage ones. The most of them is PE32 executable (DLL) (GUI) Intel 80386 for MS Windows notified by. 政治大. Linux file command, so that is why we chose the guest OS type with Win7SP1x86. 立. profile.. ‧ 國. 學. Get the malware information list from OWL database. step2.. Chose the target malware which behavior data is not collected. step3.. Launch a new VM from prepared VM image. step4.. Download and send target malware binary code to the VM launched just. step5.. Have the VM run the target binary code without hindrance. step6.. Invoke the VISO collecting main function with all necessary arguments. step7.. Delete the VM after collecting is over and saved. step8.. Go to step2 until end of malware information list. ‧. step1.. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Table 2. Class list used as label Table 2 is the rough flow path of experiment control script program. After all data was collected over, we will test them in the way that we describe in the fore part of this paper, like training phase and testing phase and see the result of test to judge whether we need to adjust our methodology. 35 .

(42) We automate all flow path mentioned above. Table 3 is class list table derived from all malwares in this experiment. In this experiment, we has eight kinds of labels and we collect kinds of behavior data from malwares. The name of eight classes are respectively “Unknown”, “W32”, “TR”, “WORM”, “BDS”, “EXP”, “Normal” and “Malware”. The normal processes include SSH daemon, HTTP daemon, JVM, FTP, any process which might be used on normal cloud VM and processes are eligible and build-in Windows NT processes. Unknown means the malware cannot be recognized by anti-virus. “Malware” label is a special label we suppose it is not normal but we. 政治大 The features we collected from VM include 1. Hidden process 2. DLL library 立. cannot clearly judge which malicious class it is belonging to.. distribution used by a VM 3. DLL library list used by a process. Following we will. ‧ 國. 學. show the testing results for feature2 and feature3. Feature1 is specification in. ‧. methodology chapter, we put psxview raw file into checking routine which is shown. y. Nat. in Table 1. We can clearly know whether there is any hidden process or not. Because. er. io. sit. the performance of feature1 is not really efficient, we don’t have enough time to test out all malwares like ldrmodules. In a small scale psxview test, about 20 malwares,. al. n. v i n the average executing time of C psxview dump is 8 hours h e n g c h i U and 22 minutes. So, if we. want to test psxview on same malwares like ldrmodules, we need at least 303 days. We only show several examples of the small scale psxview test to see that the real malware in the world usually hides what processes in runtime. Table 4 is the example of malware list. Each of them has its class. For example, '52cae62ada8b02adae128e35686cbc16' is malware id, and ‘5’ is its label number scanned by anti-virus. In real dataset, the number of malwares we downloaded from OWL malware database is 868. Normal is a special sample. The number of total samples is 869 including normal. The number of attributes extracted from all samples is 678. The number of classes concluded from all samples is 7. One class is for special 36 .

(43) case of unsure malicious class. The number of total classes is 8. For each class of “Unknown”, “W32”, “TR”, “WORM”, “BDS”, “EXP”, and “Normal”, the number of samples is respectively 445, 280, 58, 81, 3, 1, and 1. For each class of “Unknown”, “W32”, “TR”, “WORM”, “BDS”, “EXP”, and “Normal”, the number of vectors for rule2 is respectively 2589, 1644, 332, 477, 18, 6, and 24. For each class of “Unknown”, “W32”, “TR”, “WORM”, “BDS”, “EXP”, and “Normal”, the number of vectors for rule3 is respectively 88314, 55479, 11322, 16283, 606, 195, and 1467. { 0: Unknown 1: W32 2: TR 3: WORM 4: BDS 5: EXP 6: Normal 7: Malware. 立. 政治大. ‧. ‧ 國. 學 sit. al. n. . Table 3. Class list used as label. er. io. {. y. Nat. }. i n C '52cae62ada8b02adae128e35686cbc16' h e n g c h i: 5,U. v. 'ebb80a9e5a7871dfef8c4e478fd9c514' : 1, 'ed1304342c5794cb0a55840253793b45' : 2, 'ee09957625ee0d3a75e344d40368ba3e' : 4, 'f027f8eb6c5189edd8841c2aa4251b7c' : 2, 'fc87cb9c5d9b347a6c9fdef990a12e8d' : 3, 'fca087c49d12015973f757a1b525e239' : 1, }. Table 4. Example of malware list with label. 37 .

(44) Figure 13.. Example of hidden process detection (1). 立. 政治大. ‧. ‧ 國. 學 er. io. sit. y. Nat. (2) a l Example of hidden process detection v i n Ch U engchi. Figure 15.. Example of hidden process detection (3). n. Figure 14.. 38 .

(45) 政治大 For feature1 hidden process, 立 see Figure 13-16 which are several random malware Figure 16.. Example of hidden process detection (4). ‧ 國. 學. samples selected from small scale psxview test, UI0Detect.exe, sppsvc.exe, svchost.exe, and slui.exe are the suspect hidden processes after hidden process routine. ‧. check. They are all common Windows-NT DLL library or system program but they. sit. y. Nat. are infected by malware or regarded as target of disguise.. al. er. io. For feature2 and feature3, we have two kinds of test repectively for each one of. v. n. them, test1 and test2. Feature2 is our rule2 and feature3 is our rule3. In test 1, we. Ch. engchi. i n U. filter out unknown malwares. The number of remaining samples is 424. For each class, we split samples into two piles. For each class of “W32”, “TR”, “WORM”, and “BDS”, they will respectively have 140, 29, 41, and 2 for training and have 140, 29, 40, and 1 for testing. Because both the number of “EXP” class and “Normal” class are 1. Against this two cases, we split vectors of this two samples into two piles, one for training and the other for testing. Finally we have 214 samples for training and 212 samples for testing. Test 2, we use all 424 samples without unknown malwares for training and use 445 unknown malware samples for testing. The results of test1 and. 39 .

(46) test2 are going to showing in following section. The details of training and testing are specified in methodology. In test1, there are 5 kinds of class in testing data, “W32”, “TR”, “WORM”, “BDS”, “EXP” and “Normal”. The expected predicting labels include “W32”, “TR”, “WORM”, “BDS”, “EXP”, “Normal” and “Malware”. Table 5 is the result of rule2. For each class, we show the number of summary for each expected predicting label. If predicting label is the same as the class which means label scanned by anti-virus, it represents that we predict it right. If not, it is. 政治大 number of right is respectively 138, 1, 0, 0, 0, and 12. For each class of “W32”, “TR”, 立 wrong. For each class of “W32”, “TR”, “WORM”, “BDS”, “EXP” and “Normal”, the. “WORM”, “BDS”, “EXP” and “Normal”, the number of wrong is respectively 2, 28,. ‧ 國. 學. 40, 1, 1, and 0. The GHSOM predicting model is shown in Appendix1.. ‧. Table 6 is the result of rule3. For each class, we show the number of summary. y. Nat. for each expected predicting label. If predicting label is the same as the class which. er. io. sit. means label scanned by anti-virus, it represents that we predict it right. If not, it is wrong. For each class of “W32”, “TR”, “WORM”, “BDS”, “EXP” and “Normal”, the. al. n. v i n C h0, 0, 0, 0, and 731.UFor each class of “W32”, number of right is respectively 140, engchi. “TR”, “WORM”, “BDS”, “EXP” and “Normal”, the number of wrong is respectively 0, 29, 40, 1, 1, and 3. The GHSOM predicting model is shown in Appendix2.. 40 .

(47) Rule2. W32. TR. WORM. BDS. EXP. Normal. Mal.. W32. 138. 1. 2. 0. 0. 0. 2. TR. 27. 1. 0. 0. 0. 0. 2. WORM. 39. 0. 0. 0. 0. 0. 1. BDS. 1. 0. 0. 0. 0. 0. 0. EXP. 1. 0. 0. 0. 0. 0. 0. Normal. 0. 0. 0. 0. 0. 12. 0. Normal. Mal.. Table 5. Result of test1 for rule2 Rule3. W32. TR. W32. 140. 0. TR. 29. WORM. 立. BDS 政治大EXP. WORM. 0. 0. 0. 0. 0. 0. 0. 0. 0. 40. 0. 0. 0. 0. 0. 0. BDS. 1. 0. 0. 0. 0. EXP. 1. 0. 0. 0. 0. Normal. 1. 1. 0. 0. 0. ‧. io. 0. 0. 0. 0. 731. 1. y. sit. Nat. er. ‧ 國. 0. 學. 0. n. a Table iv l C 6. Result of test1 fornrule3 hengchi U. In test2, there are only one kind of class in testing data, “Unknown”. The expected predicting labels include “W32”, “TR”, “WORM”, “BDS”, “EXP”, “Normal” and “Malware”. Table 7 is the result of rule2. We show the number of summary for each expected predicting label. The GHSOM predicting model is shown in Appendix3. Table 8 is the result of rule3. We show the number of summary for each expected predicting label. The GHSOM predicting model is shown in Appendix4.. 41 .

(48) Rule2. W32. TR. WORM. BDS. EXP. Normal. Mal.. Unknown. 445. 0. 0. 0. 0. 0. 0. Table 7. Result of test2 for rule2. Rule3. W32. TR. WORM. BDS. EXP. Normal. Mal.. Unknown. 445. 0. 0. 0. 0. 0. 0. Table 8. Result of test2 for rule3. Discussion:. 立. 政治大. ‧ 國. 學. In test1 for rule2, the “W32” class of malwares can be found easily. The total number of “W32” samples is 140. There is 138 samples found and 2 samples not. ‧. found. Some of them may be judged to the multiple classes simultaneously, because. sit. y. Nat. they have the same number of votes in the result of voting mechanism. Observing the. io. er. wrong ones in detail, they doesn’t be judged to “Normal” class. For other malicious classes except “W32”, all of them not have good accuracy, but they are all judged to. al. n. v i n “W32” class or other malicious C classes. This phenomenon means their behaviors are hen gchi U similar to “W32”. If we want to completely distinguish all malicious behaviors. We may need to add more number of samples of all malicious classes except “W32”. About “Normal” class, which is a class we don’t want to have wrong predicting, we discover that they are all judged correctly. In test1 for rule3, they have the similar phenomenon to rule2. “W32” class is judged pretty correct. The other malicious cannot be judged correctly but they doesn’t be judged to “Normal” class. In “Normal” class, the result is also really accurate, but several normal processes are judged to malicious classes. Comparing to rule2, rule3. 42 .

(49) has more accuracy on malware detection, but not a little mistake on “Normal” class judgement. Go deep into the result of clustering, we check the mark withn the each cluster and decisive attributes output from GHSOM training result. We found that “BDS” and “EXP” doesn’t have any pure cluster to be marked in such a little number of samples. Then, we also found that other malicious samples using the same DLL library reflected from decisive attributes output from GHSOM training result. Like \Windows\System32\t2embed.dll, \Windows\System32\sti.dll,. 政治大 test1 for rule2. \Windows\System32\ieapfltr.dll, \Windows\explorer.exe, 立. \Windows\System32\umpo.dll, \Windows\System32\wbem\WMIADAP.exe in the. \Windows\System32\bitsigd.dll, \Windows\System32\en-US\sndvolsso.dll.mui,. ‧ 國. 學. \Windows\System32\en-US\imageres.dll.mui, \Windows\System32\sxs.dll. ‧. \Windows\System32\nsisvc.dll, \Windows\System32\en-US\rasdlg.dll.mui,. y. Nat. \Windows\System32\ieframe.dll, \Windows\System32\en-US\wisptis.exe.mui. er. io. sit. \cygwin\bin\cygiconv-2.dll, \Windows\System32\bitsperf.dll,. \Windows\System32\fvecerts.dll, \Windows\System32\wbemcomn.dll,. n. al. Ch. \Windows\System32\ncrypt.dll for rule3.. engchi. i n U. v. In the test2, we using all samples which can be scanned by anti-virus to GHSOM training and using all samples which cannot be scanned by anti-virus to predicting, so call “Unknown” class malware samples. We found that all of “Unknown” class are judged to “W32” class. We also check the mark withn the each cluster and decisive attributes output from GHSOM training result in test2. We found that “BDS” and “EXP” doesn’t have any pure cluster to be marked in such a little number of samples, too. Following is the same DLL library used by malware samples reflected from decisive attributes output from GHSOM training result. Like \Windows\System32\t2embed.dll, \Windows\System32\sti.dll, 43 .

(50) \Windows\System32\umpo.dll, \Windows\System32\wbem\WMIADAP.exe in the rule2. It is completely same as ones in test1. \Windows\System32\bitsigd.dll, \Windows\System32\wscui.cpl, \Windows\System32\sxs.dll, \Windows\System32\en-US\rasdlg.dll.mui, \Windows\System32\ieframe.dll \cygwin\bin\cygiconv-2.dll, \Windows\System32\bitsperf.dll, \Windows\System32\msdmo.dll, \Windows\System32\fvecerts.dll, \Windows\System32\ncrypt.dll for rule3. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 44 . i n U. v.