As mentioned above, the communication environment of local IVC is much simpler than a real network since we do not need to care about the problems such as packet loss, packet reordering, etc. Therefore, we can use a much simpler protocol instead of the traditional communication protocols.
We designed a fast communication mechanism between domains on top of the same VMM called fast-IVC, which is illustrated in Figure 3. As shown in the figure, fast-IVC builds up a tunnel between the communicating domains and automatically switches the traditional network socket channel to the memory-based tunnel when the source and destination ends of the connection are on the same VMM. The switch is transparent to user applications and the performance of the local IVC is improved due to the skip of the network protocol processing.
Figure 3 Comparison of the Original (left) Protocol Processing and fast-IVC (right)
Figure 4 illustrates the architecture of fast-IVC, which consists of four components:
tunnel manager, protocol monitor, tunnel protocol and event manager. We will describe these components in the following sections.
Figure 4 The Architecture of Fast-IVC
3.2.1 Tunnel Manager
The tunnel manager is a component that provides functions to create or release a tunnel in the guest OS. A tunnel is actually a shared memory block, which will be created if the source and destination of a socket connection are on the same VMM. After a tunnel is created, the tunnel should be mapped into both sender’s and receiver’s address spaces. Tunnel creation will fail if one or both of the communication ends can not map the tunnel into their address spaces.
The memory block of a tunnel can be allocated from the memory pool of the VMM, the sender, or the receiver. We do not allocate tunnel memory from the VMM since its address space limitation. Moreover, allocating memory from sender or receiver makes no differences so we allocate the memory from the sender’s memory pool.
3.2.2 Protocol Monitor
The protocol monitor detects whether or not a tunnel should be created by checking the destination IP addresses, and asks the tunnel manager to create a tunnel when necessary. As usual, a user creates a connection by using TCP sockets. If the source and destination are on the same VMM, the protocol manager will detect the fact and switch the communication protocol to the tunnel protocol. Otherwise, the communication goes through the traditional network protocol.
We intercept network protocol operations to perform the destination address checking. In TCP, the destination can be known after a socket executes accept() or connect(). In UDP, the destination is known when the socket executes sendmsg()/rcvmsg(). Therefore, we intercept the send/receive operations. Specifically, the protocol monitor determines whether to change to the tunnel protocol or to use the TCP/IP protocol stack when a send or receive operation is issued at the first time.
3.2.3 Tunnel Protocol
The tunnel protocol is a simple communication protocol used for local IVC. After a tunnel is mapped into both the sender’s and the receiver’s address spaces, the data will be transferred by the tunnel protocol. The sender pushes data into the tunnel, and the receiver is notified when the current memory chunk2 is full or no more data needs to sent. When the receiver is notified, the memory chunk should be locked from the sender until the receiver reads all the data of the chunk. After all the data is read, the chunk can be used again by the sender.
2 A tunnel consists of multiple memory chunks which are called channels, and we will describe the details in Section 3.4.5
3.2.4 Event Manager
In order to support sender-receiver cooperation during tunnel creation/release and data transmission, an event notification mechanism is needed. However, due to isolation maintained by the VMM, a domain can not send messages or events directly to the other domains. Therefore, we design an event manager to allow a guest OS to send events to VMM or another guest OS. Moreover, since the destination of an event is a socket, which is not aware by the VMM, we divide the event manager into two parts: Domain Event Manager (DEM) and Socket Event Manager (SEM). The former resides in VMM and is responsible for dispatching events to the corresponding domains, or getting events from domains. The latter resides in each guest OS and is responsible for dispatching events to the corresponding sockets and providing an event interface to the other components (e.g., protocol monitor) in the guest OS.
Table 1 shows the events that are provided by the event manager. When a tunnel is created in the sender side, the sender will notify the receiver by EVENT_CREATE_ TUNNEL, and receiver will map the tunnel into its address space so that the data can be transferred by the tunnel. The receiver notifies sender by EVENT_REJECT_TUNNEL if the tunnel mapping fails and the sender switches back to TCP/IP protocol when it gets this event. When a sender wants to close a tunnel, it sends the EVENT_SEND_CLOSE_TUNNEL event. Similarly, the EVENT_ RECV_CLOSE_TUNNEL event is sent when a receiver wants to close a tunnel.
Finally, EVENT_SEND_DATA is used to notify the receiver to get data from the tunnel, and EVENT_ RECV_GET_DATA is used to notify the sender that the channel is free so the sender can use the channel again.
Name Source/Dest. Description
EVENT_CREATE_TUNNEL Sender/Receiver Sender creates a tunnel, and receiver can use the tunnel.
EVENT_SEND_CLOSE_TUNNEL Sender/Receiver Sender closes a tunnel, and receivers can close it.
EVENT_REJECT_TUNNEL Receiver/Sender Receiver can not map the tunnel into its address space.
EVENT_ RECV_CLOSE_TUNNEL Sender/Receiver Receiver closes the tunnel, and sender can close it.
EVENT_SEND_DATA Sender/Receiver Sender has put the data into the tunnel, and receiver can get data from the tunnel.
EVENT_ RECV_GET_DATA Receiver/Sender The channel (i.e., memory chunk) can be used again by the sender.
Table 1 Events Supported by the Event Manager