3. The iPalace Video Channel Service Design and Implementation
3.2 Implementation of the iPalace Video Channel
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
18
service infrastructure, the peer-to-peer networking, and the multi-task and distributed system architecture. The service delivery system should always smoothly deliver fresh and attracting video contents and well-defined NPM experiences.
The customer criteria for embracing the iPalace Video Channel service include the reputation of NPM brand, the dependability and education of contents, the high-quality video, the ease-of-use and aesthetic UI, the stable and smooth Web service accessible worldwide and free-of-charge. In short, being an online media of NPM, the iPalace Video Channel service needs to not only meet with the high-quality brand image of NPM but also be able to cope with the huge peak-demand from worldwide.
3.2 Implementation of the iPalace Video Channel
Technically, the iPalace Video Channel is a marketing platform in Internet that provides video-stream multicasting, video-on-demand, and information retrieval services, and so on. We build up the iPalace Video Channel on the cloud to provide the video service.
Ciuffoletti (2010) describes a basic elastic web service infrastructure on the cloud as shown in Figure 5. Physical machines in the infrastructure management layer such as server arrays, storage and routers are provided by cloud computing service providers. The iPalace Video Channel is built as the user application layer on top of the infrastructure management layer that is provided by Chunghwa Telecom’s Hicloud. (Hicloud 2011) The service is implemented in the iPalace Video Channel by using high-level language technologies and APIs. General users won’t connect to the infrastructure management layer directly but can access services from the iPalace Video Channel.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
19
Figure 5: infrastructure and user application views of a basic elastic web service infrastructure (Ciuffoletti 2010)
As shown in Figure 6, the structure of the iPalace Video Channel that has a multi-task and distributed system architecture. We build up servers on VMs separately according to the tasks they are responsible for. Tasks are split and distributed to each VM so that we can not only implement each VM efficiently but also avoid centralizing workload on a particular VM. Since each VM takes a specific task, this makes distributing requests among a cluster of identical VMs much easily.
The structure of the iPalace Video Channel consists of 4-tier layers. Each VM has 2 virtual cores (2GHZ), 4G memories, and runs the CentOS 5.5, 64bit version. All the VMs have installed the Apache server package. Layer-1 contains the Main Reception Desk that is responsible for receiving requests from users who would like to access the iPalace Video Channel service. The main reception disk hosts the index webpage and bypasses the requests to the next layer for accessing videos. Layer-2 is composed by six distribution centers (DCs) that contain streaming servers and (local-cached) videos. All the DCs have installed the flash media server (the streaming server) and are responsible for broadcasting videos to users. DCs are classified into the following the video categories: main, AD (i.e., advertisement), (the introduction of) NPM, mission (i.e., the introduction of the iPalace Video Channel), collection (of historic relics) and How-to-use (i.e., the guideline of the iPalace Video Channel). The DC Main is responsible for distributing requests to different DCs by
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
20
sending their index and needed information back to users. After receiving the index, all the following user requests will be redirected to the indexed server (that corresponds to the service). The index is selected according to the category of videos chosen by users. Later we extend each DC to an elastic cluster of VMs and cope with the high peak-demands by balancing the load.
Figure 6: The system structure of the iPalace Video Channel
The backstage operations are supported in the next two layers. Layer-3 contains the Update Center, the Search Center and the Media Center. The Media Center is responsible for connecting Video DBMS when searching and updating videos. The Update Center checks with the Media Center to update new uploaded or deleted videos. The Search Center delivers the search request from the previous layer to the Video DBMS through the Media Center and returns the result to the users. The Video DBMS and Database belong to Layer-4, within which the NPM employees can upload new videos from the portal and back up videos that have been uploaded in a local NPM DB.
In Figure 7, we demonstrate the flow of the process for user requesting for the iPalace Video service.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
21
(1) User sends requests for requesting the iPalace Video service to the Main Reception Desk.
(2) The Main Reception Desk wraps the user requests with related information and directs to the DC Main.
(3) The DC Main returns the index webpage to the user
(4) After receiving the response, all the following user requests will be redirected to the DC according to the category of videos user chosen and establish connection.
Then, the DC provides the asked service, e.g., the NPM collection series as shown in Figure 7.
(5) On the other hand, during the idle time of the iPalace Video Channel service (e.g., users do not ask for other videos), the DC AD will broadcast non-stop advertisement automatically to the users.
Figure 7: Process of coming on in the iPalace Video Channel for the first time
We develop the task-oriented VMs separately according to the service and video category. Despite offering a clean task to each VM, the design may not be able to disperse the workload on servers in balance. Particularly, the VMs for DC AD and DC Collection may take most workload since the services appear to be requested frequently by users and systems. In fact, one aim of the iPalace Video Channel service is providing a non-stop multi-casting video service stored in the DC AD and the DC
‧
Collection. The potentially high peak overload of the VMs on the DC AD and the DC Collection could lead to an unsatisfactory experience of the user. In the next section, we will describe how to build an elastic cluster of VMs to tackle the potentially high peak overload of the VMs on the DC AD and the DC Collection with the aim of providing a consistent high-quality service.
In the following, we describe how to dynamically allocate VMs to achieve the load balance with satisfied service quality. The main idea is to employ an elastic cluster of VMs to take and share the workload. A cluster consists of a set of identical VMs3. Unlike all the requests are taken by a single machine, requests are distributed in balance to each machine in the cluster. In addition, the size of the cluster is dynamically increasing so that the maximal load of all VMs within the cluster is low enough to guarantee an acceptable service quality.
Initially, the cluster has only one VM which is prepared to handle requests. We set up an auxiliary monitor VM – the Monitor watching the workload of all the VMs within the cluster. The Monitor periodically detects the number of arrivals4 served by each running VM and reports the VM that has the minimal number of arrivals. If its number of arrivals is under an acceptable bound, it will be selected as the corresponding VM and the next coming arrival(s) will be dispatched to this VM. On the other hand, if it exceeds the bound (so as other VMs), we will increase the size of the cluster by initiating a new VM5 to join the cluster and dispatch the next arrival to this VM. The new invocation is needed since, when the minimal one equals the threshold, it implies that all the other VMs also have equaled number of arrivals as the threshold. The new VM has the same ability to handle user’s demands as the original VM. By the mechanism, we can achieve load balancing among an elastic cluster.
Except the Monitor, we set up another auxiliary VM – the Dispatcher which is responsible for dispatching the arrivals to the corresponding VM. The Dispatcher will retrieve the corresponding VM identified by the Monitor periodically and store it in an internal variable. We define DURATION as the interval between the Dispatcher retrieves the corresponding VM. DURATION determines the frequency of updating which VM takes the coming arrivals. The smaller the DURATION, the more often the retrieve of the corresponding VM is, so more balanced the load is.
3 The identical VM has the same ability to handle the demanding video service as the others.
4 Here an arrival corresponds to an initial service request from one individual User.
5 The new VM has the same ability to handle the arrival’s demands as the other running VMs. This can be achieved by initiating a new VM with the same image or simply by invoking an idle VM that was built with the same image.
‧
requests come within the period of DURATION will be assigned to the corresponding VM. That is, if requests increase suddenly within the DURATION, it is possible the corresponding VM gets overloaded. Thus, for the purpose of preventing violating the requirement of acceptable service quality, if the number of arrivals that the corresponding VM handles with counted by the Dispatcher equals the BOUND –the upper bound of the number of arrivals that each VM can handle with, the Dispatcher will initiate a new VM, add it to the cluster, and update the corresponding VM stated in the internal XML file of the Monitor to select it as the corresponding VM.When the cluster becomes larger, there may be more unemployed VMs which have no live arrivals in the cluster. To prevent wasting recourses, the unemployed VMs will be released to an idle VM pool which collects VMs that have been initiated but are not handling any arrivals. While the Monitor or the Dispatcher needs to initiate a new VM, it is randomly selected from this idle VM pool. Since all VMs are identical (in functions and status), getting a new VM from the idle VM pool can prevent the time of initiating a new VM from scratch. Furthermore, we keep the number of VMs in the idle VM pool in an acceptable range. When the number of VMs is out of the range, extra VMs will be released (or initiated) automatically.
The Monitor also gathers other status information of each running VM minimal number of arrivals and update the internal XML file (for machines to access the current minimal load VM) in the Monitor. The pseudo code of the algorithms is given as below: 1. init load[]= init_load (hostname[]);
2. monitor_VMs (hostname[], load[], log);
‧
3. minhost = select_min(hostname[], load[]);
4. if(load[minhost] > BOUND) 5. new = create_VM();
6. minhost = new;
7. update(hostname[], new);
8. return minhost;
The control variable BOUND specifies the upper bound of the number of arrivals that each VM can handle with. It also determines when to new a VM to increase the size of the cluster, which has an influence on the service quality.
The array hostname is used to record the names of VMs in the cluster. It is set with one VM by default. While a new VM is created (monitor(), line5-7), the hostname is updated with the new VM. When calling the procedure monitor_VMs(), we connect all the VMs listed in the hostname and collect their status, such as the average of CPU load over 1 minute from /proc/loadavg and the number of connection established (saved in load[]). We also collect some other data related to the CPU and network status of each VM in the log using three unix commands – uptime, iostat and netstat, as well as requests per second (collected from the Apache’s server status).
Uptime provides: the current (logged) time, how long the system has been running, how many users are currently logged on, and the system load average for the past 1, 5, and 15 minutes. Iostat is used for monitoring system input/output device loading by observing active devices in relation to their average transfer rates. It also reports the CPU and the device utilization. Netstat provides information about the linux networking subsystem, e.g., a list of open sockets. The information (log) is used for evaluating and adjusting our load balancing strategy.
The presented strategy is selecting the VM with the minimal number of arrivals as the corresponding VM to take the coming requests before the next monitoring process. If the value of the minimal one equals the threshold (BOUND), we will initiate a new VM (by selecting one from the idle VM pool) to share the workload. By calling the procedure set_xml(), we update the XML file with the IP of the corresponding VM. The coming requests will be directly redirected to the VM specified in the XML file without any new monitoring process. Except in the monitor procedure, the new VM will be initiated and selected as the corresponding VM and the XML file stated in the Monitor will be updated when number of arrivals that the corresponding VM handles with equals BOUND. In the end of the monitor procedure, the unemployed VMs will be released to the idle VM pool. In this way, we prevent run-time overload and resource wasted.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
25