Motivation and main contributions of this work

Based on the observations that hard disk access and WLAN NIC traffic are bursty and self-similar, we propose a DPM algorithm that handles the lengths of idle time in ON and OFF periods separately that adjusts the timeout value more precisely and decides which inactive state the SP should be switched to, respectively, so as to achieve better power saving.

That is, the propose AH-DPM algorithm can fully utilizes the self-similarity characteristic of disk access (or WLAN access) to predict request arrival patterns and adaptively adjust the timeout value and select an inactive state. As a result, the proposed AH-DPM algorithm can provide a better tradeoff between average power consumption and average response time (average packet transmission delay) for hard disks (WLAN NICs), and thus is very feasible to mobile devices for extending their battery lifetime.

The remaining of this dissertation is organized as follows. Chapter 2 reviews related work of DPM algorithms. Chapter 3 depicts our design approach and shows the flowcharts and pseudo codes of the proposed DPM algorithm. Experimental setup, experimental results, and discussion are presented in Chapter 4. We give concluding remarks in Chapter 5.

Chapter 2

Related Work

Four categories of DPM policies have been proposed: timeout, predictive, stochastic, and machine learning policies [1]. Fig. 2-1 classifies existing DPM algorithms based on these four categories. In the following, we briefly introduce the characteristics of each category.

2.1 Timeout-based algorithms

Timeout-based algorithms can be divided into two classes: static timeout (STO) and adaptive timeout (ATO) [1]. The STO scheme turns off a component after a fixed period of

idle time. Because the timeout value is fixed, the STO scheme, shown as a dotted block in Fig.

2-1, is not a DPM algorithm. In this scheme, the user has to decide the best timeout period manually. The ATO scheme is more efficient because it changes the timeout value according to the latest idle time. There are several adaptive timeout algorithms. In [7], it adjusts the

Fig. 2-1. The classification of existing DPM algorithms.

timeout value by using the ratio of the length of the previous idle period divided by the wakeup delay. If the ratio is small, the timeout value is increased. If the ratio is large, the timeout value is decreased.

In [8], the authors proposed an OS power management technique called PowerNap that modifies the timing mechanism of the OS to achieve better power saving [8]. They observed that when the OS is idle, the widely used periodic timing (PT) scheme, which a timer will issue interrupts to the OS periodically, will cause unnecessary power dissipation [8]. The solution to this phenomenon is to eliminate the periodic timer tick whenever the OS is idle. A scheme called Work Dependent Timing (WDT) was proposed, which will switch the system to a low power state when there is no task to execute [8]. The WDT will determine the nearest timeout value, write it into the hardware timer, and switch the system state to a low power consumption state. When the timer expires, the hardware will issue a hardware interrupt to wake up the whole system [8]. Generally speaking, timeout schemes have two main advantages. They are general and the throughput of serving requests can be guaranteed simply by increasing the timeout value [9]. They also have two main disadvantages. They waste a lot of energy because of waiting the timeout value to expire and they always result in performance penalty when components wakeup [9].

2.2 Predictive-based algorithms

The predictive-based algorithms can be classified into two categories: predictive shutdown and predictive wakeup [9]. They were proposed to deal with the disadvantages of

timeout schemes [9]. The predictive shutdown scheme predicts the length of an idle period when the PM detects that a component is going to enter the idle state. If the PM assesses that the length of the idle period will be longer than the break-even time [9], the component will be switched to a lower power consumption state immediately to eliminate the unnecessary waste of energy usually caused by timeout schemes. The predictive wakeup scheme predicts the expiration of an idle period. If the PM predicts that the idle period of a component is going to be ended in a short time, the component will be switched to an active state to avoid an incoming request waiting the component to switch from an inactive state to an active state.

Representative predictive-based algorithms are reviewed as follows. Srivastava et al. [10]

proposed two approaches which belong to predictive shutdown [9] for a component. The first approach uses regression analysis to arrive at a model for predicting the length of idle periods [10]. The second approach is based on the observation of the phenomenon that a long duration of an active state is followed by a short duration of an idle state with a very high probability, and the probability of an idle state followed by a short duration of an active state is fairly evenly distributed [10]. In this case, the component will be shut down when the PM observes that an idle period is about to begin. These two approaches strongly rely on offline analysis of the component behavior; thus they are not adaptive.

Huang et al. [11] addressed three predictive methods: predictive shutdown using exponential average, correction of prediction misses, and pre-wakeup. In the predictive shutdown, the formula of exponential average is as follows:

𝐼

= 𝛼 ∙ 𝐼

+ (1 − 𝛼) ∙ 𝐼

where In+1 is the new predicted value, In is the last predicted value, in is the latest idle period, and α is a constant attenuation factor in the range between 0 to 1 [11]. In the correction of prediction misses, there are two sub-issues: under-prediction and over-prediction.

Under-prediction happens when a long idle period occurs after a series of short, uniformly distributed idle periods. Over-prediction happens when a short idle period occurs after a series of long, uniformly distributed idle periods. The former situation is resolved by setting a watchdog to periodically monitor the current idle period. The latter situation is resolved by adding a saturation condition to the original algorithm. The pre-wakeup scheme is used to deal with the performance penalty due to the wakeup delay. This can be accomplished by predicting the occurrence of the next wakeup signal [11].

Chung et al. [12] proposed a DPM method using an adaptive learning tree [12]. Using the tree, the PM can accurately predict the most appropriate low-power sleep state at the start of an idle period [12]. They also proposed an enhanced scheme which adopts a fixed timeout filter in order to eliminate the unnecessary shutdown when a very short idle period occurred [12]. Ramanathan et al. [13] used the previous request inter-arrival time τ to predict the next idle period [13]. If τ is greater than the shutdown threshold k, the component will be shut down immediately because the algorithm assumes that the next idle time will be greater than k time units [13]. If τ is less than k, it keeps the component idle for a period of k unless a new request arrives [13]. This approach is similar to the algorithm proposed by Huang et al. [11]

and it is a combination of STO and predictive shutdown algorithms [13]. Nevertheless, the above predictive-based algorithms suffer from the prediction accuracy of the length of idle time.

2.3 Stochastic-based algorithms

The stochastic-based algorithm proposed by Benini et al. [14] uses stochastic processes to model the behaviors of the Service Requester (SR), Service Queue (SQ), and Service Provider (SP). The overall system architecture for the DPM is shown in Fig. 2-2 [14]. The SR will send a request to the SP. When the SP is busy and if requests keep coming, incoming requests will be stored in the SQ. If the SQ is full, incoming requests will be discarded. The Power Manager (PM) observes the status of the SR, SQ, and SP, and it decides which command, such as shutdown, wakeup, or state-transition, will be sent to the SP. The probability models used to describe the behaviors of the SR, SQ, and SP are the main issue of the stochastic-based schemes. The more precise the probability models that describe the SR, SQ, and SP, the more accurate the state transition decisions made by the PM, thus saving more energy of the system.

The following are representative stochastic-based algorithms. In [14], it models request arrival and state transition as stationary discrete-time Markov processes. The assumptions of this approach are as follows [14]:

1. The arrival of service requests can be modeled by an m-memory Markov chain. The m-memory Markov model has 2^m states, one for each possible sequence of consecutive bits.

2. The state transition delays in the SP can be modeled as random variables with a geometric distribution.

3. Model parameters and cost functions are available and accurately measured before

Fig. 2-2. Overall system architecture for the DPM [14].

However, the above constraints are not likely to occur in real life. First, in most cases, the arrivals of user requests, the state transition delays, and even the service time, are non-stationary [15]. Second, the characteristic of discrete time causes additional cost for the PM because the PM must periodically wakeup to do computation. The first drawback can be dealt with using a non-stationary stochastic process, and the second drawback can be handled by changing the time from discrete to continuous [16][18].

For the first drawback, Chung et al. [16] proposed an approach using a non-stationary stochastic process to model the arrival distribution of user requests. They proposed a mechanism called sliding window to keep historical data. The sliding window is limited in length and hence recent historical data are kept in order to reflect recent user request behavior.

Because the distribution of user requests is non-stationary, the decision table of the SP must be recalculated in every period. To overcome this drawback, the authors used table lookup and interpolation to calculate the decision table to avoid the recalculation. Ren et al. [17] modified the approach in [16] and introduced a multi-mode model using a Markov-modulated stochastic process to model the non-stationary arrival process of service requests [17]. The advantage of these two approaches is that the request arrival distribution of the SR can be adapted to any distribution. But the disadvantages are an enormous amount of memory usage and computation power required. If we apply these two approaches to several components in a mobile device, we have to derive a request arrival distribution for each component in advance and it will be time consuming and inconvenient.

As to the second drawback, Qiu et al. [18] proposed a continuous-time Markov decision process to decrease the computation of the PM. In this approach, the decision is made on an event arrival, such as a user request arrival, the SP starting to serve a user request, and the SP finishing a user request. Rong et al. [19] extended the work in [18] to model a battery-powered portable system by introducing and incorporating a new continuous-time Markovian decision process model of the battery source [19]. There are some disadvantages in this approach. First, the computation complexity both in time and space are high because of the characteristic of continuous-time based policy optimization. Second, the experiment was based on a continuous-time Markov process, that means that the inter-arrival time of user requests, the switching time of the SP, and the state transition of the SQ are exponential distributed, which are not quite realistic in the real world. In [20], the authors proposed a Variation Aware DPM, which is based on the Partially Observable Markov Decision Process, to solve the power management problem on chip multiprocessor (CMP) architectures. The algorithm has a belief state estimator, which estimates the next state of the system, and a policy maker, which assigns optimal actions to the CMP [20]. The time complexity of the

algorithm is O(nm), where n is the number of states and m is the number of actions [20]. The time complexity is high compared with timeout-based and predictive-based algorithms.

2.4 Machine learning algorithms

Several researchers applied machine learning to learn the request arrival patterns of the SR. Dhiman et al. [21] and Prabha et al. [22] proposed expert based machine learning

algorithms. An expert based machine learning algorithm selects the best DPM policy from a set of DPM policies. These policies are called experts. Each expert has a weight value which indicates the expert's priority and is adjustable by the machine learning algorithm. The weight value will be adjusted in every idle period and the expert with the highest weight value in the current idle period will be used to control an embedded system during the next idle period.

However, the performance of an expert based machine learning algorithm is highly dependent on chosen experts. In [23], the authors proposed a reinforcement learning based algorithm.

The algorithm is based on the Q-learning algorithm, which was originally designed to find a policy for a Markov Decision Process, to learn the arrival request patterns of the SP [23]. The authors modified the Q-learning algorithm to solve the DPM problem and speed up the algorithm's convergence time by updating more than one Q values simultaneously. The time complexity of the modified Q-learning algorithm is O(|SP| × |A|), and the space complexity is O(|SP| × |SQ| × |SR| × |A|), where |SP|, |SQ|, and |SR| are the number of states of the SP, SQ,

and SR, respectively, and |A| is the number of commands. In [24], the authors combine fuzzy decision processes and reinforcement learning. The authors use the fuzzy decision processes to model the behavior of the wireless sensor nodes, and adapt the modified Q-learning algorithm to decide a sequence of duty cycles in order to improve energy efficiency. Note that the time and space complexities of the modified Q-learning algorithm are higher than those of the proposed AH-DPM algorithm, which are constant in both time and space complexity, which will be explained later.

In summary, a qualitative comparison of major DPM algorithms, including the proposed AH-DPM, is shown in Table 2-1. The time complexity and space complexity of the proposed AH-DPM algorithm are both O(1) since our algorithm is a hybrid of adaptive timeout and predictive algorithms. Let p be the number of states in the SP, q be the queue length in the SQ, and r be the number of states in the SR, and let x = p × q × r [14]. The time complexity of the stochastic algorithm proposed by Benini et al. [14] is O((xa)³L), where L is the bit length of input data [25] (usually 32 bits in modern computer systems) and a is the number of commands that the PM can issue to the SP [14]. The space complexity of the stochastic algorithm is O(xa) [25]. The stochastic algorithm needs offline calculation because the system state transition matrix must be calculated in advance. Because the machine learning algorithm uses experts, which are a set of DPM algorithms, the time and space complexities, offline calculation, and manual configuration are dependent on the selected DPM algorithms. In addition, STO, ATO, and predictive algorithms need to configure some initial values manually.

Table 2-1. A qualitative comparison of representative DPM algorithms.

Static

timeout Adaptive timeout

(Douglis et al. [7]) Predictive

(Huang et al. [11]) Stochastic

(Benini et al. [14]) Machine learning

(Dhiman et al. [21]) AH-DPM (proposed) Timeout value

adjustment Static Adaptive Adaptive None Adaptive Adaptive

Time complexity O(1) O(1) O(1) O((xa)³L) Depend on experts O(1)

arrival patterns with self-similarity. They are more suitable for stationary request arrival patterns.

Chapter 3

Proposed AH-DPM Algorithm

3.1 The design of the proposed algorithm

To obtain better power saving of the SP, the proposed AH-DPM algorithm handles and adapts the average idle time in the bursty (ON) and non-bursty (OFF) periods in the request arrival pattern separately. We derive the average SP idle time in the ON and OFF periods separately using exponential average. All parameters used in the proposed algorithm are defined in Table 3-1.

The proposed AH-DPM algorithm returns two values: the next state, SSP_next, that the SP

Table 3-1. The parameters used in the proposed AH-DPM algorithm.

Parameters Description

Ton_avg Average idle time in the ON period Toff_avg Average idle time in the OFF period Tidle Most recent idle time

Ttimeout Current timeout value Ttimeout_last Last timeout value TBE Break-even time SSP_current Current state of the SP SSP_next Next state of the SP

Pexpect Expected period of the request arrival pattern, either ON_period or OFF_period

the next state. There are three main ideas in the proposed AH-DPM algorithm: (1) keeping track of the average idle time in the ON period, Ton_avg, and in the OFF period, Toff_avg, separately, (2) using the average idle time in the ON period to adjust the timeout value more precisely and using the average idle time in the OFF period to decide which inactive state the SP should be switched to, and (3) comparing the most recent idle time, Tidle, with the break-even time, TBE, to determine whether the expected period of the request arrival pattern, Pexpect, is ON_period or OFF_period. Three cases will be monitored by the proposed AH-DPM algorithm:

1. When a request arrives, based on Pexpect and the comparison between Tidle and TBE, there are three situations required to be taken care of:

a. If Pexpect equals to ON_period, calculate new Ton_avg using new Tidle. Then, Ttimeout_last is updated to the maximum value among Ttimeout_last, Tidle, and Ton_avg, with upper bound of TBE.

b. If Pexpect equals to OFF_period, and Tidle is smaller than TBE, this situation is called prediction miss because in the OFF period, Tidle should be larger than TBE. In this case, the calculations described in situation 1a will be performed since Pexpect should be in the ON period.

c. If Pexpect equals to OFF_period, and Tidle is larger than or equal to TBE, this situation is called prediction hit. Toff_avg will be calculated using new Tidle.

After handling either of the above three situations, Pexpect will be set to ON_period, SSP_next

will be set to idle state, and SSP_next along with timeout value of zero will be returned. The reason for returning zero as the timeout value is to wake up the SP immediately whenever there is a request arrived and the SP is in an inactive state.

2. When the SP has finished serving a request and the SQ is empty, the SP may be inactivated since there is no request to be served. The next state of the SP, SSP_next, will be chosen by the algorithm InactiveState. Then, SSP_next and Ttimeout will be returned.

3. When Ttimeout expires, the SP will be switched to the state indicated by SSP_next, and Pexpect

will be set to OFF_period to indicate that the request arrival pattern is now in the OFF period. In the meantime, if the SP has deeper inactive states, which implies lower power consumption and higher state transition overhead, the next deeper inactive state, relative to the current inactive state, of the SP will be assigned to SSP_next, and Ttimeout will be set to TBE. The SP will be switched to a deeper inactive state specified in SSP_next when Ttimeout

expires. This timeout iteration mechanism repeats until the SP is switched to the deepest inactive state if there is no incoming request arrival for a long time.

In summary, the benefits of the proposed AH-DPM algorithm are that our algorithm adapts to bursty request arrival patterns with self-similarity and a service provider (SP, i.e., hard disk or WLAN NIC, in this dissertation) with multiple inactive states by the following

In summary, the benefits of the proposed AH-DPM algorithm are that our algorithm adapts to bursty request arrival patterns with self-similarity and a service provider (SP, i.e., hard disk or WLAN NIC, in this dissertation) with multiple inactive states by the following