Chapter 2 Related Work
2.4 Machine learning algorithms
Several researchers applied machine learning to learn the request arrival patterns of the SR. Dhiman et al. [21] and Prabha et al. [22] proposed expert based machine learning
algorithms. An expert based machine learning algorithm selects the best DPM policy from a set of DPM policies. These policies are called experts. Each expert has a weight value which indicates the expert's priority and is adjustable by the machine learning algorithm. The weight value will be adjusted in every idle period and the expert with the highest weight value in the current idle period will be used to control an embedded system during the next idle period.
However, the performance of an expert based machine learning algorithm is highly dependent on chosen experts. In [23], the authors proposed a reinforcement learning based algorithm.
The algorithm is based on the Q-learning algorithm, which was originally designed to find a policy for a Markov Decision Process, to learn the arrival request patterns of the SP [23]. The authors modified the Q-learning algorithm to solve the DPM problem and speed up the algorithm's convergence time by updating more than one Q values simultaneously. The time complexity of the modified Q-learning algorithm is O(|SP| × |A|), and the space complexity is O(|SP| × |SQ| × |SR| × |A|), where |SP|, |SQ|, and |SR| are the number of states of the SP, SQ,
and SR, respectively, and |A| is the number of commands. In [24], the authors combine fuzzy decision processes and reinforcement learning. The authors use the fuzzy decision processes to model the behavior of the wireless sensor nodes, and adapt the modified Q-learning algorithm to decide a sequence of duty cycles in order to improve energy efficiency. Note that the time and space complexities of the modified Q-learning algorithm are higher than those of the proposed AH-DPM algorithm, which are constant in both time and space complexity, which will be explained later.
In summary, a qualitative comparison of major DPM algorithms, including the proposed AH-DPM, is shown in Table 2-1. The time complexity and space complexity of the proposed AH-DPM algorithm are both O(1) since our algorithm is a hybrid of adaptive timeout and predictive algorithms. Let p be the number of states in the SP, q be the queue length in the SQ, and r be the number of states in the SR, and let x = p × q × r [14]. The time complexity of the stochastic algorithm proposed by Benini et al. [14] is O((xa)3L), where L is the bit length of input data [25] (usually 32 bits in modern computer systems) and a is the number of commands that the PM can issue to the SP [14]. The space complexity of the stochastic algorithm is O(xa) [25]. The stochastic algorithm needs offline calculation because the system state transition matrix must be calculated in advance. Because the machine learning algorithm uses experts, which are a set of DPM algorithms, the time and space complexities, offline calculation, and manual configuration are dependent on the selected DPM algorithms. In addition, STO, ATO, and predictive algorithms need to configure some initial values manually.
Table 2-1. A qualitative comparison of representative DPM algorithms.
Static
timeout Adaptive timeout
(Douglis et al. [7]) Predictive
(Huang et al. [11]) Stochastic
(Benini et al. [14]) Machine learning
(Dhiman et al. [21]) AH-DPM (proposed) Timeout value
adjustment Static Adaptive Adaptive None Adaptive Adaptive
Time complexity O(1) O(1) O(1) O((xa)3L) Depend on experts O(1)
arrival patterns with self-similarity. They are more suitable for stationary request arrival patterns.
Chapter 3
Proposed AH-DPM Algorithm
3.1 The design of the proposed algorithm
To obtain better power saving of the SP, the proposed AH-DPM algorithm handles and adapts the average idle time in the bursty (ON) and non-bursty (OFF) periods in the request arrival pattern separately. We derive the average SP idle time in the ON and OFF periods separately using exponential average. All parameters used in the proposed algorithm are defined in Table 3-1.
The proposed AH-DPM algorithm returns two values: the next state, SSP_next, that the SP
Table 3-1. The parameters used in the proposed AH-DPM algorithm.
Parameters Description
Ton_avg Average idle time in the ON period Toff_avg Average idle time in the OFF period Tidle Most recent idle time
Ttimeout Current timeout value Ttimeout_last Last timeout value TBE Break-even time SSP_current Current state of the SP SSP_next Next state of the SP
Pexpect Expected period of the request arrival pattern, either ON_period or OFF_period
the next state. There are three main ideas in the proposed AH-DPM algorithm: (1) keeping track of the average idle time in the ON period, Ton_avg, and in the OFF period, Toff_avg, separately, (2) using the average idle time in the ON period to adjust the timeout value more precisely and using the average idle time in the OFF period to decide which inactive state the SP should be switched to, and (3) comparing the most recent idle time, Tidle, with the break-even time, TBE, to determine whether the expected period of the request arrival pattern, Pexpect, is ON_period or OFF_period. Three cases will be monitored by the proposed AH-DPM algorithm:
1. When a request arrives, based on Pexpect and the comparison between Tidle and TBE, there are three situations required to be taken care of:
a. If Pexpect equals to ON_period, calculate new Ton_avg using new Tidle. Then, Ttimeout_last is updated to the maximum value among Ttimeout_last, Tidle, and Ton_avg, with upper bound of TBE.
b. If Pexpect equals to OFF_period, and Tidle is smaller than TBE, this situation is called prediction miss because in the OFF period, Tidle should be larger than TBE. In this case, the calculations described in situation 1a will be performed since Pexpect should be in the ON period.
c. If Pexpect equals to OFF_period, and Tidle is larger than or equal to TBE, this situation is called prediction hit. Toff_avg will be calculated using new Tidle.
After handling either of the above three situations, Pexpect will be set to ON_period, SSP_next
will be set to idle state, and SSP_next along with timeout value of zero will be returned. The reason for returning zero as the timeout value is to wake up the SP immediately whenever there is a request arrived and the SP is in an inactive state.
2. When the SP has finished serving a request and the SQ is empty, the SP may be inactivated since there is no request to be served. The next state of the SP, SSP_next, will be chosen by the algorithm InactiveState. Then, SSP_next and Ttimeout will be returned.
3. When Ttimeout expires, the SP will be switched to the state indicated by SSP_next, and Pexpect
will be set to OFF_period to indicate that the request arrival pattern is now in the OFF period. In the meantime, if the SP has deeper inactive states, which implies lower power consumption and higher state transition overhead, the next deeper inactive state, relative to the current inactive state, of the SP will be assigned to SSP_next, and Ttimeout will be set to TBE. The SP will be switched to a deeper inactive state specified in SSP_next when Ttimeout
expires. This timeout iteration mechanism repeats until the SP is switched to the deepest inactive state if there is no incoming request arrival for a long time.
In summary, the benefits of the proposed AH-DPM algorithm are that our algorithm adapts to bursty request arrival patterns with self-similarity and a service provider (SP, i.e., hard disk or WLAN NIC, in this dissertation) with multiple inactive states by the following two steps to achieve better power saving. First, it derives the average idle time of the SP in the bursty (ON) period and non-bursty (OFF) period separately. Second, it uses the average idle time in the ON period to adjust the timeout value more precisely and uses the average idle
time in the OFF period to decide which inactive state the SP should be switched to. In other words, the calculation of the timeout value described in situation 1a above is to set the timeout value long enough to prevent an unexpected SP state transition to an inactive state and to keep the timeout value short enough to decrease the length of the busy wait period during the ON period.
3.2 The flowchart and pseudo codes of the proposed algorithm
Fig. 3-1 and Fig. 3-3 show the flowchart and the pseudo codes of the proposed AH-DPM algorithm, respectively. If a request arrives and the SP is in the ON period, Ton_avg and Ttimeout_last are updated accordingly, as shown in line 7 to line 8 of Fig. 3-3. On the other hand, if a request arrives and the SP is in the OFF period, two circumstances must be considered.
First, if Tidle is larger than or equal to TBE, the algorithm has made a correct prediction that the SP is actually in the OFF period. This circumstance is called a prediction hit and Toff_avg will be updated accordingly, as shown in line 14 of Fig. 3-3. Second, if Tidle is smaller than TBE, this is called a prediction miss since in the OFF period Tidle must be larger than TBE. In this case, Ton_avg and Ttimeout_last are updated instead of Toff_avg, and Pexpect is set to ON_period, as shown in line 18 to line 19 of Fig. 3-3. Once a request is served and the SQ is empty, the SP becomes idle. Then, the SP will be switched to an inactive state SSP_next with timeout value Ttimeout. SSP_next will be assigned using algorithm InactiveState, as shown in Fig. 3-2. Algorithm InactiveState is used to determine which inactive state the SP will enter. It calculates TBE of
each inactive state, and save it into BE_List. BE_List is an array that stores TBE of each inactive state. BE_List is then sorted in the descending order. Toff_avg will be compared with each TBE. If Toff_avg is greater than or equal to a certain TBE, the inactive state corresponding to the previous TBE in BE_List will be returned to algorithm AH-DPM, as shown from line 6 to line 14 in Fig. 3-2. If Ttimeout expires, the SP will be switched to state SSP_next and Pexpect will be set to OFF_period. In the meantime, the timeout iteration mechanism mentioned in section 3.1 starts. A new inactive state and TBE will be set to SSP_next and Ttimeout accordingly, as shown from lines 34 to 35 in Fig. 3-3.
Fig. 3-1. The flowchart of the proposed AH-DPM algorithm.
01 algorithm InactiveState(SSP_current, Toff_avg) 02 {
03 calculate each TBE which relates to each inactive state and SSP_current, and save them into BE_List 04 sort BE_List in descending order
05 Sret = SSP_current
06 for(index = 0; index < the size of BE_List; index++) 07 {
08 if(Toff_avg >= BE_List[index])
09 {
10 Sret = the inactive state corresponding to BE_LIST[index]
11 break;
12 }
13 } 14 return Sret
15 }
Fig. 3-2. Inactive state decision algorithm.
01 algorithm AH-DPM(SSP_current, Tidle)
26 else if(SP has finished serving a request and SQ is empty) 27 {
28 SSP_next = InactiveState(SSP_current, Toff_avg) 29 Ttimeout = Ttimeout_last
37 Return SSP_next and Ttimeout
38 }
Fig. 3-3. The proposed AH-DPM algorithm.
Chapter 4
Experimental Results and Discussion
4.1 Experimental setup
We compare our algorithm (AH-DPM) with the oracle algorithm (Oracle), the static timeout algorithm (STO) with timeout value of 30 seconds [2], the adaptive timeout algorithm (ATO) of Douglis et al. [7] with parameters (αm, βm, ρ) = (0.5, 1.5, 0.1) [2] and initial timeout value of 30 seconds [2], the predictive algorithm (Predictive) of Huang et al. [11] with parameters α = 0.3 and c = 2 [2], the stochastic (Stochastic) algorithm of Benini et al [14], and the machine learning (ML) algorithm of Dhiman et al [21]. Remind that the oracle algorithm is a theoretically optimal algorithm because it knows the arrival time of all requests; therefore, the algorithm can determine exactly when and to which state the SP should be switched for power saving. The Stochastic algorithm does not need to set any initial value, but the optimal decision policies must be calculated in advance. The ML algorithm uses STO, ATO, Predictive, and Stochastic algorithms as its experts. The parameters of each expert used in the ML algorithm are same as those listed above. Since the oracle algorithm is aware of all the requests issued from the beginning to the end, it can come out with the most power efficient results. The AlwaysOn algorithm is defined as “always keeping the SP in the active state,” and represents the worst case of average power consumption and the best case of average response
time (average packet transmission delay) in hard disk (WLAN NIC) experiments.
In the hard disk experiments, we used a typical hard disk specification of Hitachi Travelstar 5K100 as an example SP specification, as shown in Table 4-1 and Table 4-2 [26].
The state transition diagram of Hitachi Travelstar 5K100 is shown in Fig. 4-1. Note that state transitions between Performance Idle, Active Idle, and Low Power Idle are hardware controlled; thus cannot be controlled by the user or operating system. We use dotted lines to represent the hardware-controlled state transitions. The hard disk request traces were collected for a week by monitoring the ATA commands sent and received by libATA drivers under Fedora 12 [27], kernel version 2.6.32.11 [28]. The trace characteristics of hard disk requests for each day of the week are listed in Table 4-3. We compare the average power consumption and average response time for the hard disk among these DPM algorithms. The average power consumption is measured in Watt, and it is defined as total energy consumed divided by total elapsed time. The average response time is measured in millisecond and it is defined as the average elapsed time between a request arrived and the request having been served, which includes the queuing delay and service time of a request.
As to the WLAN NIC experiments, we used the specification of the Intel PRO/Wireless 3945ABG (802.11g) card, which is shown in Table 4-4 [29]. However, the state transition time and state transition power of the Intel PRO/Wireless 3945ABG card are not available.
We used the state transition time listed in [30]. The state transition power from sleep to idle is set to twice of the power consumption in idle state, and the state transition power from idle to
sleep is set to the power consumption in sleep state [31]. The state transition characteristics of the WLAN NIC are listed in Table 4-5, and the associated state transition diagram is illustrated in Fig. 4-2. In addition, the real traces of the WLAN NIC were captured using Wireshark [32], version 1.2.6, under Fedora 12 [27]. The trace characteristics of WLAN NIC packets for each day of the week are listed in Table 4-6. The average power consumption and average packet transmission delay are metrics of the WLAN NIC for performance evaluation
among these DPM algorithms. The definition of the average power consumption is identical to that for the hard disk and the average packet transmission delay is measured in millisecond, and it is defined as the queuing delay plus the packet transmission time.
Finally, we also evaluate the prediction miss rate and the inactivation ratio, which will be defined later. The prediction miss rate can reflect the average power consumption and the inactivation ratio can reflect the average response time in the hard disk and the average transmission delay in the WLAN NIC. There are two cases of prediction miss: false positive prediction and false negative prediction. A positive prediction is that the PM decides to
inactivate the SP, and a negative prediction is that the PM decides NOT to inactivate the SP.
Therefore, the false positive prediction occurs in a situation that the PM inactivates the SP (i.e., switching the SP to a lower power consumption state), but the length of the inactive period is shorter than the break-even time. That is, the SP should not be inactivated, but it is inactivated. The false negative prediction occurs in a situation that the PM keeps the SP in active state, but the length of the idle period is longer than the break-even time. That is, the SP should be inactivated, but it is not inactivated. Both cases will result in extra power
consumption penalty. According to the definitions of false positive prediction and false negative prediction, the definition of the prediction miss rate Rprediction_miss is as follows:
𝑅
𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑛_𝑚𝑝𝑚𝑚=
𝑁𝑓𝑓_𝑚𝑚𝑚𝑚𝑁 +𝑁𝑓𝑓_𝑚𝑚𝑚𝑚𝑓𝑝𝑝𝑝𝑚𝑝𝑝𝑚𝑝𝑓 (2)
where Nfp_miss is the number of false positive prediction, Nfn_miss is the number of false negative prediction, and Nprediction is the total number of predictions made by the PM. The definition of the inactivation ratio Rinactivate is defined as follows:
𝑅
𝑝𝑛𝑖𝑝𝑝𝑝𝑖𝑖𝑝𝑝=
𝑁𝑁𝑚𝑓𝑖𝑝𝑝𝑚𝑖𝑖𝑝𝑝𝑓𝑝𝑝𝑝𝑚𝑝𝑝𝑚𝑝𝑓 (3) where Ninactivate is the number of switching the SP from the active state to an inactive state (a lower power consumption state). When the SP is in an inactive state and a request arrives, the SP must switch to the active state in order to serve the request. Since existing DPM algorithms, except the oracle algorithm, did not actually implement the pre-wakeup mechanism, an incoming request will be queued in the SQ waiting for the SP to be switched from an inactive state to the active state. Therefore, it results in queuing delay. Since each SP’s inactivation will lengthen the queuing delay of the next incoming request, the higher the inactivation ratio is, the longer the average response time of the hard disk and the average packet transmission delay of the WLAN NIC are.
Table 4-1. State transition time and power consumption specifications of Hitachi Travelstar 5K100 hard disk [26].
From To Time Active Idle Performance Idle 0.02 2.0
Performance Idle Standby 0.35 2.0
Performance Idle Sleep 0.35 2.0
Table 4-2. Power consumption specification of Hitachi Travelstar 5K100 hard disk [26].
State Power consumption
Fig. 4-1. The state transition diagram of Hitachi Travelstar 5K100 hard disk.
Table 4-3. Trace characteristics of hard disk requests.
Day of Week Number
Requests of
𝑇
𝑅𝑅𝜎
𝑇𝑅𝑅Sunday 148175 0.5826582132 28.066745610 Monday 273503 0.3158121481 7.576979263 Tuesday 90081 0.9588597164 13.238787090 Wednesday 190393 0.4537697663 7.577420349 Thursday 532259 0.1623176762 3.813604001 Friday 128420 0.6725806881 10.876428390 Saturday 46820 1.8449750100 18.263384880 A week 1409651 0.4289868597 11.847667333
𝑇𝑅𝑅: Average request inter-arrival time (sec)
𝜎𝑇𝑅𝑅: Standard deviation of the request inter-arrival time
Table 4-4. Power consumption specification of Intel PRO/Wireless 3945ABG WLAN NIC [29].
State Power consumption
Table 4-5. State transition time and power consumption of the WLAN NIC [29][30].
From To Time
(μsec) Power consumption (Watt)
Sleep Idle 250 0.3
Idle Sleep 80 0.03
Fig. 4-2. The state transition diagram of the WLAN NIC.
4.2 Experimental results
Fig. 4-3 shows the experimental results of the average power consumption of the hard disk for each DPM algorithm on each day of a week. In Fig. 4-3, we found that the proposed AH-DPM algorithm is always better than the other algorithms except the oracle algorithm under the request trace of each day in a week. By observing Table 4-3 and Fig. 4-3, we found that average power consumption is proportional to number of requests, except the case on Tuesday. The average power consumption on Tuesday is still higher in spite of a lower number of requests. This is because of a higher inactivation ratio on Tuesday, as shown in Fig.
4-5. Since a higher inactivation ratio implies a higher number of state transitions, the power consumption increases due to frequent state transitions. As to the average response time, Fig.
4-4 shows that the proposed AH-DPM algorithm is larger than the ATO, AlwaysOn, ML, Oracle, and STO algorithms, and is shorter than the Predictive algorithm. By observing Fig.
4-4 and Fig. 4-5, we found that average response time is proportional to inactivation ratio.
This is because that if the SP is in an inactivated state and there is an incoming request, the SP
Table 4-6. Trace characteristics of WLAN NIC traffic.
Day of Week Number
Requests of
𝑇
𝑃𝑅𝜎
𝑇𝑃𝑅Sunday 68812 1.2524797900 13.271983260 Monday 24697 3.0321168550 43.423190660 Tuesday 48308 1.6348761390 218.246429500 Wednesday 115253 0.7495722815 10.870117770 Thursday 183500 0.4701362994 8.127996827 Friday 13013 6.5880108190 73.593723700 Saturday 8164 10.4842361800 93.448664130 A week 461747 1.2648289645 73.980365017
𝑇𝑃𝑅: Average packet inter-arrival time (sec)
𝜎𝑇𝑃𝑅: Standard deviation of the packet inter-arrival time
must be switched to the active state to serve the request. Since no existing DPM algorithms
must be switched to the active state to serve the request. Since no existing DPM algorithms