Suboptimal Selection Policies - Sleep Window Selection Policy

5.4 Sleep Window Selection Policy

5.4.2 Suboptimal Selection Policies

According to the aforementioned performance evaluations, two selection policies, including sleep ratio-based (SR) policy and energy cost-based (EC) policy, are proposed in the SSWC approach. For the SR policy, the immediate reward of each state/action pair is deﬁned as the sleep ratio mentioned above; while the energy cost is utilized as the immediate reward in the EC policy. The reward set ℛ of the POMDP is calculated via the Reward Assignment Algorithm as illustrated in Algorithm 2, where Φ indicates the type of policy, i.e, Φ = 𝑆𝑅 and Φ = 𝐸𝐶 denote the SR policy and EC policy respectively. The algorithm assigns the reward for each (𝑠𝑖, 𝑎_𝑘), ∀𝑠𝑖 ∈ 𝒮 and ∀𝑎_𝑘 ∈ 𝒜, with the considerations of tolerable delay 𝛿 and/or queue size 𝑄, which are explained as follows:

1. Tolerable delay consideration (𝜙 = ˜𝑑). With the consideration of delay constraint, the expected packet delay derived from (5.16) is utilized to exclude the infeasible action.

As can be seen in lines 4 to 10 of Algorithm 2, if the expected delay of (𝑠_𝑖, 𝑎_𝑘) satisﬁes the tolerable delay 𝛿, the immediate reward 𝑟(𝑠 , 𝑎 ) is assigned according to either

(5.11) or (5.15). Otherwise, a pre-deﬁned value 𝑅_𝑚𝑖𝑛 or 𝐸_𝑚𝑎𝑥 is assigned to represent the cost for SR or EC policy, respectively.

2. Queue size consideration (𝜙 = ˜𝑞). On the other hand, the reward assigned with the consideration of queue size can be found in lines 13 to 19 of Algorithm 2. The average number of arriving packets obtained from (5.13) is considered as a criterion in this case, while all the other processes of reward assignment are the same as that mentioned in the case of tolerable delay consideration.

3. Both of the delay and queue considerations (𝜙 = ˜𝑏). In the case of considering both the tolerable delay and queue size, all the aforementioned processes are conducted to assign proper rewards for all state/action pairs, i.e., lines 3 to 20 in Algorithm 2.

Algorithm 2: Reward Assignment Algorithm

Input: 𝒮, 𝒜, tolerable delay 𝛿, queue size 𝑄, policy type Φ, and consideration type 𝜙 Output: set of immediate rewards ℛ(𝒮, 𝒜)

foreach 𝑠_𝑖 ∈ 𝒮 do

The optimal policy of the SWS problem is unavailable owing to the reason of uncertain traﬃc states. However, thanks to the belief states of POMDP model, the uncertain traﬃc states can be estimated. Moreover, the reward of an action made in a given traﬃc state is provided by the immediate reward set ℛ. Based on these two types of information, the sub-optimal policies can be acquired via adopting a 𝑇 -step value function in the SSWC approach.

For the SR policy, the decision of window selection made at decision epoch 𝑑_𝑡 ∈ 𝒟 can be acquired as

𝐷_𝑑^𝑆𝑅_𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) = arg 𝑉_𝑑^𝑆𝑅_𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) = arg max

𝑎^𝑑𝑡_𝑘 ∈𝒜

[Γ_𝑑_𝑡(𝑏(𝑠^𝑑_𝑖^𝑡))]. (5.19)

where

Γ_𝑑_𝑡(𝑏(𝑠^𝑑_𝑖^𝑡)) = ∑

𝑠^𝑑𝑡_𝑖 ∈𝒮

𝑏(𝑠^𝑑_𝑖^𝑡)𝑟(𝑠^𝑑_𝑖^𝑡, 𝑎^𝑑_𝑘^𝑡) + 𝛾^𝑑^𝑡+1 ∑

𝑧_𝑗^𝑑𝑡+1∈𝒵

𝜎(ℬ(𝑑_𝑡), 𝑎^𝑑_𝑘^𝑡, 𝑧^𝑑_𝑗^𝑡+1)𝑉_𝑑_𝑡+1(𝑏(𝑠^𝑑_𝑖^𝑡+1)), (5.20)

with 𝑟(𝑠^𝑑_𝑖^𝑡, 𝑎^𝑑_𝑘^𝑡) selected as either 𝑅(𝑠_𝑖, 𝑎_𝑘) or 𝑅_𝑚𝑖𝑛 according to Algorithm 2 for the SR scheme. The function 𝑉_𝑑^𝑆𝑅

𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) in (5.19) is deﬁned as the 𝑇 -step value function for the SR policy at a decision epoch 𝑑_𝑡which starts at 𝑑_𝑡, and there are 𝑇 − 1 decision steps remaining.

The ﬁrst item of (5.20) denotes the reward for a belief state 𝑏(𝑠^𝑑_𝑖^𝑡) ∈ ℬ(𝑑_𝑡). The expected reward of the future belief state 𝑏(𝑠^𝑑_𝑖^𝑡+1) ∈ ℬ(𝑑_𝑡+1) is represented in the second item, where 𝜎(ℬ(𝑑_𝑡), 𝑎^𝑑_𝑘^𝑡, 𝑧^𝑑_𝑗^𝑡+1) can be obtained from (5.10). The parameter 𝛾^𝑑^𝑡+1 is denoted as a discount factor of the 𝑑_𝑡+1-step for convergence control of the value function. In other wards, the value function 𝑉_𝑑^𝑆𝑅

𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) intends to ﬁnd the maximum sleep ratio of an action according to the currently estimated traﬃc state and the expected sleep ratio of future actions made in the successive states. On the other hand, for the EC policy, the decision made at 𝑑_𝑡 and its corresponding 𝑇 -step value function can be obtained as

𝐷^𝐸𝐶_𝑑_𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) = arg 𝑉_𝑑^𝐸𝐶_𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) = arg min

𝑎^𝑑𝑡_𝑘 ∈𝒜

[Γ_𝑑_𝑡(𝑏(𝑠^𝑑_𝑖^𝑡))]. (5.21)

where the function Γ_𝑑_𝑡(𝑏(𝑠^𝑑_𝑖^𝑡)) is deﬁned in (5.20) with 𝑟(𝑠^𝑑_𝑖^𝑡, 𝑎^𝑑_𝑘^𝑡) selected as either 𝐸(𝑠_𝑖, 𝑎_𝑘) or 𝐸𝑚𝑎𝑥 based on Algorithm 2 for the EC scheme. Unlike that for the SR policy, minimum reward is utilized in the value function 𝑉_𝑑^𝐸𝐶

𝑡 (𝑏(𝑠^𝑑_𝑖^𝑡)) for the EC case in order to minimize the energy consumption of the MS.

5.5 Performance Evaluation

In this section, simulations are conducted to evaluate the performance of the proposed SSWC approach in comparison with the conventional PSC I and the evolutional PSC I. A single BS/MS pair with non-real-time downlink traﬃc is considered as the simulation scenario. The simulation is implemented via MATLAB event-driven simulator. All the required procedures and functions for sleep mode operation have been implemented in the simulator. The param-eters adopted within the simulation are listed in Table 5.1, where the energy consumption parameters are acquired from the industrial manufactured mobile WiMAX chip [49]. Each simulation run is executed with 13 minutes, in which the MS stays in the normal mode during the ﬁrst 3 minutes and starts the sleep mode operation at the 4th minute. Each obtained re-sults is average from 100 simulation runs. For the proposed SSWC approach, the traﬃc trace of the ﬁrst 3 minutes within the simulation are utilized to infer the number of traﬃc states as described in Section 5.3.1. Moreover, the 3-step value function is considered in the simulation, wherein the discount factor 𝛾 for the future 𝑖th step is selected as (0.5)^𝑖 for 𝑖 ∈ {1, 2}. It is represented that the rewards calculated later in time will have less value than an equivalent reward received closer to the present.

Fig. 5.5 illustrates the sleep mode operations of the conventional PSC I (denoted as IEEE 802.16e), the evolutional PSC I (denoted as IEEE 802.16m), the SSWC approach with SR policy (denoted as SR), and the SSWC approach with EC policy (denoted as SSWC-EC) under traﬃc state of 𝜆 = 0.1. The top subplot shows the arriving packets in frames [37400, 37700] of the simulation; while the corresponding operations of all the schemes are depicted in the remainder subplots. For each approach, the state of 1 denotes that the MS is awake in the normal mode or during the listening window of the sleep mode; while the MS

Table 5.1: Simulation parameters for SSWC approach

Parameter Value

Frame duration 5 ms

Idle period (IEEE 802.16e) 4 frame

Default Listening Window length (IEEE 802.16e/m) 1 frame Initial-Sleep Window length (IEEE 802.16e/m) 1 frame Final-Sleep Window length (IEEE 802.16e/m) 256 frame

Sleep Window length (SSWC) [1, 256] frame

Energy consumption of busy frame 280 mW

Energy consumption of idle frame 120 mW

Energy consumption of sleep frame 10 mW

Energy consumption of state switch 1 mW

Mean packet arrival rate (𝜆) 0.1, 0.2, 0.3, 0.4, 0.5 packets/frame

Mean service rate (𝜇) 3 packets/frame

374000 37450 37500 37550 37600 37650 37700

1 2 3

Traffic State (λ = 0.1)

Number of Packet

37400 37450 37500 37550 37600 37650 37700

0 1

IEEE 802.16e

MS State

37400 37450 37500 37550 37600 37650 37700

0 1

IEEE 802.16m

MS State

37400 37450 37500 37550 37600 37650 37700

0 1

SSWC−SR

MS State

37400 37450 37500 37550 37600 37650 37700

Figure 5.5: An exempliﬁed sleep mode operation among IEEE 802.16e, IEEE 802.16m, SSWC-SR, and SSWC-EC approaches under traﬃc state of 𝜆 = 0.1.

0.1 0.2 0.3 0.4 0.5

Figure 5.6: Performance comparison of average packet delay between the two proposed SSWC approaches (i.e., EC and SR) with diﬀerent delay constraints (𝛿).

stays in the sleep state is represented by the stat of 0. It can be observed that the MS spends lots of time in the awake state while the binary-exponential power-saving mechanism of IEEE 802.16e or IEEE 802.16m is adopted. With the proposed SSWC approach, on the other hand, the MS almost remains in the sleep state and only wakes up to receive the incoming traﬃc since each sleep period is determined according to the present traﬃc state estimated via the POMDP. The performance comparison among these schemes are discussed in the following subsections.

5.5.1 Eﬀect of Delay Constraints

The eﬀect of delay constraints is discussed via the performance comparisons as shown in Figs. 5.6 to 5.8. The performance of average packet delay between the two proposed SSWC approaches are illustrated in Fig. 5.6, where EC and SR denote the approaches of SSWC-EC and SSWC-SR respectively. Due to the proposed reward assignment and accurate state estimate, the performance of all the schemes satisfy the respective delay constraints over various mean packet arrival rates. As for the cases that do not consider any delay constraints, the average packet delay is bounded by the maximum length of sleep window, i.e., 640 ms

for the window of length 256 frames in the simulation. The SSWC-EC approach has relative higher packet delay than the SSWC-SR scheme resulted from the diﬀerent deﬁnition of reward functions. For the reward that is deﬁned as the sleep ratio (i.e., the SSWC-SR approach), the length of sleep window that results in the highest reward value will be selected; while the window with lowest reward value is chosen in the SSWC-EC approach in order to minimize the energy cost. These two policies result in diﬀerent sleep mode operations even if they can possess same ratio of listening frames to sleep frames. For example, considering the duration of frames [37543, 37660] as shown in Fig. 5.5, there are 3 listening frames and 115 sleep frames that appeared in both schemes. In the SSWC-EC approach, one listening frame occurs at frame number 37600 while two happen at numbers 37659 and 37660. The SSWC-SR scheme, on the other hand, has all its three listening frames occur at frame numbers 37558, 37559, and 37660. Therefore, it can be observed that both schemes result in the same sleep ratio of 115/118 = 0.975 during this considered time interval. It is intuitive to see that the SSWC-EC approach intends to minimize the energy consumption such as to accumulate all the three listening frames in order to reduce the switching cost 𝜀_𝑆 between listening and sleep windows. Nevertheless, the target of the SSWC-SR approach is to minimize the sleep ratio which results in the occurrence of ﬁrst listening frame at frame 37600. The sleep ratio becomes 57/58 = 0.983 with the length of sleep window as 57 frames and one frame for listening window, which is higher than 0.975 by adopting the SSWC-EC scheme. Consequently, as shown in Fig. 5.6, the performance with lower packet delay can be obtained in the various cases by exploiting the SSWC-SR approach.

Fig. 5.7 depicts the performance comparison of average energy consumption between the two proposed SSWC approaches. It is expected that the average energy consumption increases as the value of mean packet arrival rate is augmented in all the diﬀerent cases. With lower tolerable delay, more energy consumption is observed in both of the proposed SSWC approaches. This can be attributed to the reason that the length of sleep window is bounded by the delay constraint. In the sleep mode, the MS is provided with a series of alternated sleep windows and listening windows. For the purpose of satisfying tight tolerable delay, the shorter

0.1 0.2 0.3 0.4 0.5

Figure 5.7: Performance comparison of average energy consumption between the two proposed SSWC approaches (i.e., EC and SR) with diﬀerent delay constraints (𝛿).

sleep windows will be selected which incurs more number of listening windows during the simulation time, and consequently increases the energy consumption. Comparing the SSWC-EC and SSWC-SR approaches, almost the same level of energy consumption is obtained since similar ratio of listening frames to sleep frames exists in both schemes. However, the shorter sleep windows selected in the SSWC-SR scheme incurs more number of state transition, i.e., from sleep state to awake state and vice versa. Therefore, slightly higher energy consumption is shown in the SSWC-SR approach compared to the SSWC-EC scheme.

The performance comparisons among IEEE 802.16e, IEEE 802.16m, and the proposed SSWC approach over various packet arrival rates is shown in Fig. 5.8. Since similar perfor-mances are exhibited in the proposed SSWC-EC and SSWC-SR schemes with diﬀerent packet delay consideration, the SSWC-EC approach is utilized on behalf of the proposed SSWC scheme to compare with the convnetional binary-exponential power-saving mechanisms. For the IEEE 802.16e and IEEE 802.16m schemes, it is expected that the average packet delay will be decreased as the packet arrival rate is augmented; meanwhile, the energy consumption is raised. The IEEE 802.16m has slightly higher delay but signiﬁcant decrement of energy consumption compared to the IEEE 802.16e, which can be attributed to the elimination of

0.1 0.2 0.3 0.4 0.5

Figure 5.8: Performance comparison among IEEE 802.16e, IEEE 802.16m, and SSWC-EC approaches with diﬀerent delay constraints (𝛿).

frequent state transitions. As mentioned in Section 5.2.2, in the IEEE 802.16m, the MS ter-minates the listening window and returns to the sleep state immediately after the completion of data transportation. Without the existence of idle periods in the IEEE 802.16m, energy conservation can be preserved. However, the opportunities to promptly receive the incoming traﬃc are dissolved and consequently incurs packet transmission delay. Comparing between the proposed SSWC-EC approach and the IEEE 802.16m scheme, relative better performance can be observed in the SSWC-EC approach. It can be seen that the energy consumption of the SSWC-EC (𝛿 = 20 ms) approach is similar to that of the IEEE 802.16m at 𝜆 = 0.1, but the SSWC-EC (𝛿 = 20 ms) approach can provide signiﬁcantly lower packet delay. It is owing to the reason that the SSWC approach determines the length of each sleep window according to the traﬃc state and delay constraint. Due to the same reason, as similar packet delays existed in both schemes, the SSWC approach will result in lower energy consumption compared to the IEEE 802.16m scheme, e.g., the case with the SSWC-EC (𝛿 = 20 ms) scheme at 𝜆 = 0.2.

0.1 0.2 0.3 0.4 0.5 0

50 100 150 200 250 300

Packet Arrival Rate (packet/frame)

Average Packet Delay (ms)

EC (Q = 2 pkt) EC (Q = 6 pkt) EC (Q = 10 pkt) EC (Q = 14 pkt) EC (Q = 18 pkt) SR (Q = 2 pkt) SR (Q = 6 pkt) SR (Q = 10 pkt) SR (Q = 14 pkt) SR (Q = 18 pkt)

Figure 5.9: Performance comparison of average packet delay between the two proposed SSWC approaches (i.e., EC and SR) with diﬀerent queue size considerations (𝑄).

5.5.2 Eﬀect of Queue Length Considerations

Figs. 5.9 to 5.12 illustrate the eﬀect of queue size considerations among the power-saving mechanisms. The performance comparisons of average packet delay among the proposed SSWC approaches are shown in Fig. 5.9. Given a traﬃc state, the larger queue size results in higher packet delay since the longer sleep window is selected either to maximize the sleep ratio in the SSWC-SR approach or to minimize the energy consumption in the SSWC-EC scheme.

Since the number of arriving packets is increased with the augmentation of packet arrival rate, shorter sleep window is considered to satisfy constant size of queue, which consequently reduces the average packet delay. The SSWC-EC approach possesses higher packet delay than the SSWC-SR scheme, which can be attributed to the same reason as explained in Section 5.5.1.

Fig. 5.10 shows the performance of average energy consumption between the two proposed SSWC approaches. It is expected that the average energy consumption increases as the value of mean packet arrival rate is augmented in all the cases. Under the situations with smaller queue size, more energy consumption is required in the proposed SSWC approaches

0.1 0.2 0.3 0.4 0.5

Figure 5.10: Performance comparison of average energy consumption between the two pro-posed SSWC approaches (i.e., EC and SR) with diﬀerent queue size considerations (𝑄).

since the length of sleep window will be limited by both the traﬃc state and queue size.

Considering the scenario of constant traﬃc state, shorter sleep windows will be selected with the consideration of smaller queue size, which results in additional number of listening windows during the simulation time and hence increases the energy consumption. Owing to similar ratio of listening frames to sleep frames, both the SSWC-EC and SSWC-SR approaches possess similar energy consumption as shown in Fig. 5.10.

Fig. 5.11 illustrates the performance of packet overﬂow under various queue size consid-erations. The packet overﬂow is deﬁned as the number of packets exceeding the pre-deﬁned queue size compared to the total number of packets in the simulation. It is observed that both the IEEE 802.16e and IEEE 802.16m schemes result in comparatively high packet over-ﬂow under the consideration of queue size less than 5 packets. This can be attributed to the reason that the bursty packets frequently incur the overﬂowing packets while the queue size is small. The packet overﬂow decreases as the increment of queue size in both the IEEE 802.16e and IEEE 802.16m schemes since the large queue size provides the capability for enduring the bursty situation. On the other hand, the SSWC approaches possess relative lower packet overﬂow owing to the reason that the length of each sleep window is determined according to

5 10 15 20

Figure 5.11: Performance comparison of packet overﬂow among IEEE 802.16e, IEEE 802.16m, SSWC-SR, and SSWC-EC approaches under various queue size considerations (𝑄).

the traﬃc state and considered queue size.

Performance comparisons among IEEE 802.16e, IEEE 802.16m, and the proposed SSWC-EC approach over various packet arrival rates is shown in Fig. 5.12. The SSWC-SSWC-EC approach is adopted on behalf of the proposed SSWC scheme since both the EC and SR policies result in similar performance under the consideration of queue size. Comparing to the IEEE 802.16e and IEEE 802.16m schemes, lower energy consumption can be acquired by adopting the SSWC approach. The SSWC-EC approaches result in comparatively high packet delay since the length of each sleep window is selected according to the traﬃc state and considered queue size. The MS wakes up as the number of buﬀered packet is close to the considered queue length, which consequently increases the packet delay. Nevertheless, the primary objective of proposed SSWC approach to reduce energy consumption of the MS can still be achieved.

The merits of the proposed schemes can be observed.

0.1 0.2 0.3 0.4 0.5

Figure 5.12: Performance comparison among IEEE 802.16e, IEEE 802.16m, and SSWC-EC approaches with diﬀerent queue size considerations (𝑄).

5.6 Concluding Remarks

In this chapter, a statistical sleep window control (SSWC) approach for sleep mode operation is proposed to maximize the energy eﬃciency of an MS with non-real-time downlink traﬃc.

Following the notions of IEEE 802.16m system, the sleep mode operation is formulated as a sleep window selection (SWS) problem. The SSWC approach selects the length of each sleep window based on present traﬃc state and the considerations of tolerable delay and/or queue size. The traﬃc model construction (TMC) procedure of SSWC approach formulates a discrete-time Markov-modulated Poisson process (dMMPP) for non-real-time downlink traﬃc via analyzing the traﬃc trace, which infers all the states of the traﬃc. Based on the con-structed dMMPP, a partially observable Markov decision process (POMDP) is adopted in the traﬃc state estimation (TSE) procedure of SSWC approach to conjecture the present traﬃc state at each decision epoch. By exploiting the properties of the POMDP, two suboptimal policies, including the sleep ratio-based (SR) policy and energy cost-based (EC) policy, with the considerations of tolerable delay and/or queue size are proposed within the SSWC ap-proach to resolve the SWS problem. The eﬃciency of proposed SSWC apap-proach is evaluated

and compared via simulations. Simulation studies show that the proposed SSWC approach outperforms the conventional IEEE 802.16e PSC I and the evolutional PSC I of IEEE 802.16m system in terms of both energy eﬃciency and packet delay while the delay constraint is con-sidered. On the other hand, the SSWC approach aggregates the buﬀered packets as much as possible while satisfying the pre-deﬁned queue size, which maximizes the energy conservation of the MS. With the considerations of both tolerable delay and queue size, an eﬃcient sleep mode operation for mobile broadband wireless networks can be acquired by adopting the proposed SSWC approach.

Chapter 6 Conclusions

In this dissertation, four topics on IEEE 802.16-based mobile broadband wireless networks (MBWNs) are presented. The contributions of this dissertation involve developments of ﬂex-ible and contention-free adaptive communication approach and its corresponding scheduling algorithm for mobile stations (MSs), and two power-saving mechanisms for real-time and non-real-time traﬃc respectively. The contributions for each of the four focused areas are summarized as follows:

∙ Chapter 2: A ﬂexible and contention-free adaptive point-to-point communication (APC) approach is proposed to achieve direct transmission between SSs within IEEE

在文檔中行動寬頻無線網路之適應性通訊與省電機制 (頁 118-0)