Considering that the serving policy of the BS in 16e or the ABS in 16m is first-come-first-serve (FCFS) and the M/G/1 queueing system is adopted, the average packet delay is evaluated as follows.
4.2.1 As for IEEE 802.16e
The mean waiting time of a packet in 16e is defined by including both the queueing time and the service time as
E[W(e)] =E[WB0]PB0 + U(E[WL1]PL1 + E[WLi6=1]PLi6=1)
+ E[WBi6=0](1 − PB0 − U(PL1 + PLi6=1)) (4.25)
where U = 0 for PSC of Type I and U = 1 for Type II. WBi is the waiting time in the ith busy period with probability PBi. WLi represents the waiting time in the listening window with its probability PLi while Type II is considered. According to the Pollaczek-Khintchine mean value formula of M/G/1 queueing system as in [21], E[WBi6=0] can be obtained as
E[WBi6=0] = ρ
λ + ρ2+ λ2σ2
2λ(1 − ρ) (4.26)
where σ2 is the variance of the service time and ρ = λ/µ stands for the traffic intensity.
Moreover, based on the M/G/1 queueing system with multiple vacations [23], the mean
waiting time in the first busy period can be calculated as
E[WB0] = E[WBi6=0] + E[TV]. (4.27)
The parameter TV is the remaining vacation time, which can be expressed as (4.28) because of the average sleep cycle δ.
E[TV] =TDδ + TL
2 · PR
+ (1 − PR)(1 − ΦI(e)0,δ ) ·T(e)u,δ− SCδ−1
2 (4.28)
where PR = (φR(e)δ )/(φR(e)δ + φI(e)δ ). The remaining terms within (4.25) will be computed for both Type I and Type II below.
4.2.1.1 Type I
The only parameter that is left to be determined for Type I is the probability of the average number of packets initiated in the first busy period, i.e.
PB0 = ϕ(eI)+ λ · E[TB0]
λ · E[TN] + ϕ(eI) (4.29)
where ϕ(eI) = P∞
i=1i · ϕ(ei I) represents the average number of packets happened at the start of the normal mode.
4.2.1.2 Type II
Considering νi,n is denoted as the probability of i DL packets at the nth detection window under the condition that the sleep mode terminates after the nth cycle. The
parameter νi,n can be obtained as
Its expected value in terms of i can be computed as
νn =
C
X
i=0
i · νi,n. (4.31)
Moreover, according to the M/G/1 queueing system with multiple vacations, the mean waiting time in the listening window can be approximated as
E[WLi] ≅ TDi 2 +νi
2 · 1
µ. (4.32)
It is noted that the waiting time from the UL packets happened within the listening window is ignorable considering the comparably smaller time duration of the listening window. Consequently, the other parameters within (4.25) for Type II of 16e can be calculated as follows according to the average sleep cycle δ.
PB0 = λ · E[TB0] + ϕ(eII)
4.2.2 As for IEEE 802.16m
On the other hand, the average packet delay for 16m can be computed as the sum of the wait time for serving in the listening window and the time spent by the UL packets within the additional duration of the UL transmission, i.e. E[W(m)] = E[WL] + E[WU L]
Similarly, E[WL] can be further expressed as
E[WL] = E[WM/G/1] + E[TV] (4.36)
where E[WM/G/1] is the waiting time of a normal M/G/1 queuing system, which is equal to (4.26). Also,
E[WU L] = 1
µ +(λu/µ)2+ λ2uσ2
2λu(1 − λu/µ) . (4.37)
Furthermore, TV is the remaining vacation time of the system and its average value would be
E[TV] = TCδ∗
2 · PA+ (T(m)u,δ∗− SCδ∗−1)
2 · PB+ TCδ∗∗
2 · PC (4.38)
according to the average sleep cycle δ∗. Moreover, Pk = Φkδ∗/(ΦAδ∗ + ΦBδ∗ + ΦCδ∗), where Φkδ∗ = φk(m)δ∗ + φk ext(m)δ∗ , k ∈ A, B, C (mean values of φk(m)δ∗ and φk ext(m)δ∗ shall be utilized for k = B, C).
Chapter 5
Proposed POMDP-based Sleep Window Determination (PSWD) Approach
According to the performance analysis, it is intuitively observed that the 16m seems to be more power efficient than that of 16e. However, there are numbers of redundant under-utilized listening windows in the sleep mode operation of 16m owing to the scheme of binary-exponential growth of sleep cycles adopted by 16m. Moreover, it is also responsible for excessive energy cost during state transition, i.e., switching from sleep
AMS ABS
S1 ●●●
frame pkt1
● ●●
●● ●
τ
L
TS
1 TL TS2
Sleep Mode Normal
Mode
S ts1
Decision Epoch d1
L
δ δ
1st Control Cycle C1
Decision Epoch d2 Decision Epoch d3
2nd Control Cycle C2
TL
S2 S3
Figure 5.1: Schematic diagram of ideal sleep mode operation for an AMS.
windows to listening windows and vice versa. It is thus motivated that a more flexible sleep mode mechanism, i.e., a sleep windows decision approach should be designed which adaptively adjusts the length of sleep windows based on the traffic state. In [24] recently published, a statistical sleep window control (SSWC) approach has been proposed for the sleep windows decision problem under tolerable average packet delay for non-real-time DL traffic, that is, Type I in 16m. In this work, the design concept is further extended in order to be fulfilled for all traffic patterns and power-saving types in 16m, including both Type I and Type II. Furthermore, both the DL and UL traffic are also considered during the selection of sleep windows sizes. First of all, the definition of control cycles for the PSWD approach is stated as follows.
Definition 1 (Control Cycle). Given an ABS and an AMS that expects to enter the sleep mode or has stayed in the sleep mode, a control cycle Ci is defined as a time duration consisting of a decision epoch di, a sleep window Si, and a listening window Li. The ABS determines the length of the sleep window Si at the decision epoch di. The AMS stays in the power-saving mode during the sleep window with length TSi and wakes up for data transmission in the listening window Li until finishing serving all packets, then enters the subsequent control cycle.
Fig. 5.1 illustrates the ideal sleep mode operation of a 16m AMS which the proposed PSWD approach intends to achieve, wherein all the control signals are omitted for the sake of description convenience. Each control cycle Ci (i 6= 1) is overlapped with the adjacent control cycles Ci−1and Ci+1. The first control cycle C1begins at the last frame of AMS’s idle period in the normal mode. The remainder control cycles are individually started at the end of every listening window within the previous control cycle.
The target of PSWD approach is to find adequate length of sleep window in each control cycle in light of present traffic state, which meets the delay constraint of packets and maximizes the power-saving efficiency as possible. According to the process of
ongoing sleep mode parameter update mentioned in Section 2.3, in the proposed PSWD approach, the ABS can inform an AMS about the calculated length of each sleep window without any additional control overhead by using originally defined messages, such as AAI SLP-RSP or AAI TRF-IND. On the contrary, an AMS may also send its UL traffic condition through AAI SLP-REQ in order to provide the reference UL information to the serving ABS. It is noted that by means of exploiting such parameter negotiation scheme, each sleep window in every control cycle belongs to a brand-new initial sleep cycle, hence the proposed PSWD approach can be applied to no matter Type I or Type II of 16m.
As shown in Fig. 5.1, the calculated length of sleep window should meet the tolerable delay, e.g. for the first sleep window S1, it shall be terminated before the expiration of the first coming packet pkt1, that is, the termination selected at the decision epoch d1 has to fall within the range δ. Consequently, it is inferred that the determined length of each sleep window is dominated by the knowledge of the current traffic patterns, especially DL traffic in 16m, for the interruption resulted from UL transmission at any time may have no impact on the length and phase of the sleep cycles. Nevertheless, these kinds of traffic states are considered difficult to be acquired directly. Only the number of packets arrived in the buffer during the previous control cycle can be observed, which may provide sufficient information for the ABS to estimate the potential state of present traffic.
For the situation described above, a POMDP [25] [26] technique is fairly feasible for the ABS to speculate about the present state of traffic at each decision epoch by the observed information from the buffers. In the following three sections, details of the proposed PSWD method will be introduced, which consist of the estimation procedure for current traffic state by the POMDP scheme, the evaluation metrics, and the sleep window determination policy of PSWD approach.
B(dt) Belief state
Sleep window determinaon Arrived PDUs
observaon (from buffer)
Traffic state
B(dt+1)
S(dt-1) S(dt) S(dt+1)
z(dt-1) z(dt) z(dt+1)
B(dt-1)
Hidden Observable
dt-1 Decision epoch dt dt+1
a(dt-1) a(dt) a(dt+1)
Figure 5.2: Schematic diagram of POMDP model for PSWD approach
5.1 Traffic State Estimation
The procedure of traffic state estimation resorts to a POMDP model in order to conjec-ture the present traffic state at each decision epoch. A typical POMDP model can be expressed by a tuple hS, A, T , Z, O, Ri, where S is a set of states, A is a set of actions, T is a set of state transition probabilities, Z is a set of observations, O is a set of observation probabilities, and R is a set of immediate rewards. In the proposed PSWD approach, the source of DL and UL traffic are generated by a discrete-time Markov-modulated Poisson process (dMMPP), which is considered more general than the con-ventional Poisson traffic, and can be capable of capturing the correlation characteristics in the modern Internet and multimedia traffic at multiple time scales [27] [28] [29]. Here S consists of two components: Sd and Su which correspond to the set of dMMPP states of DL and UL traffic, respectively. Also, {Td, Tu} ∈ T are related to the state transition probability matrixes of DL/UL direction. Furthermore, the set of actions is defined as A = {a1, a2, · · · , aN} where an represents the action of selecting a sleep window of length TSan corresponding to the outputs of the PSWD approach.
Considering a sequence of control cycle {C1, C2, · · · , CT} in the proposed SSWC approach, the set of corresponding decision epoches is defined as D = {d1, d2, · · · , dT}.
Fig. 5.2 depicts the schematic diagram of the POMDP model for the proposed PSWD approach (as for DL traffic). At each decision epoch dt ∈ D, the traffic state s(ddt) or s(dut) ∈ S is considered hidden and unobservable. However, the number of packets that arrived in the buffer of ABS/AMS during the previous control cycle Ct−1 is available and can be acquired. Thus the set of observations on the quantity of arrivals is written as Zd = {zd(d1), zd(d2), · · · , z(ddT)} where zd(di) denotes the number of DL packets arrived at the ABS in the interval between the (i − 1)th and the ith decision epoches, and Zu can also be defined in this way. The observation probability of DL packets can be defined as
o(zd(dt), a(dt−1), s(ddt)), P r(zd(dt)|a(dt−1), s(ddt))
= (λ(ddt)TSa(dt−1))zd(dt) zd(dt)! e−(λ
(dt) d TS
a(dt−1))
, (5.1)
which is a conditional probability of an observation zd(dt) ∈ Z at decision epoch dtgiven the action (i.e. length of previous sleep window) a(dt−1) ∈ A chosen at dt−1 and the present DL traffic state s(ddt) ∈ S at dt, in which arrival rate equals λ(ddt).
In order to nearly achieve the optimal results, the notion of belief state is introduced in the POMDP model by referring to the recent history of previous observations so as to estimate the present traffic state more precisely than merely by utilizing most recent observations from the buffer. Given a decision epoch dt ∈ D, the set of belief state of DL traffic is defined as Bd(dt) = {b(s(dd 1t)), b(s(dd 2t)), · · · , b(s(dd Mt))}, which represents the estimated probability distribution over the set of traffic states S = {sd 1, sd 2, · · · , sd M}.
Each element b(s(dd jt)) denotes the probability of DL traffic state sd j at decision epoch dt. It is noted that 0 ≤ b(s(dd jt)) ≤ 1, ∀sd j ∈ Sd and P
∀sd j b(s(dd jt)) = 1, ∀dt ∈ D since the DL traffic must belong to one of the states within the Sd set at any given decision epoch. As shown in Fig. 5.2, the belief state Bd(dt) is updated at decision epoch dt
by exploiting previous action a(dt) ∈ A and the corresponding observation zd(dt+1) ∈ Z.
Thus each element b(s(dd jt)) of the belief state Bd(dt+1) can be derived as 1
b(s(dd jt)) = P r(s(dd jt)|Bd(dt−1), a(dt−1), zd(dt)) (5.2)
= P r(z(ddt)|a(dt−1), s(dd jt), Bd(dt−1))P r(s(dd jt)|a(dt−1), Bd(dt−1))
P r(z(ddt)|a(dt−1), Bd(dt−1)) (5.3)
=
o(zd(dt), a(dt−1), s(dd jt))P
s(dt−1)d i ∈Sdpi,jb(s(dd it−1)) P
s(dt−1)d i ∈Sd
P
s(dt)d j ∈Sdb(s(dd it−1))pi,jo(zd(dt), a(dt−1), s(dd jt)). (5.4) It is noted that the belief state is a integrated statistics for the entire history of the process, which progressively merges the effect of previously determined action and the corresponding observation at each decision epoch. Since the belief state is updated at each decision epoch, the time complexity can be calculated as O(|S|), where |S|
represents the total number of states in S, including both DL and UL traffic. By means of the belief state, more precise traffic states can be appraised via exploiting the proposed PSWD approach.