Fig. 6.3 shows the performance comparison of the number of waiting time slots versus the total number of spectrum handoff for both the proposed POSH scheme and the RCS method. Two different channel conditions are considered for comparison purpose as follows. A better channel condition Cb is chosen with the transition probability from idle to idle state for each channel as τ (ci,k = 0, ak, ci,k+1 = 0) = 0.8, 0.7, 0.65 for i = 1, 2, 3;
while that from busy to idle state is selected as τ (ci,k = 1, ak, ci,k+1 = 0) = 0.4, 0.5, 0.55 for i = 1, 2, 3. On the other hand, a worse channel condition Cw is determined with the transition probability from idle to idle state as τ (ci,k = 0, ak, ci,k+1 = 0) = 0.4, 0.3, 0.35, and that from busy to idle state is set as τ (ci,k = 1, ak, ci,k+1 = 0) = 0.1, 0.2, 0.15.
For fair comparison, the NSH method is not implemented in this case since the CR user can always stay at the channel with better condition. It is intuitive to see that the total number of waiting time slots is increased as the number of spectrum handoff is augmented. Furthermore, the secondary user has to wait for comparably more time slots in the worse channel case under both schemes. Nevertheless, the total waiting time
0 100 200 300 400 500 600 700 800 900 1000
Total Number of Transmission Time Slots
Number of Waiting Time Slots
RCS_Cw POSH_Cw RCS_Cb POSH_Cb
Figure 6.4: Performance comparison: the number of waiting time slots versus the total number of transmission time slots.
slots acquired from the proposed POSH scheme is comparably smaller than that from the RCS method under both channel conditions. It is also observed that the POSH scheme performs better as the number of spectrum handoff is increased. The reason can be attributed to the situation that more updated belief states are acquired by the POSH scheme as the number of handoff is augmented.
Fig. 6.4 illustrates the performance comparison between the number of waiting time slots and the total number of transmission time slots. It is noticed that different numbers of waiting time slots and handoff numbers will be resulted by each scheme at every specific number of transmission time slots. In other words, the combining effects from both the waiting time slots and the handoff numbers will be revealed in Fig. 6.4 at each horizontal data point. It can be observed that the proposed POSH algorithm still outperforms the RCS scheme under both the Cb and Cw channel conditions. Even though the effect from the total number of spectrum handoff has not been considered
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 102
103 104
λ
Number of Waiting Slots RCS
NSH POSH
Figure 6.5: Performance comparison: the number of waiting time slots versus traffic arrival rate (numbers of spectrum handoff = 250).
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
102 103 104
λ
Number of Waiting Slots
RCS NSH POSH
Figure 6.6: Performance comparison: the number of waiting time slots versus traffic arrival rate (number of transmission time slots = 10000).
0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7
Figure 6.7: Performance comparison: the waiting time with practical consideration versus different arrival rates.
and the NSH schemes under different values of packet arrival rate λ of the primary user.
Fig. 6.5 shows the comparison under fix numbers of spectrum handoff equal to 250 and Fig. 6.6 is performed under number of transmission time slots equal to 1200. It can be observed that the proposed POSH scheme can outperform the other two methods under different packet arrival rates. The benefits from the adoption of the POSH algorithm is especially revealed at smaller values of packet arrival rate since there can be more opportunity for the POSH scheme to select a feasible target channel.
The practical consideration for the three handoff schemes are presented as in Figs.
6.7 and 6.8. The corresponding parameters are listed as follows: Tslot = 100 ms, Tsens = 5 ms, Tsw = 10 ms, Ths = 1 ms, and Tack = 1 ms. Fig. 6.7 illustrates the expected waiting time as was obtained from (4.8), (4.10), and (4.12) from the NSH, RCS, and POSH scheme respectively. Two cases with policy time Tp = 0 and 5 ms for the proposed POSH scheme are considered; while the update time T =??.
0.2 0.3 0.4 0.5 0.6 0.7
Figure 6.8: Performance comparison: the net transmission time per slot versus different arrival rates.
the overall performance is still better in comparison with the NSH and RCS methods.
Furthermore, it is observed in Fig. 6.7 that the waiting time Tw,poshwill only be affected by Tp while the target channel of spectrum handoff has high probability in idle state.
In other words, Tp will degrade the performance of proposed POSH scheme at lower primary traffic circumstance, and will become insignificant with higher primary traffic.
Nevertheless, since POSH scheme can provide better performance among the three schemes at low primary traffic (as shown in Figs. 6.5 and 6.6), it is concluded that the degraded effect from Tp will not be significant by adopting the POSH scheme.
In order to better present the utilization of licensed spectrum, Fig. 6.8 is exploited to illustrate the net transmission time within a time slot. Two cases with update time Tp = 0 and 3 ms for the proposed POSH scheme are considered; while the policy time Tp =??. It is noticed that the update time Tu is considered additional time to expense for the proposed POSH scheme; while the policy time Tp will only occur as spectrum handoff happens. It is intuitive to observe that the net transmission time obtained from all three schemes decreases as the arrival rate of primary traffic is increased.
Furthermore, as λ is increased, the net transmission time of proposed POSH scheme
2 3 4 5 6 7 8
Expected Number of Waiting Time Slots
NSH with λ=0.5
Figure 6.9: Performance comparison: the expected number of waiting slots versus different number of channels (with 2 CR users).
becomes closer or even worse than the RCS and NSH schemes, e.g. the POSH scheme with Tu = 3 ms is worse than the RCS algorithm as λ > 0.75. Therefore, practical consideration for these handoff schemes can provide a channel selection criteria for the CR users to determine which handoff scheme should be applied to obtain their target channel.
Figs. 6.9 and 6.10 compare the performance of NSH scheme, RCS scheme, and the proposed M-POSH protocol under the circumstance of multiple CR users within multi-channels network. Fig. 6.9 shows the performance comparison under two CR users; while Fig. 6.10 illustrates the circumstance with 7 channels in the networks.
Two different arrival rates of primary traffic are considered for all schemes, i.e. λ = 0.2 and 0.5. It is apparently to observe that the NSH scheme results in the same performance under different numbers of channels and CR users since it does not perform any spectrum handoff activity. In other words, the NSH scheme in multi-user scenario
1 2 3 4 5 6 7
Figure 6.10: Performance comparison: the expected number of waiting slots versus different number of CR users (with 7 available channels).
channels is augmented; while it is increased with the augmentation of total CR users due to the potential collisions happened in the network. The performance of RCS scheme can becomes worse than that from NSH method in certain circumstances accounting to NSH scheme at least guarantee no collisions will happen in the network. From example as in Fig. 6.10, the RCS scheme with λ = 5 results in higher number of waiting slots as the number of CR users greater than 5 in comparison with the NSH method.
Nevertheless, the proposed M-POSH protocol can adaptively selecting the target channel based on the availability of network channels with the consideration of collisions between the CR users. As shown in both Figs. 6.9 and 6.10, the proposed M-POSH scheme outperforms both the RCS and NSH algorithms under different scenarios. As the network channels are in occupied conditions, e.g. the number of CR users is equal to the number of licensed spectrum as the most left data point in Fig. 6.9, the proposed M-POSH protocol will decide to stay at the current channel in order to reduce the probability of collision between the CR users. On the other hand, as long as the network contains more available channels that can be utilized, the M-POSH protocol will distribute the CR users that intend to conduct handoff to exploit those available channels. Consequently, the performance of M-POSH scheme can achieve the optimal
performance as that obtained from the POSH scheme for single user case (as presented in Fig. 6.1). The merits of the proposed POSH and M-POSH schemes can therefore be observed.
Chapter 7 Conclusion
This paper proposes a strategy for post-sensing spectrum handoff based on partially observable Markov decision process (POMDP) in the overlay cognitive radio (CR) net-works. With only partially observable state information, the proposed POMDP-based spectrum handoff (POSH) scheme selects the optimal target channel in order to achieve the minimal waiting time for packet transmission. Furthermore, in order to consider the multi-user CR network, POMDP-based multi-user spectrum handoff (M-POSH) protocol is proposed to resolve the collision problem among multiple users that intend to conduct spectrum handoff. It is observed from both the simulation and analytical results that the proposed POSH M-POSH protocols can effectively reduce the waiting time of spectrum handoff for a partially observable CR network.
Bibliography
[1] F.C.C, “Unlicensed operation in the TV broadcast bands,” ET Docket, pp. 4–186, 2006.
[2] FCC, “Fcc spectrum policy task force report,” ET Docket No. 02-135., 2002.
[3] Q. Zhao, L. Tong, A. Swam, and Y. Chen, “Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework,” IEEE J. Sel. Areas Commun., vol. 25, pp. 589–600, 2007.
[4] V. K. Bhargava, “Research challenges in cognitive radio networks,” Proc. IEEE ICCCN, pp. 1–6, 2007.
[5] S. Shanker, “Squeezing the most out of cognitive radio: A joint mac/phy perspective,” Proc.
IEEE ICASSP, 2007.
[6] A. Sahai, N. Hoven, and R. Tandra, “Some fundamental limits on cognitive radio,” Proc of Alleston Conference, 2004.
[7] S. Sengupta, S. Brahma, M. Chatterjee, and N. S. Shankar, “Enhancements to cognitive radio based IEEE 802.22 air-interface,” Proc. IEEE ICC, 2007.
[8] C. Cordeiro, K. Challapali, D. Birru, and N. S. Shankar, “IEEE 802.22: The first worldwide wireless standard based on cognitive radios,” Proc. IEEE DySPAN, pp. 328–337, 2005.
[9] L. C. Wang and A. Chen, “On the performance of spectrum handoff for link maintenance in
[11] L.-C. Wang and C. W. Wang, “Modeling and analysis for proactive-decision spectrum handoff in cognitive radio networks,” Proc. IEEE ICC, 2009.
[12] L. C. Wang and C. W. Wang, “Spectrum handoff for cognitive radio networks: Reactive-sensing or proactive-sensing?” Proc. IEEE IPCCC, 2008.
[13] X. Long, X. Gan, Y. Xu, J. Liu, and M. Tao, “An estimation algorithm of channel state transition probabilities for cognitive radio systems,” Proc. IEEE CROWNCOM, 2008.
[14] A. R. Cassandra, “Exact and approximate algorithms for partially observable markov decision processes,” Ph.D. dissertation, Brown University, 1994.
[15] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” Artificial Intelligence, vol. 101, pp. 99–134, 1998.
[16] K. J. Astrom., “Optimal control of markov decision processes with incomplete state estimation,”
Journal of Mathematical Analysis and Applications, pp. 174–205, 1965.
[17] H.-T. Cheng, “Algorithms for partially observable markov decision processes,” Ph.D. dissertation, University of British Columbia, British Columbia, Canada, 1988.
[18] Q. Zhao, L. Tong, and A. Swami, “Decentralized cognitive mac for dynamic spectrum access,”
Proc. IEEE DYSPAN, pp. 224–232, 2005.
[19] M. L. Littman, “The witness algorithm: Solving partially observable macov decision process,”
Technical Report: CS-94-40, 1994.
[20] R. D. Smallwood and E. J. Sondik, “The optimal control of partially observable markov processes over a finite horizon.” Operations Research. 21, pp. 1071–1088, 1973.
[21] R. S. Sutton and A. G. Barto, “Reinforcement learning: An introduction.” MIT Press, Cambridge, Massachusetts, 1998.
[22] L. C. Wang, Y. C. Lu, and C. W. Wang, “Latency analysis for dynamic spectrum access in cognitive radio: Dedicated or embedded control channel?” Proc. IEEE PIMRC, 2007.
[23] D. Gross and C. M. Harris, Fundamentals of Queueing Theory, 3rd ed. Wiley, 1998.
[24] W. Hu, D. Willkomm, M. Abusubaih, J. Gross, G. Vlantis, M. Gerlaand, and A. Wolisz, “Cog-nitive radios for dynamic spectrum access - dynamic frequency hopping communities for efficient IEEE 802.22 operation,” IEEE Communications Magazine, vol. 45, pp. 80–87, 2007.
[25] Z. Kostic and I. Maric, “Dynamic frequency hopping in wireless cellular systems-simulations of full-replacement and reduced-overhead methods,” Proc. IEEE VTC, pp. 914–918, 1999.