Appendix 2.A Assumptions for Stochastic Approximation
5.6 Conclusion and and Open Issues
In this chapter we studied the problem of spectrum trading with multiple sellers and time-varying spectrum opportunities. We formulated the spectrum trading as a two-level Stackelberg game whose upper and lower level subgames model the service selection of SUs and the price competition of SPs, respectively. Decentralized stochastic learning-based algorithms were proposed for the strategic learning in both levels. Simulation results demonstrated the convergence of the algorithm towards a pure strategy Nash equilibrium point in both levels. In the lower level game, the proposed method outperforms the best random selection scheme in terms of average utility, while the performance loss compared to the centralized exhaustive search is limited. Moreover, the proposed method achieves significantly improved fairness compared to the exhaustive search method. On the other hand, the price competition among spectrum sellers decreases their summed revenue, but brings benefit to the secondary users.
The major direction of extending this work is the consideration on more realistic model
that takes QoS parameters into account.
Part III
Examples of Distributed Learning
with Partial Cooperation
Chapter 6
Self-organized Channel Assignment in Two-tier Distributed Networks
In this chapter, we study the channel assignment strategy in orthogonal frequency division multiple access (OFDMA) based two-tier distributed networks where macrocells and distributed cognitive radio networks (CRNs) are overlaid. We formulate the channel selection problem as a potential game which has at least one pure-strategy Nash equilib-rium (NE). To achieve the NE we propose a stochastic learning-based algorithm which does not require the information of other players’ actions and the time-varying channel.
The cognitive radio base stations or cluster heads are considered as players in the game, and act as self-organized learning automata and adjust selection strategies based only on their own action-reward history. The convergence property of the proposed algorithm toward pure strategy NE points is shown theoretically and verified numerically. Simula-tion results demonstrate that the learning algorithm yields a 26% sensor node capacity improvement as compared to the random selection, and incurs less than 10% capacity loss compared to the exhaustive search.
6.1 Introduction
S
pectrum utilization can be improved with two-tier networks. Efficient interference mitigation is the key to maintain the performance of two-tier networks. We start our presentation with two examples of two-tier distributed networks.6.1.1 Examples of Two-tier Distributed Networks
Femtocell Networks
Femtocell technology [46] has been extensively considered in next-generation wireless standards such as 3GPP-LTE [47] as a means to enhance cell coverage and user capacity.
In femtocell networks, the low-power and low-cost indoor base stations (referred to as home base stations, HBSs) utilize the wired broadband connection as backhaul and are planned to be easily installed by consumers. By utilizing femtocells, the indoor femtocell user equipment (FUE) is able to attain high data rate due to the short distance from HBS, and operators can reduce the cost in deploying macro base stations (MBSs) with the aid of HBS to serve the FUEs in the coverage holes.
In the absence of a central controller, resource allocation in femtocell networks is implemented in a distributed manner. Resource allocation with interference mitigation can be achieved by assigning different spectrum to adjacent femtocells. These methods can be viewed as variations of frequency planning, and usually require negotiations among HBSs.
Co-channel implementation brings the advantage of efficient spectrum usage. However, it also results in CCI between the femtocell(s) and the macrocell in various ways. In Fig.
6.1, different CCI possibilities are listed according to their sources, their victims, and whether they occur in the DL or the UL. Interference scenarios #1 and #2 involve the CCI between the femtocell user equipment (FUE) and the macrocell network, scenarios #3 and #4 involve the CCI between the macrocell user equipment (MUE) and the femtocell network, while scenarios #5 and #6 involve the CCI scenarios between close-by femtocell
MBS
Figure 6.1: Possible interference scenarios related to femtocell communications.
networks. All these interference scenarios can be considered for both time division duplex (TDD) and frequency division duplex (FDD) systems. It should be noted that these scenarios are based on the assumption that femtocell is not allowed to be in DL mode while macrocell is in UL subframe (in TDD systems), or femtocell cannot use the UL frequency band of the macrocell for DL (in FDD systems).
Distributed Sensor Networks
In wireless sensor networks [48], spatially distributed, low-power and low-cost sensor nodes are deployed in a geographical area to monitor the environment. The sensor nodes usually form clusters, and in each cluster there is a energy-rich sensor node acting as the cluster head, while other sensor nodes are referred to as cluster members. A cluster head is a special sensor node with better cognitive radio functionality, and is responsible for the spectrum sensing and the channel assignment among its cluster members.
To enable the various kinds of services [33, 49, 50] provided by a pervasive sensing sys-tem, proper radio resource management [51] is important. Due to the spectrum scarcity
and the ad-hoc nature of sensor network deployment, it could be hard to assign licensed bands to sensor networks. Therefore, the cognitive radio [39] technology has been con-sidered as a promising solution to the channel assignment problem of sensor networks.
Cognitive radio technology enables dynamic spectrum access (DSA) and allows the unli-censed users by sensing the usage information of the spectrum from the radio environment.
Akan et al. [52] provided a survey on cognitive radio sensor networks. By utilizing the CR technology, the sensor networks are able to attain high data rate due to the spectrum holes. In addition, dynamic spectrum access helps mitigate the interference incurred by dense deployment of sensor nodes.
Despite the promising features of cognitive sensor networks, the deployment of such heterogeneous networks with sensor clusters underlying the same spectrum as macrocells and in the same geographical area brings new technical challenges. In particular, we are interested in the case of densely populated sensor networks where, due to extensive frequency reuse, the co-channel interference (CCI) among sensor nodes and the cross-tier interference (between the macrocell and sensor networks) affect the system performance.
6.1.2 Contributions
In this chapter, we consider a two-tier distributed network, and address the self-organized channel assignment problem. The main contributions of this work are summar-ized as follows.
• We model the distributed channel assignment problem as an ordinal potential game (OPG). The game considers time-varying channel availability (as a result of the resource allocation to MUEs) as its external state.
• We propose a fully decentralized channel assignment algorithm in which the chan-nel is selected by each link independently based on its action-reward history. The strategy update of all links are simultaneous, without any coordination. The conver-gence property of the algorithm to a pure strategy NE point is verified numerically.
Through numerical simulations, we also show that the proposed method performs quite close to the exhaustive search.
6.1.3 Game-theoretic Problem Mapping
The mapping of distributed channel assignment problem to game-theoretic formulation is summarized in Table 6.1.
Table 6.1: Mapping to game-theoretic formulation.
Elements in game Characters in channel assignment problem Players Femtocell BSs or sensor cluster heads
Strategies Channel assignments
Reward gSINR (to be defined)
External state Channel availability
This chapter is organized as follows. In Section 6.2, we review the previous works. In Section 6.3, the system model for two-tier distributed network is presented. Section 6.4 describes the game-theoretic model of the channel selection problem. Section 6.5 presents the stochastic learning procedure carried out by HBSs. Finally, numerical results are given in Section 6.6, and the conclusion is drawn in Section 6.7.
Notations: Normal letters represent scalar quantities; uppercase and lowercase bold-face letters denote matrices and vectors, respectively. Given a finite setA, ∆(A) repres-ents the set of all probability distributions over the elemrepres-ents ofA. 1l{cond} is the indicator function which equals one if the condition cond is satisfied, and zero otherwise.