Dynamic Spectrum Access with QoS and Interference Temperature Constraints
Yiping Xing, Student Member, IEEE, Chetan N. Mathur, Student Member, IEEE, M. A. Haleem, Member, IEEE, R. Chandramouli, Member, IEEE, K. P. Subbalakshmi, Member, IEEE
Abstract— Spectrum is one of the most precious radio resource.
With the increasing demand for wireless communication, effi- ciently using the spectrum resource has become an essential issue. With the Federal Communications Commission’s (FCC) spectrum policy reform, secondary spectrum sharing has gained increasing interest. One of the policy reforms introduces the concept of an interference temperature—the total allowable interference in a spectral band. This means that secondary users can use different transmit powers as long as the sum of these power is less than the interference threshold.
In this paper, we study two problems in secondary spectrum access with minimum signal to interference noise ratio (quality of service (QoS)) guarantee under an interference temperature constraint. First, when all the secondary links can be supported, a non-linear optimization problem with the objective to maximize the total transmitting rate of the secondary users is formulated.
The non-linear optimization is solved efficiently using geometric programming techniques. The second problem we address is when not all the secondary links can be supported with their QoS requirement, it is desirable to have the spectrum access opportunity proportional to the user priority if they belong to different priority classes. In this context, we formulate an operator problem which takes the priority issues into consid- eration. To solve this problem, firstly we propose a centralized reduced complexity search algorithm to find the optimal solution.
Then, in order to solve this problem distributively, we define a secondary spectrum sharing potential game. The Nash equilibria of this potential game are investigated. The efficiency of the Nash equilibria solutions are characterized. It is shown that distributed sequential play and an algorithm based on stochastic learning attain the equilibrium solutions. Finally, the performances are examined through simulations.
Index Terms— Wireless communication, Access schemes, Con- strained optimization, Mobile communication systems.
I. INTRODUCTION
E
NHANCING spectrum efficiency is an important task of regulatory authorities worldwide. A number of experi- mental studies [1] show that spectrum is used inefficiently both in space and time. Low utilization and increased demand for the radio resource suggests the notion of secondary use, which allows unused parts of spectrum owned by the primary license holder to become available temporarily for secondary (non- primary) users. The dynamic access of spectrum by secondary users is one of the promising ideas that can mitigate unsatis- fied spectrum demand, potentially without major changes to incumbents. The wireless device measures RF energy in theManuscript received November 19, 2005; revised May 16, 2006; accepted July 18, 2006
The authors are with the Department of Electrical and Computer Engi- neering, Stevens Institute of Technology, Hoboken, NJ 07030 USA (E-mail:
{yxing, cnanjund, mhaleem, mouli, ksubbala}@stevens.edu.)
Fig. 1. This figure [21] shows the power that is received from the licensed transmitter as a function of distance. The spectrum agile radio has opportunities represented by the righthand side box.
channel or the received signal strength indicator to determine whether the channel is idle or not. But this approach has a problem in that wireless devices can only sense the presence of a primary if and only if the energy detected is above a certain threshold. It is true that one can not arbitrary lower the threshold as this would result in non-detection because of the presence of noise. In the feature detection approach, which has been used in the military to detect the presence of weak signals [2], the wireless device uses cyclostationary signal processing to detect the presence of primaries. If a signal exhibits strong cyclostationary properties, it can be detected at very low signal-to-noise ratios (SNR) [3]. Then, the question is how to share the available spectrum efficiently and fairly.
The FCC spectrum Policy Task Force [4] has recommended a paradigm shift in interference assessment, that is, a shift away from largely fixed operations in the transmitter and toward real-time interactions between the transmitter and receiver in an adaptive manner. The recommendation is based on a new metric called the interference temperature, which is intended to quantify and manage the sources of interference in a radio environment. The interference temperature is defined to be the RF power measured at a receiving antenna per unit bandwidth. The key idea for this new metric is that, firstly, the interference temperature at a receiving antenna provides an accurate measure for the acceptable level of RF interference in the frequency band of interest; any transmission in that band is considered to be “harmful” if it would increase the noise floor above the interference temperature threshold as shown in Fig. 1. Secondly, given a particular frequency band in which the interference temperature is not exceeded, that band could be made available to secondary users. Hence, a secondary device might attempt to coexist with the primary, such that the presence of secondary devices goes unnoticed.
Related work on secondary use of radio spectrum has appeared in [5] [6] [7] [8] [9]. Here we consider a scenario
similar to [9], where secondary users wish to use a local, relatively short-term data service, and all users adopt a spread spectrum signaling format, in which the transmitted power is evenly spread across the entire available band controlled by the manager. A practical realization of this model would be when secondary users (with spread spectrum signaling) and primary direct-sequence CDMA (DS-CDMA) system coexist in the up-link spectrum band of the primary DS-CDMA system. In the uplink, the interference temperature can be measured at the base station which is the receiver of the primary system.
Hence, the number of measuring points can be significantly reduced. But of course our proposed secondary access is definitely not limited to this scenario. The spectrum can be TV broadcast band, or other emergency band, where the primary transmission will not contribute to the interference temperature. In [9], auction mechanisms for allocating the received power are studied. The logarithmic utilities which is a function of the received Signal-to-Interference Ratio (SIR) is maximized under the constraint of interference temperature.
But without any constraint on the minimum received SIR or maximum transmitting power for the secondary users, the auction based mechanisms may lead to inefficient solutions.
The received SIR for some secondary links may become too low and energy wastage due to several retransmissions will cause interference to other links. It is also possible that the required transmit power for the secondary user may exceed the maximum available transmitting power for the secondary users. An alternative choice would be to completely switch off some of the secondary links when all the secondary users cannot be supported by the system, by coordinated control. In this way, the active secondary links are provisioned with QoS in the sense of a guaranteed minimum achievable SIR. And those links switched off at this time period can be awakened when it can be supported with its minimum required SIR.
In our formulation, we take these factors into consider- ation. We first propose a centralized solution which is a logarithmic utility maximization with constraints. This non- convex optimization problem can be transformed into a convex optimization, which can be solved by Geometric Programming efficiently [10] [11].
In this paper, when not all the secondary links can be supported with their minimum SIR requirement, it is desirable to have the spectrum access opportunity proportional to the user priority if they belong to different priority classes. We formulated an operator problem which maximizes the operator revenue. To solve this problem, a centralized reduced com- plexity searching algorithm is introduced to find the optimal subset of allowable links. Because the nature of secondary access, distributive algorithms would be desirable. Firstly, we define a secondary spectrum sharing potential game which takes priority classes into consideration, then we propose a sequential play solution which converges to the Nash equilib- rium (NE) of the game. Operating at the NE will guarantee that the received power at the measuring point will not exceed the interference temperature constraint and the target SIR for each active link is guaranteed. Besides that, different secondary users with different accessing priorities will have different accessing opportunities with our proposed method. We note
P1
P2
PM
Receivers Transmitters
Measurement Point
h11
h12
h21
h22
h10
h20
hM0
n0
n0
n0
Fig. 2. System model for M transmitter-receiver pairs.
that sequential play requires significant information and sig- nalling to operate. To mitigate this drawback, we formulated an stochastic learning automata algorithm, where asynchronous updating is permitted, and only local information is needed to complete the probability update.
II. SECONDARYSPECTRUMSHARINGMODEL
Spectrum with bandwidth W is to be shared among M spread spectrum users, where a user refers to a transmitter and an intended receiver pair. For each i, the received SIR is given by
γi= yihii 1
L(P
j6=iyjhji) + σ2, (1) where L is the normalized spreading sequence length, yi is user i0s transmission power, hij is the channel gain from user i0s transmitter to user j0s receiver, and σ2 is the background noise power that is assumed to be the same for all users. In order to satisfy an interference temperature constraint, the total received power at a specified measurement point must satisfy
XM
i=1
yihi0≤ B, (2)
where hi0 is the channel gain from user i0s transmitter to the measurement point, and B > 0 is a pre-defined threshold. We assume that all the secondary users adopt a spread spectrum signaling format, in which the transmitted power is evenly spread across the entire available band. This allows efficient multiplexing of data streams from different sources corre- sponding to different applications, and reduces the combined power-bandwidth allocation problem to a received power allo- cation problem. Hence, the interference temperature constraint is translated to a total received power threshold B at the measuring point. The system model is shown in Fig. 2.
The model considers only a single measurement point. In practice it needs to be ensured that the interference tempera- ture is not exceeded at any of the primary receivers. A single measurement point cannot ensure this. But our model can be easily extended to scenario where multiple measuring points exist. Under this scenario multiple interference temperature constraints (Bs) should all be satisfied simultaneously. If a single measurement point is used, the interference temperature limit at this point must be set by a sufficiently large margin lower than the interference temperature actually tolerable
by the incumbent receivers. Depending on the propagation environment, the required margin can be 10s of dB.
III. CONVEXOPTIMIZATIONANDGEOMETRIC
PROGRAMMING
Convex optimization refers to minimizing a convex ob- jective function over convex constraint sets. We will use a particular type of convex optimization in the form of geometric program [10].
A monomial is a function f : <n→ <, where the domain contains all real vectors with non-negative components:
f (x) = cxa11xa22· · · xann, c ≥ 0 and ai∈ < (3) A posynomial is a sum of monomials
f (x) =X
k
ckxa11kxa22k· · · xannk (4) Posynomials are closed under addition, multiplication, and nonnegative scaling. Monomials are closed under multiplica- tion and division. If a posynomial is multiplied by a monomial, the result is a posynomial; similarly, a posynomial can be divided by a monomial, with the result a posynomial.
Geometric program is an optimization of the form minimize f0(x)
subject to fi(x) ≤ 1 hj(x) = 1
(5)
where f0 and fi are posynomials and hj are monomials.
Geometric programs are not (in general) convex optimization problems, but they can be transformed to convex problems by change of variables and a transformation of the objective and constraint functions. With a change of variables: yi = logxi
and bik= log cik, we can put it into convex form:
minimize ˜f0(y) = log(P
keaT0ky+b0k) subject to ˜fi(y) = log(P
keaTiky+bik) ≤ 0 h˜i(y) = aTjy + bj = 0
(6)
Since the functions ˜fi are convex, and ˜hi are affine, this problem is a convex optimization problem.
Convex optimization problems can be solved globally and efficiently through the interior point primal dual method [10], with polynomial running times that are O(√
N ) where N is the size of the problem.
IV. SOCIALOPTIMIZATION UNDERINTERFERENCE
TEMPERATURECONSTRAINT
Secondary user i0s valuation of the spectrum is character- ized by a utility vi(γi), where γi is the received SIR at user i0s receiver. Similar to the logarithm in Shannon’s formula, we define the logarithmic utility vi(γi) = ln(γi). This utility function captures user’s desire for higher data transmitting rate.
With energy consumption and QoS provision considerations, each secondary link has a minimum SIR constraint γti. Let Θ = {1, 2, ..., M }, be the set of transmitter and receiver link pairs, and let each transmitter’s available transmitting power be yi∈ (0, ymaxi ], ∀i.
We can formulate the social rate optimization problem with QoS and interference temperature constraints as follows;
maximizeP
ivi(γi) subject to
SIRi ≥ γit ∀i
P
ihi0yi ≤ B
yi > 0 ∀i
yi ≤ yimax ∀i.
(7)
Maximizing P
iln(γi) is equivalent to maximizing lnQ
iγi, which is then equivalent to minimizingQ
i 1
γi. Note that the objection function is posynomial. And the constraints can also be transformed into posynomial and monomial forms. So, this optimization problem is a convex optimization in geometric program form, and can be solved globally and efficiently.
We can define a normalized link gain matrix A with entries
hij
hii for i 6= j and 0 for i = j, and let H = γtA, the normalized noise vector η such that ηi =hn0
ii, and vector c with ci = hi0. Further we define ymax= (ymax1 , . . . , yMmax).
Theorem 1: If ρ(H) < 1, (I − H)−1γtη ≤ ymaxand (I − H)−1γtηc ≤ B, then there exists power vector y∗> 0, which satisfies the above described optimization problem.
Proof The dominant (largest) eigenvalue of the matrix H, denoted by ρ(H), is less than one (ρ(H) < 1) and (I − H)−1γtη ≤ ymax imply there exists a positive power vector
˜y = (I − H)−1γtη which satisfies the SIR bound and the maximum transmitting power constraints. If further ˜yc < B, each user can increase their power by a factor of B/P
iyihi0, which increases the SIR for every user hence will increase the objective function, so we can always find another ˜y ≤ y∗ ≤ ymax which will satisfy the SIR constraints, make the total received power constraint tighter and maximize the objective function. ¤
With the total received power constraint B at the measuring point and the QoS constraint, there are some cases when not all the secondary links can achieve their minimum SIR requirement, which raises the problem of system feasibility.
Theorem 1 gives the condition under which there will be a feasible power allocation for the secondary users. When the feasibility condition is not satisfied, only a subset of the secondary links can be accommodated. Then, depending on the goal of optimization, the spectrum accessing process will be different. If the goal is still to maximize the total utility which is proportional to the total transmitting rate, the strategy would be to exhaust all the possible active link combination, then by checking the feasibility condition and conducting optimization (7) to find the optimal feasible link set and the power allocation. An alternative accessing process that is more fair would be to maximize the number of active secondary links with QoS and interference temperature constraints, which was addressed in [17].
V. OPERATOR PROBLEM WITH DIFFERENT PRIORITY CLASSES
If different users have different contracts defining different priorities or throughput, than it is perfectly fair, then the operator should provide different throughput for such users.
In this case secondary links, depending on their willingness to pay belong to L priority classes. Let ai be the priority parameter for link i. The operator problem is then to maximize the network revenue.
[Operator Problem]:
maxP
i∈(i:xi=1)ai. subject to
SIRi ≥ γit ∀i ∈ (i : xi= 1) P
ihi0yi ≤ B ∀i ∈ (i : xi= 1)
yi > 0 ∀i ∈ (i : xi= 1)
yi ≤ ymaxi ∀i ∈ (i : xi= 1).
(8)
where xi is a collection of binary variables, xi = 1 means the ith link transmits otherwise xi = 0. By maximizing this revenue, secondary users who pay more will get accessing priority over those who pay less. We model the relation between the price pi a user paid and the priority parameter ai as follows,
ai= pαi (9)
where 0 ≤ α ≤ 1 is an operator designable parameter. Small α corresponds to putting more emphasis on system capacity (number of active secondary links), while large α corresponds to putting more emphasis on guaranteeing service to the user paying higher price. Specifically, α = 0 corresponds to the problem of maximize the number of active secondary links (capacity).
VI. OPTIMAL SUPPORTED LINK SUBSET SEARCHING
It can be proved that the operator problem is NP complete through the steps similar to [16]. In order to reduce the search space and hence reduce the complexity of searching for the optimal supported link subset, we first characterize some properties of the supported link subset.
We say that a power vector y supports all transmitters at a SIR target γt, if and only if
y > γt(Ay + η). (10) That is, each receiver i has a SIR γi> γit. To compute a power vector that satisfies (10) in a distributed fashion we describe a distributed constrained power control (DCPC) algorithm.
Suppose the power adjustment made by the ithterminal at the nthtime instant according to the DCPC is given by
yi(n) = min{ymax, γt yγi(n−1)
i(n−1)} = min{ymax, γt(ηi+P
j∈Θyj(n − 1)hhji
ii)}, 1 ≤ i ≤ M, (11) It has been shown in [15], that for any given γt, DCPC converges to a unique positive power vector determined by the fixed point solution to
y = min{ymax, γt(Ay + η)}. (12) A power vector y which satisfies the fixed point equations in (12), will be referred to as the stationary power vector. When all transmitters can be supported, the DCPC converge to the fixed point solution given by
y = γt(Ay + η). (13)
For every subset of transmitters Θ0⊆ Θ, let yΘ0denote the stationary power vector of a system which consists only of the set Θ0. Also, let SΘ0 be the subset of transmitters which are supported (at γt) under the stationary power vector yΘ0 (i.e., in a system where DCPC runs only with the set of transmitters Θ0). Also let ¯S denote the complement of set S. And let
yΘ/Θi 0 =
½ yiΘ, if i ∈ Θ0
0, otherwise (14)
Theorem 2: For a link set Θ0 ⊆ Θ, if the secondary link system with DCPC consisting of link set Θ0 is infeasible, the system consisting of set Θ will also be infeasible.
Proof Consider two cases:
1) i ∈ Θ0 be in the non-supported set ¯SΘ0.
2) ¯SPΘ0 = Φ, i.e., all the links in Θ0 are supported, but
i∈Θ0hi0yiΘ0 > B.
For case 1)
yi= ymax, ∀i ∈ ¯SΘ0
Thus, from the fact that the hij’s are non-negative and Lemma 2 in [16] which states that yΘ0 ≤ yΘ/Θ0 (comes from the fact that using DCPC with an initial power vector yΘ/Θ0, results in a non-increasing powers sequence which converges to yΘ0).
yΘi ≤ ymax= yΘi 0 < γt(ηi+ X
j∈Θ0
hji
hiiyjΘ0)
≤ γt(ηi+X
j∈Θ
hji
hiiyjΘ) (15) Thus, i is also in the non-supported set ¯SΘ.
For case 2) Because we have yΘ0 ≤ yΘ/Θ0 so X
i∈Θ
hi0yiΘ> X
i∈Θ0
hi0yiΘ0> B,
i.e., the interference temperature bound B is violated. ¤ Theorem 2 establishes that the tree pruning algorithm [18]
which will be described following is valid for our scenario.
Thus the search for optimal supported subset of links can be confined to a smaller searching space. This optimization needs all the system information including all the link gains and the number of secondary links to conduct the calculation.
Therefore a centralized controller is needed to coordinate the access process.
The tree pruning algorithm is described below in Algorithm 1, where each secondary link is identified by a unique number, M is the total number of secondary link pairs. In the Satisf y procedure, both the interference temperature and the SIR constraints are examined. While set Ff inal contains all the candidate sub-link sets. Then the operator problem described in (8) is transferred to the following problem,
arg max
k
X
i∈Ff inal(k)
ai. (16)
An instance of the tree pruning algorithm on a seven edge (seven secondary link pairs) example network (Fig. 3) is given in Fig. 4. Initially every edge in the family is allocated a unique subset. Thus the initial family comprises of as many subsets as the number of edges. For the given example, the initial
Algorithm 1 T reeP runing
1: i = 1
2: Fi = {{1} {2} ... {M }}
3: Fi+1 = Extend-Family(Fi)
4: while Fi+16= φ do
5: ∀fi ∈ Fi; Fif inal= {fi: fi6⊆ Fi+1(k), ∀k}
6: i = i + 1
7: Fi+1 = Extend-Family(Fi)
8: end while
9: Ff inal=S
iFif inal
procedure Fext= Extend-Family(Forig)
1: Fext= {φ}
2: for i = 1 : length(Forig) do
3: for j = Forig(i, |Forig(i)|) + 1 : M do
4: if Satisf y({Forig(i)j}) then
5: Fext= Fext∪ {Forig(i)j}
6: end if
7: end for
8: end for
Fig. 3. Example system with seven secondary links.
family is { {1} {2} {3} {4} {5} {6} {7} }. In the next and the following rounds two fundamental operations take place.
The first operation is expansion, where each of the subsets in the family is expanded by including one more unique edge.
The uniqueness is in the sense that only edges that have no vertex in common with any of the other edges of the set to be extended are added. This is because it is assumed that a user cannot transmit to nor receive from two other users at the same time. This issue is included in the Satisfy() routine. For the given example, the family after first expansion is { {1, 3}
{1, 4} {1, 5} {2, 4} {2, 5} {2, 6} {3, 5} {3, 6} {3, 7} {4, 6}
{6, 7} {1} {2} {3} {4} {5} {6} {7} } The second operation is pruning, where the subsets which are in-turn subsets of larger sets are eliminated from the family. For the given example, the family after first pruning is { {1, 3} {1, 4} {1, 5} {2, 4}
{2, 5} {2, 6} {3, 5} {3, 6} {3, 7} {4, 6} {6, 7} }. These two operations are performed iteratively until expansion stops. For the given example, the final family is { {1, 3, 5} {1, 4} {2, 5}
{2, 4, 6} {3, 5} {3, 6, 7} {4, 6} {5} {6, 7} {7} }.
VII. POTENTIALGAMES ANDSECONDARYSPECTRUM
SHARING
The optimal (centralized) search algorithm described in the previous section gives us the best performance. However, the nature of secondary spectrum sharing is temporary and dis- tributed, a practical secondary spectrum sharing scheme must be distributed. In this section, we develop such a distributed algorithm to solve the operator problem discussed before.
Fig. 4. Instance of tree pruning algorithm on a seven secondary links example system.
This distributed process is composed of two phases. The coordination phase controls the optimal set of active secondary links which can access the spectrum, and the power control phase is to allocate transmit power to support the minimum target link SIR γti given the set of active links.
A. Distributed Power Control
Power control plays an important role in dynamic spectrum sharing. Here, secondary nodes maintain their power levels so that the sum of the interference caused by them in a band is below the interference threshold. This allows un- derlay transmission thereby maintaining co-existence between primary and secondary users. Clearly, power control in the dynamic spectrum access scenarios has to be distributive in nature. Power control reduces the transmitted power and hence the power spectrum density at the measuring point.
This improves the efficiency of the spectrum sharing. Besides, power control also reduces the internal interference of the secondary spectrum sharing networks.
When there are M active links, we use the standard DCPC (11) to allocate the transmitting power. This DCPC will make the received SIR converge to the target SIR γit distributively except for cases where maximum transmitting power yimax is reached.
B. Potential Games
Suppose there are M transmitter and receiver link pairs competing for the secondary spectrum access opportunities.
Let k be a time (iteration) counter and N (x(k)) be the aggregate received power at the measuring point at time k given by,
N (x(k)) = XM
i=1
yi(k)hi0xi(k), (17)
In order to maximize the network revenue while keeping the aggregated received power at the measuring point under the interference threshold, we define the utility function ui(x) shown in Fig. 5 for each link pair as follows,
ui(x) =
P
j∈(j:xj =1)aj
3P
jaj +23, N (x(k)) < B, minj∈(j:xj=1)γj > γt
B
3N (x(k))+13, N (x(k)) ≥ B, minj∈(j:xj=1)γj > γt
minj∈(j:xj =1)γj
3γt , minj∈(j:xj=1)γj< γt
(18)
0 1/3 2/3 1
u(x)
0 1/3 2/3 1
N(x)
u(x)
0 1/3 2/3 1
u(x)
mini ∈ (i: x
i=1)γt γi sumi ∈ (i: x
i=1) a
i sumi ai
Fig. 5. Utility function ui(N (x)).
In the secondary spectrum sharing game each user max- imizes its utility function ui(x(k)) by its choice of being active or not. By maximizing this utility function, the system will reach an operating point where the network revenue is maximized while satisfying QoS and interference temperature constraints. To emphasize that the ith user has control only over its own choice, we use an alternative notation ui(xi, x−i), where x−i denotes that vector consisting of elements of x other than the ith element. And after each changing of the active link set, the DCPC will be activated to allocate the transmitting power.
Proposition 1: The secondary spectrum sharing game is a potential game and has a pure strategy (deterministic) equilibrium.
This proposition comes from the fact that we can define a potential function Φ = u(x) which satisfies 4Φ = 4ui(x).
Hence, we can start from an arbitrary deterministic strategy vector x, and at each step one player increases it’s utility. That means, that at each step Φ is increased identically. Since Φ can accept a finite amount of values, it will eventually reach a local maxima. At this point, no player can achieve any improvement, and we reach a Nash equilibrium (NE). A practical method to achieve the NE would be to use sequential play where each player maximizes its own utility function sequentially while other players’ strategies are fixed.
To achieve this sequential play, a simple random access scheme similar to [22] can be introduced where each user make the update with probability Pa = 1/N . More specif- ically, at the beginning of each time slot, each user flips a coin with probability Pa, and, if successful, makes a new decision based on the current values for the utility function value; otherwise, the user takes no new action. This scheme ensures that on average exactly one user makes decisions at a time, but of course has a nonzero probability that two or more users take actions simultaneously. But as reported in [22], this will not destroy the potential function’s upward monotonic trend. Of course, one way to reduce the collision would be to decrease Pa, but this will also decrease the convergence speed.
Theorem 3: The sequential play will never converge to a solution where the total received power at the measuring point exceeds the interference temperature bound.
Proof Suppose x0is a Nash equilibrium solution of the game, and suppose at this point N (x0) > B. Then by the definition of the utility function (18), we know that one of the link pairs with xi = 1 can always increase its payoff ui(xi, x0−i) by changing its strategy to xi= 0, hence, this point x0can never be a Nash equilibrium. We know that the sequential play will never converge to a point which is not a Nash equilibrium.
Hence, the above theorem. ¤
Theorem 4: The sequential play will always converge to a solution where all the active links are supported with their target SIR.
Proof Similar to the previous proof, suppose x0 is a Nash equilibrium solution of the game, and at this point let γi< γt. Then by the definition of the utility function (18), the link j = arg minj∈(j:xj=1)γj can always increase its payoff by changing its strategy to xj= 0, hence this point x0 can never be a Nash equilibrium. ¤
To characterize the efficiency of the Nash equilibrium point achieved by the sequential play, let xobe the Nash equilibrium strategy profile. This point has the property that either
j∈(j :xminoj=0)N (xj = 1, xo−j) > B, (19) which means, at the Nash equilibrium point, if any single secondary link j with xj = 0 changes its choice to xj = 1 unilaterally, the total received power at the measuring point would exceeds the interference temperature threshold B, or if adding one more link, some of the active links will not achieve their target SIR. We note that multiple Nash equilibria may exist in this game. Finally, sequential play does not allow asynchronous updates by individual users. This may cause signaling and other overhead. To overcome this issue we consider a stochastic learning based distributed solution that is described next.
C. Learning Automata
Stochastic learning technique has been successfully used in wireless packet networks for on-line prediction, tracking and discrete power control [19] [20]. It is shown to be computa- tionally simple and efficient. Learning algorithm determines probabilistic strategies for players by considering the history of play. The probability updating algorithm used by each of the user is as given below:
1) Set the initial probability pi(0).
2) At every time step k, the ithuser chooses xi(k) = 1 or 0 (to transmit or not) according to its action probability pi. 3) After distributed power control (DCPC) phase, each player obtains a feedback ui(x(k)) based on the set of all actions x.
4) Each player (i) updates its action probability according to the rule:
pi(k + 1) = pi(k) + bui(k)(1 − pi(k)) x(k) = 1 pi(k + 1) = pi(k) − bui(k)pi(k) x(k) = 0,
(20)
where 0 < b < 1 is the step size, and ui(k) is utility function which lies in the interval (0, 1).
5) If pi converges, stop. Otherwise, go to step 2).
The probabilistic update used in (20) is a stochastic learning automata updating known as linear reward-inaction (LR−I) [12].
To characterize the proposed learning automata algorithm, we first define the expected payoff for player i as gigiven by,
gi(p1, ..., pM) = E[ui|jth player employs strategy pj, 1 ≤ j ≤ M ] = X
x1,...,xM
ui(x1, ..., xM) YM
s=1
psxs, (21)
where psxs = ps if xs= 1 and psxs = 1 − ps if xs= 0.
Definition 1 The N-tuple of strategies (po1, ..., poM) is said to be a Nash equilibrium, if for each i, 1 ≤ i ≤ M , we have
gi(po1, ..., poi−1, poi, poi+1, ..., poM)
≥ gi(po1, ..., poi−1, pi, poi+1, ..., poM) ∀pi∈ [0, 1]. (22) In general, each poi above will be a mixed strategy and we refer to (po1, ..., poM) satisfying (22) as a Nash equilibrium in mixed strategies. A Nash equilibrium is said to be in pure strategies if (po1, ..., poM) is a Nash equilibrium with each poi being either 0 or 1. Nash equilibrium is a profile of strategies such that each player strategy is an optimal response to the other players’ strategies.
Theorem 5: The proposed learning automata algorithm con- verges to one of the Nash equilibria of the game.
Proof For any choice of pure strategies, payoffs are the same for all players (users), i.e.,
ui(x) = uj(x), ∀i, j ∈ M, ∀x ∈ {0, 1}M
So we identify this game as a coordination game where at least one Nash equilibrium in pure strategies exists. And further, at least one Nash equilibrium must be Pareto efficient. When considering a game with common payoff, under the LR−I learning algorithm, the automata team converges to one of the Nash equilibria [13]. ¤
Theorem 6: The proposed learning automata algorithm will never converge to a point where the total received power at the measuring point exceeds the interference temperature bound.
Theorem 7: The proposed learning automata algorithm will always converge to a point where all the active links are supported with their target SIR.
These two theorems can be proved through the same argument as Theorem 3 and Theorem 4 with the fact that it is known that the stochastic learning algorithm with common payoff games always converges to a pure strategy rather than to a mixed strategy [13]. So we will only consider pure equilibria.
The learning automata algorithm needs less information and control signalling to operate than the sequential play.
The sequential play requires that each player updates its strategy one by one. And at the time of decision, in order to compute the utility function, the information required includes all active users’ SIRs, priorities, and the current interference temperature. Significant control signalling is also required to accomplish this process distributively. An alternative choice would be to run this sequential play at a central controller.
For the learning automata, asynchronous updating is permitted.
The only information needed is a feedback of the current utility function value. To compute this utility function, each active user should report its SIR and priority parameter to the measurement point which acting also as a central controller.
But the tradeoff is that learning automata converges much slower than the sequential play. The information needed by these algorithms can be distributed through a common control channel which has been assumed in many similar literatures in this area.
One assumption in this paper is that the interference tem- perature remains constant during the secondary use of the spectrum. With mobile users, the interference temperature may vary. One solution to account this variance would be that the secondary access algorithm is triggered periodically. A short period would ensure that when the interference temperature is violated, the system would recover quickly.
VIII. NUMERICALRESULTS
In this section, we first present some numerical examples for a simple secondary sharing system with only three transmitting and receiving pairs. The target SIR is selected to be γt= 12.5, and the noise power is σ2= 5 × 10−13, which approximately corresponds to the thermal noise power for a bandwidth of 1 MHz. We consider low rate data users, using a spreading gain of L = 128. Path gains are obtained using the simple path loss model hj= K/d4j where K = 0.097. This gives the following gain matrix:
H0= 10−7
0.0097 0.1552 0.0148 0.0019 0.0034 0.0066 0.0748 0.0237 0.0307
(23)
When the interference temperature bound and noise ratio is B/σ2 = 200 (We use these ratios only to illustrate clearly how the system works. Of course we can use lower ratio by reducing the target SIR), this three secondary link pair system is feasible. Using the geometric programming optimization method, we find the maximum aggregate utility is 8.3567, with γ1 = 12.5 , γ2 = 12.5, γ3 = 27.2541, y1 = 0.0164W, y2 = 0.0988W, and y3 = 0.0107W for each link. When B/σ2 = 60, the social optimization is infeasible, but we can resort to our proposed potential game. Under equal priority case a = [1, 1, 1], when using sequential play, after convergence, only link subsets {1, 3} or {2, 3} can coexist with γ = 12.5. When the learning automata algorithm is used, the evolution of the total received power N (x) is shown in Fig.
6. As discussed, we see that the total received power converges to a value below the interference temperature threshold B.
The evolution of the choice probability p is shown in Fig. 7.
After convergence, only links 2 and 3 are active with equal probability initialization. The optimal link subset searching results in the same optimal link subset {1, 3} and {2, 3}. It can be observed in Fig. 6 that during the settling phase, N (x) exceeds B. So, depending on the application, there should be some predefined extra margin to accommodate the fluctuation during the settling phase. But this may reduce the capacity of the secondary system.
0 200 400 600 800 1000 0
2 4 6 8x 10−11
time (iteration)
Total Received Power (N(x))
N(x) B
Fig. 6. Evolving of the total received power (N (x)) over time for learning automata algorithm.
0 200 400 600 800 1000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
time (iteration)
Probability (P)
P1 P2 P3
Fig. 7. Choice probability p of the activation strategy over time for learning automata algorithm.
The proposed optimal supported link subset searching algo- rithm can significantly reduce the searching space as shown in Fig. 8. Each point in the curve for the reduced complexity search algorithm (tree pruning) results from an averaging over 10 random secondary link geometric distributions with M link pairs, B/σ2= 60 and γt= 12.5. The naive exhaustive search algorithm needs to check 2M subsets to find the optimal sup- ported secondary link set, while it can be seen that the search complexity for the reduced complexity search in this typical scenario is bounded by M4, which is polynomial complexity.
Obvious in the worst case when both the target SIR and the interference temperature bounds are absent when the reduced complexity search is similar to exhaustive searching.
Despite the sub-optimal nature of the sequential play algo- rithm, the convergence speed is dramatically reduced as shown in Fig. 9. It can be seen that even with 35 total secondary link pairs, the sequential play converges within 100 iterations.
As we have previously mentioned, the actual utility results after convergence depend on the initial starting point for the sequential play. In Fig. 10, we illustrate the variation in the utility obtained with various initializations (100 trials are
5 10 15 20 25 30 35
100 102 104 106 108 1010 1012
Number of link pairs (M)
Number of checking
Exhaustive Search Reduced Complexity Search M4
Fig. 8. The reduced complexity search algorithm (tree pruning) v.s. the naive exhaustive search
5 10 15 20 25 30 35
10 20 30 40 50 60 70 80 90 100
Number of total secondary link pairs
Number of total iterations
Fig. 9. Convergence of the sequential play
considered) for an secondary spectrum sharing scenario with 10 link pairs, B/σ2 = 960 and γt = 12.5. The priority vector is set to be a = [10, 10, 5, 5, 5, 1, 1, 1, 1, 1]. We can see that considerable utility improvements can be achieved if the algorithm is run repeatedly with different initializations and the best configuration is selected.
Fig. 11 depicts the performance of the sequential play results with respect to the optimal subset searching results.
Each point in the sequential play solution curve represents an averaging over 10 trials with an secondary spectrum sharing scenario with 10 link pairs, B/σ2= 960 and γt= 12.5. The priority vector is set to be the same as before. Even though the sequential play converges to local optimal solutions, it can be seen from Fig. 11 that as we increase the number of different initialization, the sequential play converges to the optimal solution.
The capacity of the network is evaluated as the total number of active link pairs in our context. When utility function is concerned, to maximize the capacity corresponds to setting α = 0 in the utility function, where 0 ≤ α ≤ 1 is an operator designable parameter. Small α corresponds to putting more emphasis on system capacity (number of active
0 20 40 60 80 100 0.72
0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.8 0.81
number of initialization
utility
umax
Fig. 10. utility for different initializations of the sequential play
0 5 10 15 20
0.77 0.775 0.78 0.785 0.79 0.795 0.8 0.805 0.81 0.815
number of initializations
utility
sequential play optimal search
Fig. 11. Optimal subset searching v.s. sequential play with different initializations
secondary links), while large α corresponds to putting more emphasis on guaranteeing service to the user paying higher price. Specifically, with α = 0, we can get similar graph as Fig. 11 with a=[1,1,1,1,1,1,1,1,1,1]. It can be observed that as the number of the initialization increases, the sequential play converges to the optimal capacity solution.
The converged solution of the sequential play algorithm depends on the initialization which is chosen from the two strategies: active or inactive with equal probability when all the links have equal priority. Hence, on average, when not all the secondary links can be active at the same time, under a long run, they will have the same chance to be active under the condition that they are in the Nash equilibria solution set.
This property provisions the sequential play with fairness.
Further, we compared our proposed sequential play algo- rithm with a Transmit Power Control (TPC) scheme that limits the aggregate interference to the transceiver presented in [5]. In the TPC scheme, the primary transceiver system can control the interference from the Cognitive Radio (CR) devices by broadcasting a beacon signal, containing a parameter α (relevant to the interference it can tolerate). The resulting inter- ference levels at the primary receiver caused by the individual
1 2 3 4 5 6 7 8 9 10
0 5 10 15
link pair index
SIR
1 2 3 4 5 6 7 8 9 10
0 20 40 60 80
link pair index
SIR
Fig. 12. Sequential play v.s. TPC scheme
CR devices are equal. This scheme ([5]) did not consider the QoS issues, therefore resulting in a degraded performance.
As in Fig. 12, there are 10 secondary CR link pairs. The target SIR is 12.5. These links are randomly deployed. As can be observed, under the TPC scheme, only one link pair will achieve the target SIR, while under our proposed scheme, there are 5 links which achieved the target SIR. Hence, under the interference temperature constraint, our proposed scheme efficiently utilized the spectrum for data transmission.
IX. CONCLUSIONS
We have considered spectrum sharing among a group of spread spectrum users with a constraint on the total interfer- ence temperature at a particular measurement point, and a QoS constraint for each secondary link. A social optimization set- up of this problem is formulated which is solved efficiently by using a geometric programming method. There are cases, when this system with all secondary links active will be infeasible. A reduced complexity optimal link subset searching is introduced to find feasible subsets of links, which can significantly reduce the searching space compared with naive searching. Then we define the secondary spectrum sharing problem as a potential game which takes different priority classes into consideration. Firstly, this game is solved through sequential play. The sequential play is shown to converge to the Nash equilibria with acceptable speed but with relatively significant controlling signal and operating information. Then a learning automata algorithm is introduced which only re- quires a feedback of the utility value to converge to the Nash equilibria. But the drawback is the low convergence speed. The achieved Nash equilibrium is characterized to be a point with a good tradeoff between the efficiency and the complexity.
ACKNOWLEDGMENT
This work is supported in part by the grant NSF CAREER 0133761. The authors would like to thank the anonymous reviewers for their constructive comments which led to a significant improvement of the manuscript.
REFERENCES
[1] http://www.sharedspectrum.com/?section=measurements
[2] Kolodziejski and J. Betz, “Detection of Weak Random Signals in IDD Non-Gaussian Noise,” IEEE Trans. On Comm., Vol 48, No. 2, February 2000.
[3] Sai Shankar N, Carlos Cordeiro, and Kiran Challapali, “Spectrum Agile Radios: Utilization and Sensing Architectures,” IEEE DySPAN 2005.
[4] Federal Communication Commission, “Spectrum Policy Task Force,”
Rep. ET Docket no. 02-135, Nov. 2002.
[5] Jaap C. Haartsen, Lasse Wieweg, Jorg Huschke, “Spectrum Management and Radio Resource Management Considering Cognitive Radio Systems,”
XXVIIIth URSI General Assembly Oct 2005.
[6] Simon Haykin, “Cognitive Radio: Brain-Emprowered Wireless Commu- nications.” IEEE JSAC. vol. 23, NO.2, Feb 2005.
[7] Jon M. Peha and Sooksan Panichpapiboon, “Real-Time Secondary Mar- kets for Spectrum.” Telecommunications Policy, Volume 28, Numbers 7-8, August 2004, pp. 603-618.
[8] Arnon Tonmukayakul and Martin B. H. Weiss, “Secondary Use of Radio Spectrum: A Feasibility Analysis.” TPRC, Oct 2004.
[9] J. Huang, R. Berry and M. L. Honig, “Auction-based Spectrum Sharing,”
to appear in ACM/Springer Mobile Networks and Applications Journal (MONET)
[10] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.
: Cambridge Univ. Press, 2004
[11] D. Julian, M. Chiang, D. O’Neill, and S. Boyd, “QoS and fairness con- strained convex optimization of resource allocation for wireless cellular and ad hoc networks.” IEEE INFOCOM, pp. 477-486, New York, USA, June 2002.
[12] K.S. Narendra and M.A.L. Thathachar, “Learning Automata: An Intro- duction,” Englewood Cliffs: Prentice Hall, 1989
[13] P.S Sastry, V.V. Phansalkar, and M.A.L. Thathachar, “Decentralized Learning of Nash Equilibria in Multi-Person Stochastic Games With Incomplete Information,” IEEE Trans Systems, Man, and Cybernetics., vol. 24. pp. 769-777, May 1994
[14] D. Fudenberg and J.Tirole, Game Theory. Cambridge, MA: MIT Press, 1991.
[15] S.A. Grandhi, J.Zander and R. Yates, “Constrained power control.”
Wireless Personal Commun. 1(4),1995
[16] M.Andersin, Z.Rosberg, and J.Zander, “Gradual removals in cellular PCS with constrained power control and noise.” Wireless Networks, 2(1):27-43, 1996.
[17] Yiping Xing and R. Chandramouli, “QoS Constrained Secondary Spec- trum Sharing.” IEEE Dynamic Spectrum Access Networks (DySPAN).
Baltimore, 2005.
[18] Chetan Nanjunda Mathur, M. A. Haleem, R. Chandramouli and K.P.
Subbalakshmi, “Complexity of Channel Assignment Problems with Con- straints in Multi-hop Wireless Networks.” technical report, 2005.
[19] S. Kiran and R. Chandramouli, “An adaptive energy efficient link layer protocol using stochastic learning control.” Proc. IEEE Intl. Conf. on Communications (ICC)., Alaska, 2003
[20] Yiping Xing and R. Chandramouli,“Distributed discrete power control for bursty transmissions over wireless data networks.” proc. IEEE Intl.
Conf. on Communications (ICC)., Paris, 2004
[21] P. Kolodzy, “Spectrum Policy Task Force: Findings and Recommen- dations,” in International Symposium on Advanced Radio Technologies (ISART), March 2003
[22] N. Nie and C. Comaniciu, “Adaptive Channel Allocation Spectrum Etiquette for Cognitive Radio Networks”, Proceedings of IEEE DySPAN, November 2005.
PLACE PHOTO HERE
Yiping Xing (S’03) received the B.S degree from the University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2001, and the M.E. degree from the Stevens Institute of Technol- ogy, Hoboken, NJ, in 2004, both in electrical engi- neering. He is currently working toward the Ph.D.
degree in the Department of Electrical and Computer Engineering at Stevens Institute of Technology.
His current research interests include radio re- source management for cellular and ad hoc net- works, access control for cognitive radios, and game theory for wireless networks. He received the Outstanding Research Award in 2005 and the Graduate Fellowship Award in 2006 from Stevens Institute of Technology. He is also the recipient of the IEEE CCNC 2006 best student paper award.
PLACE PHOTO HERE
Chetan Nanjunda Mathur is currently pursuing his Ph.D. in Computer Engineering at Stevens Institute of Technology, New Jersey, USA. He was born in Bangalore, India in 1981. He received his BE degree in Computer Science from Visveshwaraiah Institute of Technology, Bangalore, India in 2002. He has an MS in Computer Engineering from Stevens In- stitute of Technology, New Jersey, USA. Part of his MS thesis was patented by Stevens Institute of Technology. In the past few years he has published several research papers in the fields of Cryptography, Coding theory and Dynamic spectrum access. He has also received numerous awards including the IEEE best student paper award presented at IEEE Consumer Communications and Networking Conference (CCNC 2006) and the IEEE student travel grant award presented at International Conference on Communications (ICC 2005). He is an active student member of IEEE and is in the advisory board of Tau Beta Pi, the national organization of engineering excellence.
PLACE PHOTO HERE
M. A. Haleem (S’94-M’03) is currently a postdoc- toral researcher at the Department of Electrical and Computer Engineering, Stevens Institute of Technol- ogy, Hoboken, NJ. He received the B.Sc. Eng. de- gree with specialization in Electrical and Electronic Engineering from University of Peradeniya, Kandy, Sri Lanka, in 1990, the M.Phil. degree in Electrical and Electronic Engineering from Hong Kong Uni- versity of Science and Technology, Hong Kong in 1995, and Ph.D. degree in Electrical and Computer Engineering from Stevens Institute of Technology, Hoboken, NJ in 2005.
Dr. Haleem has been with the Wireless Communications Research Depart- ment, Bell Laboratories, Lucent Technologies Inc., Crawford Hill, Holmdel, NJ, from 1996 to 2002 as a consultant and a Member of Technical Staff.
He has been with the Department of Electrical and Electronic Engineering, University of Peradeniya, Sri Lanka, from 1990 to 1993 and held the position of Lecturer.
