以賽局理論為基礎的無線網路資源管理機制

(1)

國立臺灣大學電機資訊學院電信工程學研究所博士論文

Graduate Institute of Communication Engineering College of Electrical Engineering and Computer Science

National Taiwan University Doctoral Dissertation

以賽局理論為基礎的無線網路資源管理機制

Game-Theoretic Resource Management in Wireless Networks

王志宇 Chih-Yu Wang

指導教授：魏宏宇博士和劉國瑞博士

Advisor: Hung-Yu Wei, Ph.D. and K.J. Ray Liu, Ph.D.

中華民國 102 年 6 月

June, 2013

(2)

(3)

(4)

(5)

謝

在深夜的冷氣房裡，試著總結這六年的生活。博士生沒有上下班打卡的壓力，換句話說就是 24 小時都可能處於工作中的狀況。配上自己總愛在宿舍獨自埋頭苦思寫論文的習慣，自然變得比較難維持規律的生活。但總體來說，這樣的博士生生涯還是挺合我胃口的。心境潮起潮落，研究有困境也有突破，但每段時間後總是能作出「啊，好像確實有做了點什麼呢」這樣的小結。在精華的歲月裡能控制自己的時間，

做些自己喜歡的事，和喜歡的人交流，還有什麼不滿意的呢？

第一個要感謝的是我的指導教授魏宏宇老師。魏老師是我接觸過對研究最有熱情的人之一，搭配上他的博學多聞以及靈活的想法，幾乎沒有老師無法掌握的研究主題。在這段接受魏老師指導的時間裡，我學習了研究的基本態度、技術和執行的方式，也不停的揣摩著老師為人處事的方式。魏老師不論在生活或研究上都給予了我相當大的自由度。最開始跌跌撞撞的幾年想必讓老師十分擔憂，但老師的教誨我一字一句的記在心裡寫在腦海裡，並在後來的幾年裡努力地去遵循實現。

這些教誨，也將會是我一生的準則。

劉國瑞老師在我於馬里蘭大學的一年短期研究時擔任我的指導教授，劉老師的威嚴與充滿前膽性和說服力的言語讓我對研究的前景有更深的體悟，其所帶領維持的美國學術研究態度及鼓勵實驗室同學間進行合作的風氣讓我體會到其他研究生涯的可能性。感謝劉老師的指導，讓我在這一年裡開拓了視野，了解自己的不足，並在研究上獲得了扎實的礪練。

此外，在博士論文口試時，感謝陳文村、張仲儒、王蒞君、黃經堯、

廖婉君和張時中老師等口試委員給予我的指導和建議，使我能更加完善我的博士論文，並在未來的研究上能有更正確踏實的方向和基礎。

在研究的路上，我很幸運能有優秀的同伴們扶持，不論是台灣大學的無線行動網路實驗室、馬里蘭大學的 Signals and Information Group、

或中研院資訊所的同事們，你們優秀的學術能力是我努力學習的對象，

雖然我的個性並不是常駐在實驗室的類型，但你們仍不吝接受我，並願意指導我、合作及互相打氣。沒有你們，我無法順利完成這個階段。

最後，我要感謝我的父母給予的栽培，沒有你們的愛、關懷和支持，就沒有現在的我。感謝我的弟弟，沒有你的陪伴我的一生會少了很多歡笑。感謝素貞妳的陪伴，遇見妳，是我在博士生涯裡最美好的一件事。

(6)

(7)

中文要

次世代無線廣域網路 (WWAN) 通訊標準，如 IEEE 制定的 802.16 (WiMAX) 和 3GPP 制定的 LTE-Advanced 標準，由於其承諾將達成由 ITU-R 所規畫的 4G 無線網路願境而在近期獲得眾多的關注。然而，在無線網路可用資源日趨虧乏的情況下，這些通訊標準採用了大量新技術以提升資源的使用效率。然而，在現實的各種限制下，這些新技術依然要面臨以有限的計算資源和處理時間內進行資源分配最佳化的挑戰。

無線網路裡的資源管理問題的困難點在於他們牽扯到參與者的自私行為與其之間的競爭關係。在無線網路的資源分配問題裡，由於資源有限，當有一參與者 (如手機、平板和基地台等) 取得較多資源時，其他參與者往往會失去部分資源。此特性使其問題自然的出現了競爭行為。當參與者受控或本身即為使用者時，我們可假定此類參與者為理性行為者，而理性行為者在競爭環境下往往會表現出自私的行為。此類行為和傳統處理資源分配問題的最佳化解法下假定所有參與者皆為了某共同目標而遵守系統指令相違背。若放任此類自私行為不管，傳統解法往往會面臨參與者行為不符預期且系統效能不彰的問題。近年來，將賽局理論應用於無線網路資源分配理論的相關研究日漸熱門。

賽局理論是一套用來分析玩家之間的互動 (如競爭行為) 的數學理論工具。我們可以從賽局理論的角度來看待傳統的資源分配管理問題，並藉由賽局理論的分析和理論基礎提出嶄新的解決方案。此類方案應保持合理的效能、實作性，並能夠對自私行為進行有效的控管。

在本論文中，我們將探討異質網路、裝置對裝置通訊、以及群體廣播等最新的網路系統，將針對各個系統裡的資源管理問題，提出新穎的以賽局理論為基礎的方法論。我們所探討的系統都存在明顯的競爭環境，並在我們的分析中發現了有害的自私行為。因此，我們根據分析的結果和各個系統的特性，分別提出了可以有效管制自私行為的新穎解決方案。我們透過理論證明或模擬實驗的方式，驗證了這些解決方案在管制自私行為的同時，也保證了合理的系統效能和實作性。

在異質網路中，我們首先探討了微型基地台的涵蓋範圍問題，我們首先觀察到在此類問題中，微型基地台從自私使用者所收集到的資訊可能會有謊報而無法反映系統現況的問題。針對此類自私行為，我們提出了以投票機制為基礎的賽局機制設計，藉此確保自私使用者會選擇誠實回報他們所觀察到的系統狀況。接下來，我們分析在異質網路裡的微型基地台定價問題。我們證明了藉由最佳的差異化合約設計和微型基地台之間的搭配，我們可以有效地提升無線網路系統的服務品質，同時系統服務商的利潤也有顯著的提升。最後，我們探討頻帶集成的最佳規畫與設定問題。我們注意到在此類問題中，基地台一樣需要從使用者收集其心目中的服務質量需求，而自私的使用者一樣有可

(8)

能會藉由謊報而獲得不正當的利益。針對此問題，我們提出了以拍賣機制為基礎的設計，並以理論證明了自私使用者在此機制下會誠實地回報他們的服務質量需求。

在裝置對裝置通訊系統中，我們觀察到此類系統的點對點傳輸特性讓使用者更容易謊報他們所觀察到的資訊。同時，我們也注意到此類系統的傳輸品質可以藉由簡單的資源交換機制獲得提升。從這兩點出發，我們提出了以交換機制為基礎的賽局機制設計。此設計的運算複雜度低，達到的資源分配結果滿足柏拉圖最適，同時也保證了使用者會誠實地向基地台回報他們所觀察到的資訊。

最後，在群體廣播系統裡，我們首先討論了一個較抽象的社會學習與網路外部性問題。我們提出了一個新的賽局模型：中國餐廳賽局以處理這個問題。藉由分析這個賽局，我們可以預期自私使用者在有網路外部性的網路裡的最佳決策。從這個賽局模型出發，我們探討了在可伸縮編碼群體廣播系統裡的使用者影片訂閱問題。我們認為此類問題其實是一個網路裡的決策問題，並可以使用我們的中國餐廳賽局來分析。以此為出發點，我們提出了一個多維馬爾可夫決策過程來描述此系統的長期效能演進。我們的分析顯示，當我們在系統套用最佳定價時，我們不只最大化系統服務商的利潤，同時也提升了整個系統的社會福利。

關字：無線網路、資源管理、賽局理論、異質網路、裝置對裝置通訊、群體廣播、自私行為

(9)

Abstract

Next-generation wireless wide-area network (WWAN) standards, such as IEEE 802.16 and 3GPP LTE-Advanced, raised lots of attentions in these days since they are established for achieving the 4G standard requirements proposed by ITU-R [1], which illustrates a colorful vision for the future wireless communication. Nevertheless, the resource for wireless networking, such as spectrum, is very limited and difficult to expand. In order to fulfill this vision with limited resource, these new wireless standards introduce numerous advanced techniques to increase the resource utilization efficiency. Most challenges in these techniques can be transformed into resource allocation problems under the resource limitation constraints.

The resource management problems in wireless networks are difficult since they involve complex competitions and selfish behaviors from participants in the wireless networks. An operation that increases the participant's allocated resource inevitably reduces the resource for other participants, which results in competitions. Additionally, when the participants are or controlled by real humans, we can fairly assume that these participants are rational and therefore selfish. The competition effect and selfishness of participants in wireless networks imposes an serious threat to all existing solutions which are based on traditional perspectives, which usually inherent an assumption that all participants faithfully follow the orders of the system for some global objective. In recent years, researchers discovered that game theory is suitable for analyzing the wireless networking systems, especially for resource man-

(10)

agement problems. Game theory is a mathematical tool applied to model and to analyze the outcome of interactions, such as competitions, among multiple decision-makers. It can help analyze and predict the selfish behaviors of participants in wireless networks. It also provides a series of tools to regulate those undesired selfish behaviors and eliminate the performance degradation from the competition effects. By introducing game-theoretic approaches, we can propose novel solutions that are efficient, practical, and robust to the selfish behaviors of participants in wireless networks.

In this dissertation, we propose novel game-theoretic approaches to several resource management problems in the state-of-the-art wireless systems, which are heterogeneous networks, D2D communication, and multicasting system. Given the analysis based on game theory, we identify the potential threats from selfishness and propose novel solutions to each problem in order to address the selfish behaviors of participants while keeping reasonable efficiency and practicability. Extensive simulations are also executed for evalu- ating the performance of each proposed solutions.

In heterogeneous networks, we first study the femtocell coverage control problem. We first identify that the system information collected from them does not necessarily reflect the true status of the system due to the selfish nature of mobile stations. Thus, we design FEmtocell Virtual Election Rule (FEVER), a voting based mechanism that not only is proved to be truthful and has low implementation complexity, but also strikes a balance between efficiency and fairness to meet the different needs. Then, we study the femtocell service price problem in heterogeneous network. Femtocell technology can be used to improve service quality and increase profit by attracting customers. Meanwhile, differentiated contracts for different types of users also show great potential for profit increase. We show that by applying differentiated contracts in femtocell service. The profits of service providers can be significantly increased. Finally, we study the carrier aggregation mechanism

(11)

in LTE-Advanced system. We observe that selfish users may untruthfully report their QoS requirements in order to manipulate the carrier activation and resource block allocation. Therefore, we propose a truthful auction with a greedy resource allocation algorithm in order to guarantee that all rational UEs truthfully report their QoS requirements.

In D2D communication system, we observe that the ad-hoc characteristic of D2D communication poses the truth-telling issue into the system. Ad- ditionally, we find out that that the transmission quality in D2D communications can be significantly improved through a proper resource exchange.

Based on this observation, we propose a Trader-assisted Resource Exchange (T-REX) mechanism, an exchange-based mechanism that converges in polynomial time and achieves Pareto optimal. We prove that all rational D2D pairs will truthfully report their information when the trader preference functions are properly designed.

Finally, in the multicastins system, we first discuss the general social learning problem in a network with exteranlity effects. We propose a game- theoretic framework called Chinese restaurant game. Through analyzing the Chinese restaurant game, we derive the optimal strategy of each agent and provide a recursive method to achieve the optimal strategy. Based on this framework, we analyze the scalable video coding multicasting system, which is an effective solution for video streaming services in wireless networks. We observe that the requests from users in such a system in fact is a social decision making problem and can be formulated with Chinese restaurant game.

We propose a stochastic framework based on Multi-dimensional Markov De- cision Process (M-MDP) to evaluate the corresponding system efficiency. We show that the optimal pricing strategy, which maximizes the expected revenue of the service provider, also increases the social welfare of the system.

Keywords: wireless networks, resource management, game theory, het- erogeneous network, device-to-device communication, multicast, selfish

(12)

(13)

List of Figures

1.1 An illustration of Interference in Heterogeneous Networks . . . 16

1.2 Strategic Game Approach to Heterogeneous Networks . . . 21

1.3 Stackelberg Game Approach to Heterogeneous Networks . . . 22

2.1 Downlink Cell-breathing in an overlay macrocell-femtocell system . . . . 29

2.2 Selfish Behaviors of MSes under traditional access policy . . . 30

2.3 Femtocell Cell-Breathing Control Framework . . . 35

2.4 Two-Stage Game Model . . . 40

2.5 MSes' Single-Peaked Preferences over P_f . . . 44

2.6 FEVER Mechanism . . . 47

2.7 Simulation Results: FEVER Mechanism under Different Choice of κ . . . 58

2.8 Simulation Results: Efficiency and Fairness Tradeoff . . . 59

2.9 Simulation Results: SG-FEVER Mechanism under different SG modes . 60 3.1 Negative Correlation between Macrocell and Femtocell service quality in Shared-Spectrum System . . . 85

3.2 Different Contract Structures in Split-Spectrum System . . . 86

3.3 Different Contract Structures in Shared-Spectrum System . . . 88

3.4 Different Contract Structures in Shared-Spectrum System . . . 88

3.5 Effect of Service Quality Difference . . . 90

4.1 An illustration of CA-enabled LTE Cell . . . 97

4.2 Information Reporting and Resource Allocation Process . . . 107

4.3 Simulation Results: Number of UEs . . . 115

(20)

4.4 Simulation Results: Cell Radius . . . 116

4.5 Simulation Results: Requested Data Amount . . . 117

5.1 D2D Resource Exchange . . . 122

5.2 D2D Resource Allocation Framwork . . . 126

5.3 An example of the T-REX mechanism . . . 137

5.4 Cheating in CYC preference . . . 140

5.5 Simulation Results: Interference . . . 145

5.6 Simulation Results: Convergence . . . 147

5.7 Simulation Results: Prioritization . . . 148

6.1 The effect of different Table Size Ratio and Signal Quality . . . 171

6.2 Price of Anarchy with Different Utility Functions . . . 173

6.3 Average utility of Customers in Resource Pool scenario when r = 0.4 . . 175

6.4 Average utility of Customers in Available/Unavailable scenario when r = 0176 6.5 Sequential Cooperative Spectrum Sensing and Accessing . . . 178

6.6 Spectrum Accessing in Cognitive Radio Network under Different Schemes 182 7.1 A SVC multicasting platform offering two 2-layer videos . . . 187

7.2 An illustration of State Transition in the proposed M-MDP system . . . . 202

7.3 System Performance under Different Server Capacity . . . 214

7.4 System Performance under Different Available Service Time Ratio . . . . 215

(21)

List of Tables

2.1 Notations . . . 39

3.1 Notations . . . 75

7.1 Transmission Throughput . . . 212

7.2 Video Specifications . . . 212

7.3 User Specifications . . . 213

7.4 Expected revenue under different discounted factor γ . . . 214

(22)

(23)

Chapter 1 Introduction

1.1 Motivation

Wireless networking experiences significant technique advances in recent decades.

These advances, along with the accelerating deployments in all developed and most de- veloping countries, is pushed by the increasing demands from the mass all over the world.

Next-generation wireless wide-area network (WWAN) standards, such as IEEE 802.16 and 3GPP LTE-Advanced, raised most attentions in these days since they are established for achieving the 4G standard requirements proposed by ITU-R [1], which illustrates a colorful vision for the future wireless communication in every place, every moment.

Nevertheless, the resource for wireless networking, such as spectrum, is very limited and difficult to expand. In order to fulfill this vision with limited resource, these new wireless standards introduces numerous advanced techniques, such as multi-input-multi-output (MIMO), orthogonal frequency-division multiple access (OFDMA), carrier aggregation (CA), heterogeneous networks, device-to-device (D2D) communications, multicasting/

broadcasting services, and so on. Some of them are used for achieving ultra-high throughput, some are for maintaining service quality, and some are for increasing resource utilization efficiency. These techniques are technically guaranteed to boost the service quality of next generation wireless communications. Nevertheless, challenges still exist in applying these techniques in real world scenarios.

Traditionally, the challenges in applying these techniques are from two perspectives:

(24)

practicability and optimality, under the resource constraints. The practicability of a technique usually is the first to be addressed since a technique without a practical realization or implementation cannot be applied in real world at all. Optimality issue arises when we are interested in the optimal configuration in order to achieve the optimal yet theoretic performance. In many scenarios, there is a tradeoff between the solutions from these two perspectives. Most researchers focus on seeking a practical yet efficient solution for implementing the techniques. Nevertheless, all solutions are under the constraints of limited resource in wireless networks. Most challenges then can be transformed into a resource management problem: given the available resource, how and on what degrees the new technique can help improve the network performance?

We observe that there are some common characteristics in wireless network resource management problems. Let's consider a typical wireless network with user devices (mo- bilephones, tablets, etc.) and infrastructures (base stations, core networks, etc.) interact with each other through wireless communications. A participant's operation, such as re- questing resource, reporting QoS requirements, or determining transmission power, may not only influence the service quality experienced by herself but also affects the ones experienced by others in the same network. Additionally, when the participants are controlled by real humans, we can fairly assume that these participants are rational. A rational participant's objective is to maximize her utility, which could be related to her experienced service quality, data delivery/reception amount, or even revenue/profit in the system. Given the fact that the resource is limited in wireless networks, an operation that increases the participant's allocated resource inevitably reduces the resource for other participants, and therefore degrades the service quality. The competition among the participants forms naturally. Combining the competition effect and rationality of participants together, the operations of these participants become selfish in wireless networks.

The selfishness of participants in wireless networks imposes an serious threat to all existing solutions which are based on traditional perspectives, especially for resource management problems. The solutions from traditional perspectives usually inherent an assumption that all participants faithfully follow the orders of the system for some global

(25)

objective. This assumption becomes invalid when participants are selfish, which could lead to 1) undesired operations which are unexpected from traditional perspectives, and 2) degraded system performance due to fierce competitions among participants. A new solution concept for wireless network resource management problems is required to analyze, predict, or even regulate these selfish behaviors in order to prevent undesired performance degradation.

Game-theoretic approach, a new perspective for wireless networking problems, can help analyze and predict the selfish behaviors of participants in wireless networks. Game theory is a mathematical tool applied to model and to analyze the outcome of interactions among multiple decision-makers. It has been shown to be a powerful tool for analyzing complex interactive system in economic and politic. Additionally, it also provides a series of tools to regulate those undesired selfish behaviors and eliminate the performance degradation from the competition effects. In recent years, researchers discovered that game theory is also suitable for analyzing the wireless networking systems. Recall- ing a typical wireless network with various participants interact with each other through wireless communications by following designed protocols. Some or all participants can be considered as players in the game, while the wireless communication techniques and designed protocols can be considered as the game rules. Finally, the service quality or performance experienced by participants can be considered their utility in the game. Fol- lowing this formulation, we transform a wireless network resource management problem into a game. The behaviors of selfish participants can then be studied through the well- established solution concepts in game theory, such as best responses and Nash equilibrium.

The undesired behaviors, such as untruthful information report, unfair competition, and other cheating actions, can then be identified.

By introducing game-theoretic approaches into the wireless network resource management problems, we potentially can propose novel solutions that are efficient, practical, and robust to the selfish behaviors of participants in wireless networks. Those undesired behaviors in wireless networks that are identified through game-theoretic analysis could be regulated by powerful tools provided in game theory. For instance, a device may request

(26)

some resource with an amount exceeding the amount she required in a resource allocation process. Such a selfish behavior may increase the probability that this device's require- ment is satisfied, but it also degrades the overall resource utilization efficiency when the resulting resource allocation deviates from the optimal one due to this untruthful request.

A game-theoretic pricing design can be implemented in the resource allocation process to prevent such an issue. Through imposing a proper price on the resource, the requests from the devices will more likely reflect their true requirements since increasing their demands also lead to higher payment to them. Various game-theoretic techniques, such as voting, pricing, auction, and so on, can be applied on various resource management problems in wireless networks in order to regulate the undesired selfish behaviors and increase the system efficiency. Nevertheless, most of them also require further expansion or modifications in order to satisfy the practicability constraints in wireless networks.

Our objective is to understand the applications of game theory in various state-of-the- art wireless networks, such as heterogeneous networks, device-to-device communications, and multicasting system. In each system, we formulate the critical resource management problem as a game, and then identify the potential selfish and undesired behaviors of participants through game-theoretic analysis. Finally, we propose novel solutions to regulate these behaviors in order to increase the system efficiency while satisfying the practicability constraints. Both the theoretic improvements and limitations of game theory in different wireless network resource management problems will be thoroughly studied in this dissertation. In general, we show that game-theoretic approaches indeed benefit the wireless network by providing efficient, practical, and robust solutions to various resource management problems.

1.2 Dissertation Outline

In this dissertation, we first provide brief preliminaries on the wireless systems we would like to investigate and the basic game theory concepts we would apply in Chapter 1.4. Then, we propose novel game-theoretic approaches to several resource management problems in the state-of-the-art wireless systems in the following chapters, which are het-

(27)

erogeneous networks (Chapter 2, 3, and 4), D2D communication (Chapter 5), and multicasting system with social learning (Chapter 6 and 7). All the studied systems share the same characteristic that devices may selfishly compete for the limited resource in the wireless system, as we illustrated in each chapter. Given the analysis based on game theory, we then propose novel solutions to each problem in order to address the selfish behaviors of participants while keeping reasonable efficiency and practicability. Finally, we draw our conclusions in Chapter 8.

1.2.1 Cell-Breathing in Heterogeneous Networks: A Voting Approach (Chapter 2)

Overlay macrocell-femtocell system, a popular type of heterogeneous network, aims to increase the system capacity with a low-cost infrastructure. To construct such an infrastructure, we need to solve some existing problems. First, there is a tradeoff between femtocell coverage and overall system throughput, which we defined as the cell-breathing phenomenon. In light of this, we propose a femtocell downlink cell-breathing control framework to strike a balance between the coverage and data rate. Second, due to the selfish nature of mobile stations, the system information collected from them does not necessarily reflect the true status of the system. Thus, we design FEmtocell Virtual Elec- tion Rule (FEVER), a voting based direct mechanism that only requires users to report their channel quality information to the femtocell base station. Not only is it proved to be truthful and has low implementation complexity, but also strikes a balance between efficiency and fairness to meet the different needs. The simulation results verify the enhanced system performance under FEVER mechanism.

1.2.2 Service Price in Heterogeneous Networks: Optimal Contract Design (Chapter 3)

Most service providers offer an unlimited data service plan under a flat, fixed-rate contract to meet the huge demand. However, because service quality and user experience can

(28)

vary dramatically in wireless communications, such a contract design is unable to provide equal service quality for all users, which greatly limits the profit potential of service providers. As a result, mobile industries look to femtocell technology to improve service quality and increase profit by attracting customers. Meanwhile, differentiated contracts for different types of users also show great potential for profit increase. In this chapter, we investigate unlimited data service plans in terms of enhancements from both femtocell systems and differentiated contracts. The incentive compatibility (IC) issue in differentiated contract design is considered under the overlay macrocell-femtocell system in both split-spectrum and shared-spectrum models. The profits under optimal differentiated contracts, with and without the IC condition are compared to traditional flat fee contracts, and numerical results show that optimal differentiated contracts indeed generate more profits and serve more users.

1.2.3 Carrier Aggregation in LTE-Advanced System: An Auction De- sign (Chapter 4)

Carrier aggregation is introduced in LTE-Advanced for aggregating non-contiguous spectrum into a virtual carrier. UEs with carrier aggregation capability can increase their peak data rates by transmitting through the aggregated virtual carrier that virtually provides a larger transmission bandwidth. Nevertheless, it deserves further study on how carrier aggregation should be implemented and configured in order to address the diverse carrier quality experienced by UEs and their heterogeneous QoS requirements efficiently.

Additionally, most existing resource allocation methods relies on the assumption that UEs always report their information truthfully, which may be unrealistic when UEs are rational from a game-theoretic perspective. In order to address the preceding concerns, we provide a utility-based game-theoretic approach to the carrier aggregation design in LTE- Advanced system. We first formulate the resource allocation problem in carrier aggregation as a non-linear optimization problem, which is proved to be NP-hard. Then, we propose a truthful auction with a greedy resource allocation algorithm in order to 1) find an efficient carrier activation and resource allocation solution under the QoS requirements

(29)

of UEs, and 2) guarantee that all rational UEs truthfully report their QoS requirements.

Finally, we conduct extensive simulations in order to evaluate the system performance of the proposed auction design.

1.2.4 Device-to-Device Communications in LTE-Advanced System:

A Resource Exchange Approach (Chapter 5)

Device-to-Device (D2D) communication could improve the efficiency of resource utilization in cellular networks by allowing nearby devices to communicate directly with each other. Nevertheless, one main challenge in D2D communication is resource allocation, given the diverse channel qualities of D2D devices. Additionally, when D2D devices and users are rational, then from a game-theoretic perspective, the ad-hoc characteristic of D2D communication poses the truth-telling issue into the system.

We observed that the transmission quality in D2D communications can be significantly improved through a proper resource exchange. Based on this observation, we propose a novel D2D resource allocation framework for an LTE- Advanced system. We theoretically prove that any arbitrary algorithm, either distributed or centralized, will converge in the proposed framework whenever all performed exchanges are beneficial. Based on the concept of beneficial exchange, we propose a Trader-assisted Resource Exchange (T- REX) mechanism, an exchange-based mechanism that converges in polynomial time and achieves Pareto optimal, as an efficient and flexible solution to the D2D resource allocation problem. The eNodeB regulates the D2D resource allocation through designing the trader preference functions in the T-REX mechanism. By applying game-theoretic analysis to the D2D communication system, we prove that all rational D2D pairs will truthfully report their information when the trader preference functions are properly designed.

Finally, our simulation results show that the proposed T-REX mechanism significantly mitigates the interference experienced by D2D devices in LTE-Advanced systems.

(30)

1.2.5 Chinese Restaurant Game: Social Learning vs. Network Ex- ternality (Chapter 6)

In a social network, agents are intelligent and have the capability to make decisions to maximize their utilities. They can either make wise decisions by taking advantages of other agents' experiences through learning, or make decisions earlier to avoid competitions from huge crowds. Both these two effects, social learning and negative network externality, play important roles in the decision process of an agent. While there are existing works on either social learning or negative network externality, a general study on considering both effects is still limited. We find that Chinese restaurant process, a popular random process, provides a well-defined structure to model the decision process of an agent under these two effects. By introducing the strategic behavior into the non-strategic Chinese restaurant process, we propose a new game, called Chinese Restaurant Game, to formulate the social learning problem with negative network externality. Through analyzing the proposed Chinese restaurant game, we derive the optimal strategy of each agent and provide a recursive method to achieve the optimal strategy. How social learning and negative network externality influence each other under various settings is studied through simulations. We also illustrate the spectrum access problem in cognitive radio networks as one of the application of Chinese restaurant game. We find that the proposed Chinese restaurant game theoretic approach indeed helps users make better decisions and improves the overall system performance.

1.2.6 Stochastic SVC Multicasting model using Chinese Restaurant Game (Chapter 7)

Heterogeneous multimedia content delivery over wireless networks is an important yet challenging issue. One of the challenges is maintaining the quality of service due to the scarce resource in wireless communications and heavy loadings from heterogeneous demands. A promising solution is combining multicasting and scalable video coding (SVC) techniques via cross-layer design which has been shown to be effectively enhancing the

(31)

quality of multimedia content delivery service in the literature. Nevertheless, most existing works on SVC multicasting system focus on the static scenarios, where a snapshot of user demands is given and remains the same. In addition, the economic value of SVC multicasting system, which is an important issue from the service provider's perspective, has seldom been explored. In this work, we study a subscription-based SVC multicasting system with stochastic user arrival and heterogeneous user preferences. A stochastic framework based on Multi-dimensional Markov Decision Process (M-MDP) is proposed to study the negative network externality existing in the proposed system and theoretically evaluate the corresponding system efficiency. A game-theoretic analysis is conducted to understand the rational demands from heterogeneous users under different subscription pricing schemes. By transforming the original dynamic and complex M-MDP revenue optimization problem into a traditional average-reward MDP problem, we show that the optimal pricing strategy which maximizes the expected revenue of the service provider can be derived efficiently. Moreover, the overall user's valuation on the system, e.g., social welfare, is maximized under such an optimal pricing strategy. Finally, the efficiency of the proposed solutions is evaluated through simulations.

1.3 Contributions of Dissertation

In summary, we made the following contributions in this dissertation:

Heterogeneous Networks

1. We are the first to study the cell-breathing phenomenon in heterogeneous networks.

We propose a novel femtocell cell-breathing control framework for managing the load-balancing and coverage control among overlay cells.

2. Based on the cell-breathing framework, we formulate a game-theoretical model for discussing the cheating issue in overlay network with selfish mobile stations. We introduce the concept of voting theory into the cell-breathing control framework.

The proposed voting-based FEVER mechanism is proved to be truthful. This truth-

(32)

ful strategies form a dominant-strategy Nash Equilibrium in FEVER mechanism, which can be easily implemented. In addition, we prove that FEVER mechanism offers the flexibility to strike a balance between capacity efficiency and allocation fairness.

3. We propose a novel wireless service differentiation framework to investigate the profit service providers make under a variety of differentiated contracts in the heterogeneous system. This framework addresses the differences between wireless service quality among users, which is the basis of the service differentiation.

4. We draw a comparison between the shared-spectrum and split-spectrum systems in our service differentiation framework and have derived the optimal (profit max- imizing) contracts under three schemes: flat fee contracts, differentiated contracts without incentive compatible concerns, and incentive compatible differentiated contracts. In a split-spectrum system, it is difficult to further extract profits from MSs as the only incentive compatible contract is a flat fee one. By contrast, in a shared- spectrum system, there are differentiated contracts generating profits by raising service prices for the MSs with good service qualities in femtocells, while providing cheaper prices to other MSs with poor service qualities.

5. We address the heterogeneous characteristics of carrier quality, coverage, and UE QoS requirements in the proposed game-theoretic approach to carrier aggregation mechanism. We make use of the carrier aggregation to enhance the system performance by satisfying the QoS requirements of UEs more efficiently. Specifically, we consider two type of UEs, throughput-sensitive and delay-sensitive UEs, in this work. The proposed solution effectively reduces the delay for delay-sensitive UEs while satisfying the throughput requirements of throughput-sensitive UEs. Addi- tionally, we propose a truthful auction design specifically for the heterogeneous carrier quality and QoS requirements of UEs. We theoretically prove that the proposed design indeed provides proper incentive for the UEs to truthfully report their QoS requirements.

(33)

6. All proposed solutions for each problem in heterogeneous networks are evaluated through extensive simulations in our LTE-Advances simulator, which uses the models and parameters suggested in 4G evaluation document [1] and 3GPP LTE standard [2].

Device-to-device Communication

1. We propose a novel LTE-Advanced D2D resource allocation framework based on the resource exchange approach. We reuse most existing LTE-Advanced compo- nents and followed the same signalling flow logic in order to minimize the protocol impacts.

2. We theoretically prove that the resource exchange approach is equivalent to the traditional resource allocation approach in the solution feasibility. Additionally, we prove that any arbitrary algorithm, either distributed or centralized, will converge in the proposed framework whenever all exchanges are beneficial. To the best of our knowledge, we are the first group to present the resource exchange approach to the D2D resource allocation problem.

3. We propose the Trader-assisted Resource Exchange (T-REX) mechanism as an efficient and flexible solution to the D2D resource allocation problem in the proposed framework. The T-REX mechanism identifies the beneficial exchanges through analyzing the corresponding exchange graph. The algorithm's complexity is polynomial, which makes it a practical solution to large-scale D2D networks. In addition, the derived allocation is Pareto optimal; therefore, the efficiency is guaranteed. In addition, we prove that the T-REX mechanism is strategy-proof when the trader preference functions are properly designed.

4. All the proposed solutions were evaluated through the proposed LTE-Advanced D2D simulator, which uses the models and parameters suggested in the latest 3GPP technical contribution [3]. Our simulation results showed that the proposed T-REX mechanism significantly mitigates the interference experienced by D2D devices.

(34)

Additionally, the convergence of the proposed framework is verified and evaluated in the simulations.

Social Learning and Multicasting System

1. We propose a novel game, called Chinese Restaurant Game, to formulate the so- cial learning problem with negative network externality by introducing the strategic behavior into the non-strategic Chinese restaurant process. Through analyzing the Chinese restaurant game, we observe that the timing of making decision significantly influences a participant's utility. We show that there exists a tradeoff between two contradictory advantages, which are making decisions earlier for choosing better actions and making decisions later for learning more accurate believes.

2. Chinese restaurant game is general enough to model various learning and decision making problems in social network, cloud computing, and wireless networks. We demonstrate how the model can be applied in real applications by studying the channel access and sensing problems in cognitive radio network. Through simulations, we show that both the sensing accuracy and utilities of network users are enhanced by applying the best strategies derived from Chinese restaurant game.

3. We develop a Markov decision process based stochastic framework to analyze the resource allocation in a SVC multicasting system with heterogeneous user demands.

By considering the stochastic user arrival, such a framework is more general than the existing snapshot-based approaches in the literature.

4. We propose a game-theoretic model, which is based on Chinese restaurant game, to analyze the behaviors of heterogeneous users. We study how rational and intelligent users submit their demands, i.e., subscriptions, under two pricing schemes: one-time charge scheme and per-slot charge scheme, and derive the equilibrium conditions of the game. To the best of our knowledge, this is the first work bringing game theoretic analysis to the SVC multicasting system.

(35)

5. We theoretically evaluate the economic value of the SVC multicasting system. Specif- ically, we investigate the revenue-maximized policy and pricing strategies in both one-time charge and per-slot charge schemes. We propose an efficient algorithm to derive the optimal policy and pricing strategies of the SVC multicasting system.

Both theory and simulation results confirm that the derived solution not only maximizes the expected revenue but also optimizes the social welfare.

(36)

1.4 Preliminaries

1.4.1 4G Heterogeneous Networks

High system capacity is one of the fundamental requirements of wireless communication system. Various techniques have been proposed in WiMAX and LTE-Advanced (LTE-A) for enhancing the quality and transmission capacity such as Multi-Input Multiple- Output (MIMO), Carrier Aggregation (CA), and other techniques [4]. While most advanced signal and transmission techniques potentially enhance the performance of wireless systems [4], they eventually reach the theoretical limitation due to the physical laws:

the signal quality. Most of next generation wireless networks are planned to operate in high frequency spectrum. In such spectrum, the signals will degrade significantly in long distance and indoor environments. This suggests that more areas will experience weak signal receptions unless the network deployment is densified.

In order to boost network capacity in a flexible and cost-efficient manner, the concept of Heterogeneous Networks (HetNets) has been introduced in LTE-A standard [5]. A heterogeneous network consists of macrocells, which are deployed for serving large coverage areas, and low-power and low-cost nodes such as picocells, femtocells, relay nodes, or remote radio heads (RRHs), which provide services in areas with dedicated capacity.

The wireless signal quality can be greatly enhanced through the assistance from the low- power nodes when they are properly deployed in the coverage holes in the macrocells.

Additionally, these low-cost nodes are more economically attractive as they usually require lower-cost infrastructure and lower requirements in terms of backhaul connections.

A typical HetNet in LTE-A is composed of lower-power base stations (BSs) underlying in the existing macrocell system. These small BSs are intended to increase the signal strength, offload the macrocells, and enhance the spectrum utilization. The deployment of HetNets can be planned and conducted by the service provider in advance, or requested and deployed by users themselves. The service area and operating spectrum of the small cells is usually partly or fully overlapping with the macrocell. Heterogeneous small cell base stations have been introduced in HetNets [5]. We briefly state as follows:

(37)

• Picocells are low-power (23 to 30dBm) cell towers providing similar features as macrocells except smaller coverage (hundreds of meters) and user load (tens of users). They use the same backhaul as the macrocells and are deployed by the service provider.

• Relays are small stations that deliver the data between macrocells and MSs in a multi-hop over-the-air scheme. They are mostly deployed by the service provider in order to extend the coverage of existing networks. A relay requires over-the- air backhaul capacity between macrocell BS and uses a similar transmit power as picocells.

• RRHs are radio control units that are connected directly to the macrocells through fibers but deployed with a distance from the macrocell BS. The macrocell has full control on the RRHs and operate them as its own wireless interface.

• Femtocells are also known as Home eNode Bs (HeNBs) in LTE systems [6]. A femtocell BS (femtoBS) can be regarded as a simple, low-transmission power (i.e.

23 dBm or less) base station installed by users in an unplanned manner. Through the deployment of femtoBSs, subscribers are able to access to networks via broadband backhaul. That is, femtocells may utilize Internet protocol (IP) and flat base station architectures. FemtoBSs may operate in open-access, closed-subscribed group (CSG), or hybrid-access scheme, depending on the choice of the cell owner.

In these possible choices of small cells, the femtocell has the following advantages: It increases indoor signal coverage and system capacity on demands, providing higher link quality with lower transmission power, and utilizes the existing broadband connection as its backhaul. Nevertheless, the femtocell system faces several challenges. Intercell interference, one of the most severe issues, takes place since femtoBSs typically operate in a licensed spectrum. Therefore, their coverage overlaps with other base stations in the same spectrum, as shown in Fig. 1.1, in which networks often suffer from interference.

Additionally, the backhaul may also be an issue since it is likely that the femtocell operates in a broadband connection with limited quality of service (QoS), such as significant long

(38)

MacroBS

FemtoBS FemtoBS

Interference

Figure 1.1: An illustration of Interference in Heterogeneous Networks

delay. The low QoS of relied connections may limit the ability of the service provider to control the femtocells in real time.

1.4.2 Device-to-Device Communications

Heterogeneous networks basically make use of the proximity gain from the user device to the base stations of small cells. There is another way to make use of the proximity gain by implementing device-to-device (D2D) communication [7] in cellular system.

Traditionally, two cellular devices communicate with each other through multi-hop transmission, with base stations (BSs) as their intermediate infrastructure. Such a transmission scenario is inefficient in terms of both resource utilization and transmission delay when these two devices are in close proximity to each other. In D2D communication, two nearby devices to communicate with each other directly. This approach improves the transmission quality from the proximity [8], reduces the transmission delay by utilizing one-hop direct connection instead of two-hop cellular connection, and provides an extra dimension for resource reuse in the cellar system. 3GPP has begun to examine the service require- ment for Proximity-based Services (ProSe), which is the D2D communications for LTE- Advanced, and then has started ProSe radio access network standardization recently [9].

Four service scenarios are considered: whether the devices are within and out of network coverage, and whether the devices are allowed to discover nearby devices only or could utilize direct communications with each others.

There are two major challenges in D2D communication: peer discovery and resource

(39)

allocation [7]. D2D devices should have the ability to discover the potential opportunities to execute D2D communication with their peers. When devices are within the network, the BSs, or evolved Node-B (eNB) in LTE system, could participate in this process by identi- fying potential D2D pairs or providing (approximate) location and proximity information to devices. When devices are out of the network coverage, a distributed peer discovery function is necessary. Nevertheless, the accurate channel gain still needs to be measured through a proximity signalling technique, which requires further studies.

Resource allocation is another key challenge in D2D communications in a cellular system [7]. As specified by [7], D2D communications can be executed in unlicensed or licensed spectra. Although the former choice could utilize the existing wireless technology such as Wi-Fi and Bluetooth, it is relatively unreliable due to its openness to other out-of- system devices. For the latter choice, in which the resources utilized by D2D communications are dedicated resources (spectrum, resource blocks, etc.) licensed to the cellular system or a specific purpose (public safety, for instance) [10]. When devcies are within the network coverage, the BSs should regulate the resource allocation of D2D communications in order to prevent undesired interference to existing cellular users, and enhance resource utilization efficiency. When devices are out of the network coverage, they should have the ability to identify the available resource and reduce the potential interference to other devices in a distributed fashion.

1.4.3 Multimedia Multicasting System

Multimedia service is one of the most popular and fast growing applications in wireless networks. It is also the most challenging one since the loading generated by the multimedia content is much heavier than other services, along with more strict QoS requirements such as short delays. In some application scenarios, such as live broadcast streaming [11] or Internet protocol TV [12, 13], users in the same network request for the same multimedia content. In such scenarios, multicasting services could be applied to efficiently deliver the content to multiple users by making use the broadcasting characteristic of wireless communications.

(40)

The multicasting service has been standardized as Enhanced Multimedia Broadcast Multicast Services (E-MBMS) in LTE and Multimedia multicast and broadcast service (MBS) in WiMAX. The basic concept of multcasting is delivering the same content once to multiple users at once using relatively robust modulation and coding schemes. The resource for multicasting service could be either predefined in order to prevent unnecessary interference to existing unicast services, or opportunistically reuse when the interference is tolerable.

How the resource should be allocated for such an service is the main challenge and has been explored by numerous researchers. In brief, the resource should be efficiently allocated to different multicast group, which is determined by the demands, according to the diverse transmission quality experienced by users. Additionally, the service should have minimal impact on other traditional unicast services, which imposes a (dynamic) resource constraint on the multicasting service. Most researchers agree that a cross-layer design between wireless communication layer and application layer is necessary since the service quality highly depends on the quality of received multimedia content and preferences of users, which cannot be linked to the wireless communication performance in a straight- forward way. For instance, the users may be heterogeneous in experienced transmission quality, device capability, or preference on the video content/quality, which should be taken into account when configuring the mutlicasting service. The relation of wireless transmission performance to the multimedia content quality should also be carefully addressed in the cross-layer design. The relation usually depends on the multimedia en- coding/decoding technique applied in the application layer. For instance, scalable video coding (SVC) [14] encodes a video into multiple layers. A basic-quality video can be derived by decoding the basic layer along, while higher quality video can be derived by decoding multiple layers in sequential order. This characteristic makes it a perfect match to multmedia multicasting service providing multiple qualities of multimedia contents.

(41)

1.4.4 Game Theory

Basic Game Elements

Game theory is a powerful tool applied to model and to analyze the outcome of interactions among multiple decision-makers. A traditional game consists of three basic ele- ments: players, strategies, and utilities. Players are the participants and decision makers in the game. They can take some predefined actions to affect the interaction with other players and make influence on the final game result. Players are individual decision makers - given specified information (game rules, state of the system, applied actions of other players, etc) of the game, they apply strategies to decide the actions (reactions) taken in the game. Formally speaking, strategies are functions that map collected information to applied actions. The set of all possible strategies is defined as a strategy space, and the set of a possible combination of each player's strategy is called a strategy profile.

Given this information, players may apply different strategies when making decisions.

With strategy profiles and related arguments, the game structure will produce a corresponding outcome. Outcomes can be considered as the results produced by an input strategy profile, which may carry different implications to different players. A player's evaluation of the outcome is given by a utility function. Utility functions, as quantified evalua- tions to the outcomes of a game, map the outcomes into real-value spaces. Since different game strategies may bring out different outcomes, we can see the element of utility as a function of a strategy evaluation. Under the framework of a game theory, the behavior of players' interactions can be properly modeled. The process may be also helpful to supply insights into the problems we investigate. For researchers in communication areas, game theory is very useful to analyze problems involving interactions among elements in the system, the resource allocation problem particularly.

In most cases, we assume that all players are rational. Thus, they tend to adopt strategy that can maximize their utility. In such games, every player is trying to maximize their own utility. Furthermore, if the players refuse to collude with each other, the game can be modeled as a non-cooperative game. In most game models, the purpose of the theoretic analysis is to find out the equilibrium, namely, the most likely produced outcome of a

(42)

steady state in the system.

Strategic Game and Nash Equilibrium

A strategic game is a type of games that all players behave simultaneously with perfect knowledge on other players' possible actions. Specifically, suppose that there is a game that involves two or more players, in which each player is assumed to know the actions of the other players. All players choose their actions simultaneously, and then the outcome of the game is also settled. In such a case, a rational player should predict what actions other players will choose before she chooses her action.

The expected outcome of the game can be found through finding the Nash equilibrium. Let's assume that there exists an action profile that after each player has chosen her action accordingly, no player can increase her utility from changing her action when other players' actions remain unchanged. If such an action profile is applied, no players have the incentive to deviate from the applied action since the deviation gives her equal or less utility. When the above conditions are met, the action profile constitutes a Nash equilibrium.

Using a typical HetNet system as an example, femtocells can be considered as the players in the strategic game, while their actions are their applied transmission power, occupied resources, or other operations that will potentially influence the service quality of other cells. Then, a femtocell's utility can be defined as the service quality, such as the throughput or delay time, experienced by UEs in the cell. An example is illustrated in Fig. 1.2, where multiple femtoBSs are determining their transmission power. Given other femtocell's transmission power, a femtocell may have her optimal transmission power that maximizes her utility. In such an approach, we would like to identify the stable outcome of the game, that is, the Nash equilibrium in the HetNet.

Strategic game approach is straight forwarding, but the results may not be appealing:

the Nash equilibrium can be an inefficient outcome comparing to the optimal solution due to the competitive effect in strategic game. Some regulation designs, such as penalties on the femtocells, may be necessary to improve the system performance.

(43)

P

₁

= 23 dBm

P

₂

= 21 dBm

P

₃

= 23 dBm

P

₄

= 10 dBm P

₅

= 15 dBm

P

^*

= ? dBm

Figure 1.2: Strategic Game Approach to Heterogeneous Networks

Stackelberg Game and Subgame-Perfect Nash Equilibrium

The Stackelberg game is a sequential game specifically for the systems with hierarchical structure, which is a natural approach to the resource allocation in HetNets. In a Stack- elberg game, two types of players, leaders and followers, are defined. In the game process, the leader should apply or announce her action first. Then, the followers response to the leader's action accordingly. Since all players are rational, the followers should choose their actions that maximize their own utility. By using this insight, the leader can predict the rational responses of the followers if she chooses certain actions. The leader then can choose her action that maximize her own utility based on her analysis on the rational response of the followers. The leaders, which should be macrocells in HetNets, have the advantages to apply their actions wisely before the followers, which are the femtocells in HetNets (Fig. 1.3). By strategically determining their applied action, the macrocells can lead the game to their desired outcome when they have enough information to predict the response of the femtocells in HetNets.

In a Stackelberg game, we will study the subgame perfect Nash equilibrium. Subgame perfect Nash equilibrium is a popular refinement to the Nash equilibrium under the sequential game. It guarantees that all players choose strategies rationally in every possible

(44)

Leader

MacroBS

FemtoBSs

Followers

1. MacroBS announces price, SINR targets, etc.

2. FemtoBSs determine their power, SINR targets, etc.

Figure 1.3: Stackelberg Game Approach to Heterogeneous Networks

subgame. A subgame is a part of the original game. In a typical leader-follower Stackel- berg game, there are two subgames: the follower subgame and the leader subgame. The subgame perfect Nash equilibrium in Stackelber game can be found through backward induction: the Nash equilibrium of subgames in the follower subgame, in every possible outcome of the leader subgame, is derived first. Then, taking the Nash equilibrium of the follower subgame as the predicted responses of followers, the leader chooses her best strategy that maximize her expected utility in the leader subgame. The subgame perfect Nash equilibrium is then derived by combing the Nash equilibrium of follower subgame and the best responses of the leaders.

Stackelberg game is ideal for system involving central authority or hierarchical structures, such as HetNets consists of both macrocell and femtocells. Nevertheless, the requirements to fully understand the response of femtocells given any possible action of macrocell in Stackelberg game may be impractical when the HetNet is complex. In such a case, learning techniques, such as reinforcement learning, could be applied to help macrocell find her best strategy in the Stackelberg game.

Voting and Truth-telling

Voting is an important research topic in economics and politics. It is commonly used in today's society for determining important choice or policy which affects the mass, espe-