CHAPTER 5 PERFORMANCE EVALUATION
5.2 Simulation Results
5.2.2 Performance in VIENNA
In this section, we demonstrates how we choose the parameters in the Pro-posed OLLA, Traditional OLLA [9], and Baseline OLLA [15]. Also, we com-pare the performance of the Original, which is a method applying in VIENNA without OLLA and with limited CSI; Traditional OLLA, which is the first OLLA; Baseline OLLA, which is enhanced OLLA; Proposed method; Perfect, which is without and with perfect CSI. Furthermore, the benefit of the proposed feedback is demonstrated in this section.
We consider 4 metrics: Throughput, BLER, Geometric mean throughput, and cell-edge user throughput. Throughput is the data that the base station can send within a certain time. BLER is the ratio of the number of erroneous transport blocks and the transmitted transport blocks. The geometric mean rate is the product of the average throughput of all users, written as
n
s Y
u∈U
Ru,
where Ru is the average throughput of useru. By definition, the cell-edge user throughput the 5th percentile point of the CDF of user throughput. In the thesis, it represents the throughput of the last user because the number of UE in the simulation is 20.
5.2. SIMULATION RESULTS 70
5.2.2.1 Traditional Method
The traditional method changes the MCS in a fixed step. If the base station receives a NACK, the next MCS will be decreased by ∆down; if the base station receives an ACK, the next MCS will be increased by 1. The mechanism can be written as
M CSt+1 =
M CSt− ∆down , if receiving an NACK M CSt+ 1 , if receiving an ACK
(5.2)
The ratio of NACK is defined as
The ratio of NACK = Total Nack steps Total convergence steps.
The convergence steps and the ratio of NACK both have an impact on the
Figure 41: Relationship between Step Size, Convergence Steps, and Ratio of Nack throughput. The convergence steps represent the capability of recovering from the sudden change in the channel. If the chosen MCS can find the suitable MCS as fast as possible, the performance could recover faster. However, the searching process may deteriorate the performance. If the chosen steps are too aggressive, it may cause failed transmission and sacrifice the throughput. Thus, observing these two metrics can help us to predict the performance easier.
It can be seen in Fig 41, the larger the ∆downis, the smaller the ratio of NACK.
For the reason that if the ∆down is larger, the probability of selecting MCS larger than the real SINR is smaller.
In Fig. 42, it can be observed that while ∆down is increasing, the BLER is decreasing. The throughput does not decrease with BLER. As mentioned before, both convergence steps and BLER can affect this metric. Since the throughput is highest as ∆down = 3, we choose this value in this thesis for the traditional method.
doi:10.6342/NTU201900453
5.2. SIMULATION RESULTS 71
(a) Throughput. (b) BLER.
(c) Geometric mean rate. (d) Cell-edge throughput.
Figure 42: Performance in Traditional Method 5.2.2.2 Baseline Method
The baseline method changes the ∆down and ∆down according to elapsed time [15].
M CSt+1 =
M CSt− ∆down,t , if receiving an NACK M CSt+ ∆up,t , if receiving an ACK
, (5.3)
where ∆down,t = AOffset+ AInitial· exp−γt, ∆up,t = a(AOffset+ AInitial).
There are several parameters in the baseline. AOffset is the offset value, ∆down,t will converge to AOffset in the end. We compare the parameters of the baseline;
AInitialhas the impact in the beginning; γ determined the rate of ∆down,tconverging to AOffset; a is the ratio between ∆down,t and ∆up,t. In Fig. 43, the γ has smaller impact on the performance. If AInitial= 0, its behavior is the same as traditional method. On the condition that Aof f set = 0, the performance looks better. We choose AInitial= 1, gamma = 1, and Aoffset = 2 in the end. The convergence steps is not the smallest in this setting, but it only sacrifices the convergence steps and improves the ratio of NACK a lot.
Fig. 44 shows how the baseline works with the chosen setting. The first ∆down is 3 and then become 2. The ∆up is always 1. The step size is varied with time.
5.2. SIMULATION RESULTS 72
Figure 43: Comparison of different Parameters of the Baseline
Figure 44: Demonstrate how the chosen MCS changes while AInitial = 1, gamma = 1, and Aoffset = 2
doi:10.6342/NTU201900453
5.2. SIMULATION RESULTS 73
5.2.2.3 Comparison of Convergence Steps between different Methods in dif-ferent Types of Feedbacks
The convergences steps is defined as -objective function. The formula of con-vergences steps can be written as
mean
" T X
t=0
rt
#
rt =
0, if the base station knows that it has reached the suitale MCS.
1, otherwise.
. (5.4)
38%
Figure 45: Comparison of Convergence Steps between different Methods in dif-ferent types of Feedbacks
SU indicates that OLLA exploits SUMIMO feedback; MU indicates that OLLA exploits MUMIMO feedback, and is not activated in single beam case;
MU, SB indicates that OLLA exploits MUMIMO feedback, and is activated in single beam case. The reason that SU do not need to consider OLLA in single beam case is that it can use SUMIMO feedback directly.
Fig. 45 shows the convergence speed in each OLLA methods. All the methods aim to converge as fast as possible. It can be observed that the proposed method shows the stronger capability in finding a good strategy of converging fast and dealing with the different condition in the channel.
5.2.2.4 Performance in different Types of Feedbacks
Fig. 46 explains that why the SU-feedback is adopted in this work. Notice the throughput of SU and MU, SB; the throughput is higher if we considering the capacity of single beam case. Also, directly using SU-feedback is better, because
5.2. SIMULATION RESULTS 74
the base station does not need to spend time on searching for the suitable MCS while scheduling in single beam. Furthermore, it is noticing that the BLER is high if we use MU-MIMO feedback to do OLLA in a single beam because the base station has to spend more effort in searching for the suitable MCS for each possible grouping.
Overall, SU feedback shows better performance, so we adopt SU feedback in this thesis.
(a) Throughput. (b) BLER.
Figure 46: Performance in different Types of Feedbacks
5.2.2.5 Performance in different Methods
When it comes to throughput, we can observe Fig. 47. Firstly, the throughput is double with OLLA. The traditional OLLA increases the throughput by 105%
in MU-MIMO and by 137% in NOMA+MUMIMO, respectively. This result im-plies that increasing the accuracy of the SINR benefits the throughput signifi-cantly. Moreover, the improvement between MU-MIMO and NOMA+MUMIMO is more obvious. The gain between MU-MIMO and NOMA+MUMIMO is almost can be ignored in the original method, while the gain is 7.4% in the proposed method. Furthermore, if we compare the throughput of the proposed method with the baseline method. It increases by 14.3% and by 6.7%, in MU-MIMO and in NOMA+MUMIMO, respectively. The cell-edge user throughput can be observed in Fig. 48, it is terribly small with original method, because the MU-MIMO feed-back is the lower bound of the expectation of SINR. Without OLLA, weak users can only transmit data with small MCS and have less chance to schedule with PF scheduler due to the underestimated SINR. In this situation, OLLA can improve the cell-edge user throughput a lot. Unfortunately, it does not guarantee the cell edge throughput and BLER. Because our target is to find a strategy to converge
doi:10.6342/NTU201900453
5.2. SIMULATION RESULTS 75
7.4%
16.7% 15%
(a) NOMA+MU-MIMO and MU-MIMO.
7.4%
16.7% 15%14.3%
105%
(b) MU-MIMO.
6.7%
137%
(c) NOMA+MU-MIMO.
Figure 47: Throughput in different Method
(a) BLER. (b) Geometric mean rate. (c) Cell-edge throughput.
Figure 48: Performance in different Methods
as fast as possible, this optimal strategy may be very aggressive. Fig. 49 demon-strates how each OLLA changes the selection of MCS according to the HARQ
5.2. SIMULATION RESULTS 76
information.
Still, the proposed OLLA performs well in terms of geometric mean rate. In short, the result indicates that improving the convergence speed has a positive impact on the throughput and fairness, but do not guarantee the BLER.
Figure 49: Demonstration for each OLLA Method
5.2.2.6 Impact of RN ack
Figure 50: Relationship between RN ack, Convergence Steps, and Ratio of Nack From previous simulations, we learns that if only considering convergence may cause higher BLER. Thus, we proposed a mechanism to control the BLER. We use reward shaping to control the bahavior of the trained agent. The reward function
doi:10.6342/NTU201900453
5.2. SIMULATION RESULTS 77
is defined as
ri,t =
−6, if ai,t < maxAckM CSi,tP airi,j or ai,t > minN ackM CSi,tP airi,j 0, else if |maxAckM CSi,tP airi,j − minN ackM CSi,tP airi,j| 6 1
and ai,t == maxAckM CSi,tP airi,j
−1, else if ai,t 6 realSINRi,j
−RN ack, else if ai,t > realSIN Ri,j
.
(5.5) In theory, RN ack depicts how reluctant is the agent to choose the MCS, which has
Figure 51: Demonstration of RN ack = 0 and RN ack = 6 a higher probability of causing higher BLER.
Fig. 50 indicates that RN ACK can control the how aggressive is the chosen steps effectively.
From Fig. 51, it seems that while RN ACK = 6, the chosen MCS is very conser-vative.
In Fig. 52, the BLER tends to be smaller as RN ack is larger. It is noticing that BLER become higher when RN ack = 6. It may be caused by the characteristics of the PF scheduler. Because this type of scheduler considers fairness and throughput at the same time, it might tend to choose the user with aggressive estimated SINR.
To verify this hypothesis, we modify the scheduler and call the new scheduler as a converge-first scheduler. In this converge-first scheduler, all the users have to find their own suitable MCS at first. After all the groupings find the suitable MCS,
5.2. SIMULATION RESULTS 78
(a) Throughput. (b) BLER.
Figure 52: Impact of RN ack on Performance
the original PF scheduler starts. This setting can prevent the scheduler from only choosing the grouping with aggressive estimated SINR. The results are shown in Fig. 53. It can be observed that the BLER is negatively correlated with RN ack. Despite that this type of the scheduler does not show the advantage in terms of throughput, it is useful for understanding the influence of the proposed method.
It is noticing in Fig. 52 that the convergence subframes in NOMA+MUMIMO are less than in MU-MIMO. It implies that if more the users can be scheduled at the same time, quicker can the base station find the suitable MCS of users.
(a) Throughput. (b) BLER.
Figure 53: Impact of RN ack on Performance in converge-first Scheduler
5.2.2.7 Performance of Proposed Method with Constraints of Retransmissions We observed that the behavior of the trained agent tends to choose conser-vative step in the beginning steps if RN ack is larger, so we hypothesise that the performance could be improved even if the base station does not find the most
doi:10.6342/NTU201900453
5.2. SIMULATION RESULTS 79
(a) Throughput. (b) BLER.
Figure 54: The trend of each metrics varies with the number of retransmissions.
suitable MCS. To verify this hypothesis, we constrain the number of the retrans-missions. The impact of the limitation of the number of retransmissions and the value of RN ack on the performance in Fig. 54. The impact of not finding the most suitable MCS can be observed as well. We compare the performance in two con-ditions: one is RN ack = 0, the other is RN ack = 6 because these two conditions have two different tendencies in choosing the MCS in the beginning.
In Fig. 54, it can be seen that the throughput is highest if there is no limit in retransmissions. Nevertheless, the throughput is not low while the retransmission is 0.
From the previous simulation, it can be observed that if RN ack = 0, the chosen steps tend to be aggressive, while the chosen steps tend to be more conservative as RN ack = 6. Thus, while the number of the retransmission=0, the RN ack = 6 is smaller due to the more conservative choice. The reason that why the BLER is not highest while the number of retransmissions is unlimited might be is that if there is limitation, the base station will choose the maximal available assigned MCS of UE, which might be much smaller than the suitable MCS or the other UEs’ modified MCS according to proposed OLLA, while the limitation of the UE is reached. Thus, the PF-scheduler has higher possibility to choose the other UE due to the higher modified but possibly too aggressive MCS and the smaller MCS of the UE, who has achieved limitation. Besides, the base station has to know which is minimal MCS within the MCSs received NACK, and the average steps, which definitely contains a NACK, is usually between 3 and 4. It implies that the UE might return at least a NACK within 4 retransmissions if it has reached the suitable MCS. However, if the suitable MCS is not achieved, the NACK might have never appeared. This is why the BLER is not correlated with the number of the retransmissions.
5.2. SIMULATION RESULTS 80
(a) Throughput. (b) BLER. (c) Geometric mean rate.
Figure 55: Comparison between original Method and proposed Method with Constraint of Retransmission=0 in NOMA+MU-MIMO
This constraint also brings another benefit, when the number of the retransmis-sions is 0. No retransmisretransmis-sions mean that the overhead is the same as the original feedback, but the performance is better as shown in Fig. 55. The throughput is doubled; The BLER is below 10%; The cell-edge use throughput increased sig-nificantly. It implies that the initial value according to the proposed OLLA is good.
The reason that why the proposed OLLA can be explained in Fig. 21. In MU-MIMO, if the user paired with a different user, the real SINR, γ, will be different.
Due to the interference, the user can only return the lower bound of the expected SINR according to the assumption of the distribution of SINR. The traditional mapping is an only one to one mapping. However, our proposed method can serve as a multiple dimensional mapping. Due to the multiple outputs, the capacity of the single user does not have to be limited, the SU-MIMO feedback can be adopted.
The capacity of the multiple beams can be measured through the training process.
It is noticing that the good initial value is not our designed target, but the initial value is lower could be because of the strong punishment while receiving NACK; so if the better performance is desired for this type of application, the redesign of the rewarding function is needed. Still, the design of the framework of reinforcement learning in this type of problem could remain.
In short, unlike supervised learning, the labelling is necessary, reinforcement learning may have the higher potential of searching for complicate mapping, which has higher uncertainty but the goal is clear. Despite that the mapping can not map to the perfect MCS directly due to the uncertainty from the previous analysis;
still, it is good enough.
doi:10.6342/NTU201900453
CHAPTER 6
CONCLUSION AND FUTURE WORK
We have investigated the design of reinforcement learning based OLLA mech-anism in order to be more robust to the various communication environment and improve the well-known convergence issue in OLLA. The impact of the design of the reinforcement learning and the communication system are investigated when applying OLLA. The suggestions and results are listed in the following.
Firstly, PMI of user and paring users, CQI, and historical data of MCS, which are received ACK/NACK, are effective features. The proposed model is able to find out the relationship between these features and improve the performance.
Secondly, we verify the design of the training model based on the domain knowledge of the communication. It is noticing that with the proposed OLLA mechanism, the convergence steps can be improved by 38% with SU-MIMO feed-back in comparison with the baseline method. Also, it is more robust to the different types of feedback. Furthermore, the throughput is increased by 14% in MU-MIMO and by 7% in NOMA+MUMIMO in comparison with the baseline method.
Thirdly, utilizing the potential of the capacity of single beam case and applying SU-MIMO are beneficial while applying OLLA. It can be seen that the throughput and fairness can be improved considerably with this setting.
Fourthly, we found that the convergence steps is not the only factor, which can affect the performance. The behavior of the chosen MCS have the influence as well.
Thus, we design the reward shaping and control the number of the retransmissions to control the performance of the OLLA effectively. Moreover, controlling the number of the retransmissions have an extra benefit to prevent the transmission from suffering the overhead issue.
In short, the proposed OLLA can improve the performance significantly. The training procedure is effective for the convergence speed and the behavior of the process of converging. SU-MIMO feedback is suggested while operation OLLA.
The constraint of the retransmissions provides the other possibility of the trained models.
81
REFERENCES
[1] N. Wooseok, B. Dongwoon, L. Jungwon, and K. Inyup, “Advanced interfer-ence management for 5g cellular networks,” IEEE Communications Magazine, vol. 52, no. 5, pp. 52–60, 2014.
[2] T. Yoo, N. Jindal, and A. Goldsmith, “Multi-antenna downlink channels with limited feedback and user selection,” IEEE Journal on Selected Areas in Communications, vol. 25, no. 7, 2007.
[3] M. Trivellato, F. Boccardi, and F. Tosato, “User selection schemes for mimo broadcast channels with limited feedback,” in Vehicular Technology Confer-ence, 2007. VTC2007-Spring. IEEE 65th. IEEE, 2007, pp. 2089–2093.
[4] J. Schaepperle and A. Regg, “Enhancement of throughput and fairness in 4g wireless access systems by non-orthogonal signaling,” Bell Labs Technical Journal, vol. 13, no. 4, pp. 59–77, 2009.
[5] M.-J. Yang and H.-Y. Hsieh, “Moving towards non-orthogonal multiple access in next-generation wireless access networks,” in Communications (ICC), 2015 IEEE International Conference on. IEEE, 2015, pp. 5633–5638.
[6] S. Timotheou and I. Krikidis, “Fairness for non-orthogonal multiple access in 5g systems,” IEEE Signal Processing Letters, vol. 22, no. 10, pp. 1647–1651, 2015.
[7] H. Sun, Y. Xu, and R. Q. Hu, “A noma and mu-mimo supported cellular network with underlaid d2d communications,” in Vehicular Technology Con-ference (VTC Spring), 2016 IEEE 83rd. IEEE, 2016, pp. 1–5.
[8] B. Kim, S. Lim, H. Kim, S. Suh, J. Kwun, S. Choi, C. Lee, S. Lee, and D. Hong, “Non-orthogonal multiple access in a downlink multiuser beamform-ing system,” in Military Communications Conference, MILCOM 2013-2013 IEEE. IEEE, 2013, pp. 1278–1283.
[9] A. Sampath, P. S. Kumar, and J. M. Holtzman, “On setting reverse link target sir in a cdma system,” in Vehicular Technology Conference, 1997, IEEE 47th, vol. 2. IEEE, 1997, pp. 929–933.
[10] M. G. Sarret, D. Catania, F. Frederiksen, A. F. Cattoni, G. Berardinelli, and P. Mogensen, “Dynamic outer loop link adaptation for the 5g centimeter-wave concept,” in European Wireless 2015; 21th European Wireless Conference;
Proceedings of. VDE, 2015, pp. 1–6.
[11] V. Buenestado, J. M. Ruiz-Aviles, M. Toril, S. Luna-Ram´ırez, and A. Mendo,
“Analysis of throughput performance statistics for benchmarking lte net-works,” IEEE Communications Letters, vol. 18, no. 9, pp. 1607–1610, 2014.
doi:10.6342/NTU201900453
REFERENCES 83
[12] F. Blanquez-Casado, G. Gomez, M. del Carmen Aguayo-Torres, and J. T.
Entrambasaguas, “eolla: an enhanced outer loop link adaptation for cellular networks,” EURASIP Journal on Wireless Communications and Networking, vol. 2016, no. 1, p. 20, 2016.
[13] S. Park, R. C. Daniels, and R. W. Heath, “Optimizing the target error rate for link adaptation,” in Global Communications Conference (GLOBECOM), 2015 IEEE. IEEE, 2015, pp. 1–6.
[14] R. A. Delgado, K. Lau, R. Middleton, R. S. Karlsson, T. Wigren, and Y. Sun,
“Fast convergence outer loop link adaptation with infrequent updates in steady state,” in Vehicular Technology Conference (VTC-Fall), 2017 IEEE 86th. IEEE, 2017, pp. 1–5.
[15] T. Ohseki and Y. Suegara, “Fast outer-loop link adaptation scheme realizing low-latency transmission in lte-advanced and future wireless networks,” in
[15] T. Ohseki and Y. Suegara, “Fast outer-loop link adaptation scheme realizing low-latency transmission in lte-advanced and future wireless networks,” in