Chapter 6 Performance evaluation
6.2. The Improvement of Maximum Throughput of FA-MAC
On: time:12: this flow should run 12 seconds.
exponential XXX YYY ZZZ: means the interval of packet is an exponential distribution with MEAN=XXX, MAX=YYY, MIN=ZZZ (the unit is micro second).
length: const 1450: the generated packet size is constant 1,450 byte.
Table 6.5 The network flow applied to CRUs CR MAC and Traffic flow CR sender CR receiver
ABi-MAC UDP Flow Greedy UDP flow Greedy UDP flow
ABi-MAC TCP Flow Greedy TCP flow None
Bit Error Rate (BER):
In the above two cases, we also add different bit error rates to each FA-MAC to evaluate the robustness with the influence of different wireless spectra condition. The BER here is defined as the probability of bit error happens after the receiver finishes the channel coding.
Through this evaluation, we can know the reliability of different FA-MACs.
6.2. The Improvement of Maximum Throughput of FA-MAC
We can see in Figure 6.3 that all three kinds of frame aggregation mechanism do improve the maximum throughput of Uni-MAC. First, we look at the maximum throughput improvement of Uni-AMSDU. The throughput increases about 70% (see Table 6.6) with each TXOPCR value. The reason is that with the A-MSDU aggregation, CR sender can transmit at most 2 UDP packet aggregated in one A-MSDU at a time (the A-MSDU maximum size is set to 3839 that can contains up to two 1,450 byte long UDP packets). Second, the result of Uni-AMPDU is different from Uni-AMSDU. When TXOPCR value is one, Uni-AMPDU works same as Uni-MAC but with little additional overhead from A-MPDU operation (BAR
63
and BA). That’s why the throughput of UNI-AMPDU is 1% less the Uni-MAC. The throughput improvement becomes better when TXOPCR increases because Uni-AMPDU can transmit at most TXOPCR data frames in a data transmission round. The bigger TXOPCR value reduces more transmission overhead. Last, one can see from the curve of the improvement of Uni-TLA, which combines the improvement from the above two frame aggregation mechanisms.
The result of TCP flow is shown in Figure 6.4 and Table 6.7. We can see the curves grow almost the same trend as the result of UDP in Figure 6.3. The difference is that the improvement degree of Uni-AMSDU and Uni-TLA become less when TXOPCR increases.
This is because TCP can treat as a two-way asymmetric traffic flow and some of the transmission opportunity is taken by the small TCP ACK packets.
64
Table 6.6 The improvement of Maximum UDP Throughput when different Frame Aggregation Mechanism is applied. (in percentage)
TXOPCR = 1 TXOPCR = 2 TXOPCR = 3 TXOPCR = 4 TXOPCR = 5
UNI-AMSDU 78.70% 70.71% 66.46% 63.78% 61.94%
UNI-AMPDU -1.08% 14.11% 32.32% 47.52% 60.39%
UNI-TLA 76.81% 91.93% 109.48% 123.26% 134.09%
Figure 6.3 Maximum UDP Throughput of Uni-MAC with different Frame Aggregation Mechanism (with UDP payload 1450 bytes)
65
The maximum throughput improvement of ABi-MAC with different frame aggregation mechanism is applied:
To evaluate the maximum throughput of UDP flow of ABi-MAC, we put a greedy UDP traffic flow (with 1450 byte UDP payload) on both side of CRU pair due to the bi-direction bandwidth reservation design. So the maximum throughput is now defined as the sum of received throughput on both side CRU pair in cases 1. Due to the design of A-MPDU over
Table 6.7 The improvement of Maximum TCP Throughput when different Frame Aggregation Mechanism is applied (in percentage)
TXOPCR = 1 TXOPCR = 2 TXOPCR = 3 TXOPCR = 4 TXOPCR = 5
UNI-AMSDU 93.91% 72.43% 57.34% 49.75% 44.60%
UNI-AMPDU -1.30% 26.98% 32.16% 36.13% 39.31%
UNI-TLA 92.33% 83.31% 75.91% 72.26% 69.56%
Figure 6.4 Maximum TCP Throughput of Uni-MAC with different Frame Aggregation Mechanism
66
ABi-MAC in section 5.5 we can only use the TXOPCR value which is multiplies of two. One should note this when comparing the result of Uni-MAC with ABi-MAC.
In Figure 6.5 and Table 6.8, the improvement of A-MSDU is still obvious and the improvement of A-MPDU is also increase with the TXOPCR value. The improvement of ABi-TLA is the sum of the improvement of ABi-AMSDU and Bi-AMPDU.
Table 6.8 The improvement of Maximum UDP Throughput when different Frame Aggregation Mechanism is applied (in percentage)
TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-AMSDU 62.694% 56.302% 52.912% 50.761% 49.014%
ABi-AMPDU -23.364% -1.601% 14.398% 28.116% 38.626%
ABi-TLG 30.250% 54.447% 69.239% 80.636% 88.279%
Figure 6.5 Maximum UDP Throughput of ABi-MAC with different Frame Aggregation Mechanism (with UDP payload 1450 bytes)
67
Figure 6.6 and Table 6.9 shows the maximum throughput of TCP flow. One can see the TCP throughput is better than the TCP throughput of UNI-MAC (in Figure 6.4) with same TXOPCR. This is because of the two-way bandwidth reservation design of ABi-MAC. The design not only reduces the RTT of TCP flow but also reduces the overhead of QP.
The improvement of ABi-AMSDU becomes bigger (from 28% ~ 74%) with the bigger TXOPCR (see Table 6.9) which is different from the result of UNI-MAC (from 93% ~ 45% in Table 6.7). This is because with A-MSDU frame aggregation, many TCP ACKs can aggregates into a single A-MSDU and save the overhead. However, according to the design of Uni-MAC, after TCP sender transmits the TCP packets, the TCP receiver needs an additional CR transmission round to reply the TCP ACK. At this moment, the TCP receiver may note use all the TXOPCR due to the A-MSDU aggregation which aggregates several TCP ACK into an A-MPDU. On the other hand, the TCP receiver can reply the aggregated ACKs in between the transmission of TCP sender’s packet through the design of ABi-MAC that utilize the transmission opportunity in a more efficient way. The same phenomenon happens when A-MPDU is applied, that’s why the improvement of ABi-AMPDU has a bigger improvement with bigger TXOPCR (improves 52% when TXOPCR is 5 and the improvement of Uni-AMPDU is 39% when TXOPCR is 5). Due to the above reasons, the improvement of ABi-TLA is bigger when bigger TXOPCR is applied.
68
In the following section, we add different bit error rate in the simulation to evaluate the performance of FA-MAC’s affected by different bit error rate.
The maximum UDP throughput of ABi-MAC with different bit error rates:
Figure 6.7 shows the application layer throughput of UDP throughput of ABi -MAC with different bit error rate. First, one can see that the performance of FA-MAC degrades slightly when applied BER is 10-6. Second, the throughput of ABi-AMSDU and ABi-TLA degrade
Table 6.9The Improvement of Maximum TCP Throughput when different Frame Aggregation Mechanism is applied. (in percentage)
TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-AMSDU 28.70% 73.09% 77.54% 75.19% 74.21%
ABi-AMPDU -22.78% 9.42% 26.98% 40.56% 52.73%
ABi-TLG 39.36% 93.28% 123.35% 121.58% 131.08%
Figure 6.6 Maximum TCP Throughput of ABi-MAC with different Frame Aggregation Mechanism
69
more than the throughput of ABi-MAC and ABi-AMPDU when BER becomes 10-5. This is because the size of single of frame of ABi-AMSDU and ABi-TLA is bigger than the frame size of ABi-MAC and ABi-AMPDU due to the A-MSDU aggregation mechanism. The bigger MPDU has more probability to encounter the bit error. Once the bit error occurs, the CRU sender won’t get the ACK reply and should hop back to the control channel that hardly decreases the efficiency. On the other hand, the ABi-AMPDU can transmit all the collected MPDUs with A-MPDU mechanism and block ACK even if there is an error occurs in one of the frames. That’s why the degradation of the throughput of ABi-AMPDU is slightly than the throughput of ABi-AMSDU. Last, since the single data frame size is bigger than 10,000 bits (1,500 bytes * 8 = 12,000 bits), all FA-MACs cannot operate when high BER likes 10-4 is applied.
Figure 6.7 Maximum Application Layer UDP Throughput of ABi -MAC with different Bit Error Rates (with UDP payload 1450 bytes, TXOPCR = 10)
The maximum TCP throughput of ABi-MAC with different bit error rates:
Figure 6.8 shows the application layer throughput of TCP throughput of ABi-MAC with different bit error rate. First, one can see that the performance of FA-MAC degrades slightly
0
70
when applied BER is 10-6 just likes the result of UDP throughput above. Second, the throughput degradation of ABi-AMSDU is bigger than ABi-AMPDU due to the big frame aggregated by the A-MSDU aggregation. The reason is same as the UDP case as mention in the previous section. Finally, all FA-MACs cannot operate when 10-4 BERis applied since TCP payload is 1448 bytes long that cannot bear such the high data rate.
Figure 6.8 Maximum Application Layer TCP Throughput of ABi-MAC with different Bit Error Rates (TXOPCR = 10)
6.3. The Effect on Primary Users’ Throughput and Packet Transmission Delay from FA-MAC
In this chapter, we want to see the influence of CRU when it tries to use the channel with other PUs in the neighborhood. We only evaluate ABi-MAC with different frame aggregation mechanism in this case. Through the maximum throughput measurement result of both Uni-MAC and ABi-MAC, we can find out ABi-MAC is still better than Uni-MAC in all respect. Thus, we only use ABi-MAC for further testing.
0
71
We apply three different channel busy degrees to evaluate the influence on PUs as mentioned in sec 6.2. Each PU pair applies 30%, 50%, and 80% of maximum UDP load (with 1,450 UDP payload) on their own channel to represent light, medium, and heavy channel busy degree. With different degree, we can learn the performance of FA-MAC (ABi-MAC applied with different frame aggregation mechanisms in this chapter) more completely. For each load degree, we will show the result of ABi-MAC with different frame aggregation mechanism and different traffic flow (TCP/UDP). The result is discussed in the following.
ABi-MAC with 30% PU load
In Table 6.10, we can see that the throughput of PU still remain the same at about 1,050 KB/s with the influence of CRUs. This result shows that the protection to PUs of ABi-MAC works properly with different frame aggregation mechanism is applied. On the other hand, the throughput of CRU pair still benefit from frame aggregation mechanism as shown in Figure 6.9. First, the improvement of A-MPDU is still obvious when comparing the curves of ABi-MAC and ABi-AMSDU. However, the curves of ABi-MAC and ABi-AMSDU stop growing when TXOPCR reaches four. This is because of the channel vacate mechanism of ABi-MAC. In this case, the average packet generation interval of 30% load is about 1.5 ms (see the setting in Table 6.4). In average, CRU would hear the channel claiming request from PU after transmits two frames in one CR transmission round. This is the reason why the throughput stops growing while TXOPCR is bigger than four.
Second, the improvement of ABi -AMPDU aggregation grows bigger while the TXOPCR is bigger. Due to A-MPDU aggregation and block ACK mechanism, ABi-AMPDU and ABi-TLA can both transmit at most TXOPCR frames in one CR transmission round even when TXOPCR is larger than three. However, this will increase the packet delay of PUs packet when PUs wants to claim the channel but the channel is reserved. On can see average packet delay
72
time in Figure 6.10. Uni-TLA causes the most delay because it would occupied the channel for the longest time with both frame aggregation mechanisms are applied.
The situation is pretty much the same when CRU is applying TCP flow. First, the throughput of PU still remains the same at about 1,050 KB/s with the influence of CRUs in Table 6.11. Second, the curve of ABi-MAC and ABi-AMSDU stop growing at the same TXOPCR value. Last, ABi-AMPDU and ABi-TLA cause the biggest PU’s packet delay time.
73
Table 6.10 The PU’s Throughput of 30% Load with the effect of CRUs (KB/s) TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-MAC 1050.644 1050.461 1050.492 1050.431 1050.644
ABi-AMSDU 1050.461 1050.735 1050.735 1050.431 1050.37
ABi-AMPDU 1050.339 1050.613 1050.613 1050.613 1050.339
ABi-TLA 1050.766 1050.431 1050.37 1050.492 1050.248
Figure 6.9 UDP Throughput of CRU with 30% PU Load
Figure 6.10 PU’s Average Packet Delay Time (30% PU Load)
74
Table 6.11 The PU’s Throughput of 30% Load with the effect of CRUs (KB/s) TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-MAC 1050.49 1050.80 1050.61 1050.64 1050.64
ABi-AMSDU 1050.64 1050.46 1050.77 1050.64 1050.58
ABi-AMPDU 1050.58 1050.49 1050.43 1050.49 1050.43
ABi-TLA 1050.46 1050.46 1050.40 1050.37 1050.34
Figure 6.11 TCP Throughput of CRU with 30% PU Load
Figure 6.12 PU’s Average Packet Delay Time (30% PU Load)
75
Figure 6.13 Application Layer UDP Throughput of ABi-MAC with Different Bit Error Rates (With UDP payload 1450 bytes, TXOPCR = 10)
Figure 6.14 Application Layer TCP Throughput of ABi-MAC with different Bit Error Rates (TXOPCR = 10)
Figure 6.13 and Figure 6.14 shows the UDP/TCP throughput of different FA-AMCs under different wireless spectrum condition. We only show the result when TXOPCR is 10 since each FA-MAC can get the best performance with this setting. On can see the influence to the throughput from different BERs is same as the result in case one. The FA-MACs applies with A-MSDU aggregation suffers more throughput degradation.
0
76
ABi-MAC with 50% PU load
When PUs are creating 50% load, the effect from CRUs is almost the same as PU creating 30% load.
In Table 6.12, we can see that the throughput of PU still remain the same at about 1,841 KB/s with the influence of CRUs. In Figure 6.15, the throughput of CRU pair still benefit from frame aggregation mechanism like the way we mentioned before at section 6.4. First, the improvement of A-MPDU is obvious when comparing the curves of ABi-MAC and ABi-AMSDU. However, the throughput difference between TXOPCR value 2 and 4 is become less than it does in Figure 6.9. This is because the average packet generation interval of 50%
load is about only 0.9 ms (see the setting in Table 6.4). In average, the change that CRU overhears the channel claiming request from PU at the second data transmission round is bigger. This is the reason why the throughput difference between TXOPCR value 2 and 4 is less than 30% PU load.
Second, the improvement of ABi -AMPDU aggregation grows bigger while TXOPCR is bigger and causing bigger PU’s packet delay time. However, maximum throughput is less than the 30% PU load case due to more channel access time is occupied by PUs.
The situation is the same when CRU is applying TCP flow. First, the throughput of PU still remains the same at about 1,841 KB/s with the influence of CRUs in Table 6.13. Second, the throughput difference between TXOPCR values equal to 2 and 4 of ABi-MAC and ABi-AMSDU is less. Last, ABi-AMPDU and ABi-TLA cannot achieve the high throughput as in 30% PU load case.
77
Table 6.12 The PU’s Throughput of 50% Load with the effect of CRUs (KB/s) TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-MAC 1842.053 1841.81 1841.81 1841.901 1841.566
ABi-AMSDU 1841.901 1841.81 1841.81 1841.201 1841.323
ABi-AMPDU 1841.901 1841.901 1841.688 1841.688 1840.197
ABi-TLA 1841.688 1841.688 1840.41 1840.866 1837.731
Figure 6.15 UDP Throughput of CRU with 50% PU Load
Figure 6.16 PU’s Average Packet Delay Time (50% Load)
78
Table 6.13 The PU’s Throughput of 50% Load with the effect of CRUs (KB/s) TXOPCR = 2 TXOPCR = 4 TXOPCR = 6 TXOPCR = 8 TXOPCR = 10
ABi-MAC 1841.749 1841.749 1841.688 1841.536 1841.901
ABi-AMSDU 1841.749 1841.719 1841.262 1841.81 1841.445
ABi-AMPDU 1841.749 1841.81 1841.688 1841.505 1841.171
ABi-TLA 1841.871 1840.653 1841.14 1841.566 1838.705
Figure 6.17 TCP Throughput of CRU with 50% PU Load
Figure 6.18 PU’s Average Packet Delay Time (50% Load)
79
Figure 6.19 Application Layer UDP Throughput of ABi-MAC with different Bit Error Rates (UDP payload 1450 bytes, TXOPCR = 10, 50% PU Load)
Figure 6.20 Application Layer TCP Throughput of ABi-MAC with different Bit Error Rates (TXOPCR = 10, 50% PU Load)
80
In this case, the throughput of PUs is influenced by CRU when any frame aggregation mechanism is applied as shown in Figure 6.21. The throughput is even influenced by ABi-MAC with no frame aggregation mechanism is applied. This is because the average packet interval of 80% PU load is about 500 us. Since we change the sensing time of CRU to 200 us only, it is still possible for CRUs to access the channel. On can see the PU’s packet delay increase about 2 ms. However, this will decrease the throughput of PUs. In both TCP and UDP traffic flow. For the worst case: ABi-TLA causes the biggest influence after TXOPCR
is bigger than 4 due to the Tow Level Aggregation.
81
Figure 6.21 The Throughput of PUs with the effect of CRUs (80% PU Load)
Figure 6.22 UDP Throughput of CRU with 80% PU Load
Figure 6.23 PU’s Average Packet Delay Time (80% PU Load)
82
Figure 6.24 The Throughput of PUs with the effect of CRUs (80% PU Load)
Figure 6.25 TCP Throughput of CRU with 80% PU Load
Figure 6.26 PU’s Average Packet Delay Time (80% PU Load)
83