Chapter 4 Design
4.2 Redundancy Reducing
4.2.1 Detail of Forward-wait Queue Implementation
Basic Scheme
For intuition, fwwQueue can be implemented as an aging and sorted queue. Each forwarding packet associated with an extra delay time is inserted into sorted fwwQueue according to the value of the time tag. Every time unit, fwwQueue decreases each packet’s waiting time tag. Once a packet’s time is counted down to zero, FloodingRD takes this packet out for further processing.
when a packet is inserted into the queue we let packets change their waiting time tag according to packet’ sequence number.
Figure 6 gives a simple example. Packet No.5 with 7 ticks waiting time is inserted into fwwQueue. During the insertion, it meets packet No. 4 with remaining 8 waiting ticks. After exchanging the waiting time tag with packet No. 4, packet No. 5 has 8 waiting ticks and would be sent out after packet No. 4. And then, it meets packet No. 6. Packet No. 5 should be inserted between packet number 4 and packet number 6, because its waiting time is shorter than packet No. 6’s. There is no need to change wait time tags further.
Figure 6. An example of inserting a packet into fwwQueue
4.2.2 Parameters
Because the extra delay time is an absolute protocol overhead, they should be minimized as much as possible. The parameters related to fwwQueue are listed in Table 3. FW_FIXED_DELAY is used to wait for ACK packets sent from destinations.
FW_RANDOM_DELAY defines the range of random delay that is used by
forwarders to compete with each others. The total delay time can be computed as follows:
Extra Delay Time =
FW_FIXED_DELAY + random() % FW_RANDOM_DELAY
FwwQueue Parameters
Name Description
FW_FIXED_DELAY For waiting ACK from destinations FW_RANDOM_DELAY For competing with other forwarders
Table 3. Parameters of fwwQueue
4.2.3 Results
We designed a simple experiment to test the performance of forward-random delay. The topology is shown in Figure 7. There are three forwarders between the source and the destination. The traffic type is greedy UDP. Four daemons with different configuration are tested in this scenario. The briefs of these daemons are listed in Table 4. Configuration details are listed in section 6.2.
The daemon Basic has been used in last experiment in section 4.1.3. The daemon F enables forward-random delay while the daemon RF enables both forward-random delay and retransmission mechanisms. The daemon AT is a FloodingRD with ability of turning parameters automatically. The name of this daemon means “Auto Tuning.”
In this scenario, technology Auto Tuning lets the forwarders compete with each others.
Figure 7. Topology
Symbol Describe
Basic Disable all functionalities F Enable forward-random delay
RF Enable forward-random delay and retransmission AT Turning parameters automatically
Table 4. Types of FloodingRD
Figure 8 illustrates comparison among these four daemons about UDP throughput while Figure 9 shows the condition of redundancy. Daemon F applied forward-random delay but doesn’t reduce the redundancy. Because the configuration was set for daemon RF, 40 ms as FW_RANDOM_DELAY for daemon F was not enough to cancel transmission on other forwarders.
Although daemon RF can reduce redundancy, its UDP throughput was the lowest.
Because of the high collision probability, UDP throughput suffered from retransmission mechanism. When the value of RQ_RETX_TIMEOUT got longer, FloodingRD should wait longer for a lost packet and the circumstance would become worse.
Here we provide two related experiment result as contrasts to Figure 8. Daemon Basic can get average throughput of 177.60 Kbytes/s under the same environment without additional forwarders, while daemon AT can reach 226.65 Kbytes/s.
According to these experiment results, we can conclude that Auto Tuning indeed reduce redundancy and the additional overhead.
UDP Throughput
decrease throughput of whole system. To reduce the influence of these small packets, the designers merged these small packets with large data packets or combined them together into a large one.
Although our early study proved its potency, to minimize variable factors during developing period, this functionality has been disabled for a long time.
4.4 Issues about Parameters
4.4.1 Components Overview and Relationship between Parameters
Before discussing about the parameters, the overview of components and relativity among them should be given firstly. As shown in Figure 10, there are two kinds of data packets: the packets from other hosts are “forwarded packets,” while the others are called “local-out packets.” In addition to data packets, there are control messages transferred in our system. At present there are only two types of control message: de-retransmission message and ACK packets.
Figure 10. The overall structural view inside FloodingRD
Forward-wait Queue delays forwarded packet for waiting ACK packets from destinations and preserves them for a random period for redundancy reduction.
Because local-out packets need not be applied extra delay, they will be put into retxQueue immediately.
RetxQueue is responsible for retransmitting lost data packets. Users can configure its length and the maximum retransmission times as well as the period for waiting ACK packets. This functionality is designed for sensitive protocols, such as TCP. However, throughput of greedy UDP will suffer from the waiting time of lost packets. Increase of retransmission times (RQ_RETX_TIMES) and extension of waiting period (RQ_RETX_TIMEOUT) influence greedy UDP throughput seriously.
Fortunately, according to experiment results shown in section 4.1.3, retransmitting lost packet once is effective. Moreover, doubling retransmission times can not improve TCP performance further.
The minimum valid value of RQ_RETX_TIMEOUT is mainly affected by processing time of the next host. The related factors are operating speed and other parameter configurations of FloodingRD. The operating time of a machine is influenced by CPU speed, NIC efficiency, and the traffic load as well. Parameters that influence how long FloodingRD would preserve a packet include forward-fixed delay, forward-random delay, length of retxQueue, size of sliding window and other
Extending the length of retxQueue and the window size may adapt FloodingRD to high-variant traffic. But now there is no practice experiment result to certify this.
Moreover, increasing these parameters will decrease greedy UDP throughput.
At the bottom, there is a “Merger” used to merge small packets into a large one.
This functionality is disabled for a long time. RQ_RETX_TIMEOUT must be extended for this functionality, because the merger will preserve packets for a short time.
4.4.2 How to Decide the Value of a Parameter
There are two methods to decide the value of a parameter. One is to evaluate it directly by practical experiments, and another is to deduce it by other related parameters’ values and the logical relationship between them. Although practical experiments can reach an accurate result, the perfect settings would lose efficacy after environment changed. Theoretical values can help developers to discover faults inside implementation. If there are conflicts between the practical results and the logical values, developers must examine the design and implementation carefully.
4.4.3 Recommended Value for Main Parameters
Table 5 lists proposed values of main parameters. Settings of fwwQueue are evaluated by practical experiments under greedy UDP traffic. Configuration of retxQueue considers both UDP traffic and TCP traffic. RQ_RETX_TIMEOUT is a logical value inferred by the setting of other parameters. The time unit is millisecond.
Parameter Value Parameter Value
RQ_LENGTH 5 RQ_RETX_MAX_COUNT 1
RQ_WINDOW_SIZE 3 FW_FIXED_DELAY 8
RQ_FAST_RETX_THRES 2 FW_RANDOM_DELAY 40
RQ_RETX_TIMEOUT 80
Table 5. Reference value of main parameters
4.5 Tuning Parameters Automatically
4.5.1 Introduction
As aforementioned, the performances of a network are influenced by many factors, including the hardware properties, the network load, as well as the values of various system parameters. Automatically adjusting system parameters according to the network conditions usually results in better performances than fixing values of those system parameters. Such a technique is called “Auto Tuning.” At present, Auto Tuning can tune the retransmission waiting time to an appropriate value dynamically and prevent unnecessary extra forward delays.
4.5.2 Appropriate Retransmission Waiting Time
FloodingRD makes use of forwarded packets from intermediate nodes and ACK control messages from destination nodes as acknowledgements of transmitted packets.
If the next hop of a packet is the destination node for that packet, the RTT (Round
According to the analyses of the experiment results, we select the double time of RTT as the basis of RQ_RETX_TIMEOUT. A small constant value is added to prevent from unnecessary retransmissions. For a node, the measured RTT of a packet is updated each time when its corresponding ACK returns.
4.5.3 Reasonable Forward-delay time
Section 4.2 describes how an extra forward delay helps reduce redundancy. The forward-fixed delay is used to prevent forwarders from sending unnecessary packets that can be canceled by ACK messages from the destination node. However, this delay time is purely an overhead for the forwarders far away from the destination node. Similarly, the forward-random delay is also unnecessary for forwarders that have no contenders aside them.
Figure 11 shows a simple topology that depicts all kinds of situations for forwarders. The light-blue nodes marked with “S” or “D” stand for the source and the destination nodes, respectively. Other nodes marked with different numbers represent different types of forwarders. FloodingRD in a forwarding machine analyzes incoming packets and messages during a unit period (currently, the unit period is set to one second) to decide which type of forwarders it is. Note that a FloodingRD may play different types of forwarders for different connections that have different source and destination nodes, individually. As such, a forwarder is able to apply reasonable extra forwarded delays to packets that to be forwarded based on which forwarder type it is for a connection.
Figure 11. A simple topology to show different types of forwarders
Type-1 nodes represent forwarders that are in the direction opposite to the destination node. There is a common feature among these nodes: the rate of the successful packet forwarding is less than other types of nodes. On these nodes, retxQueue would be blocked and therefore reduce the waste of bandwidth. More bandwidth can be further saved by extending FW_FIXED_DELAY to aggravate the blocking of retxQueue. Rather than cancel all transmissions on these forwarders, this method consumes little bandwidth but recovers regular functionality more quickly.
A type-2 node stands for forwarders who should not apply any extra forward delay since the destination node is far away from it, and there is no other forwarder nearby.
Type-3 forwarders can notice the existence of each other by receiving duplicate forwarded packets. If another nearby forwarder exists, these adjacent forwarders begin competing the forwarding of a packet. Firstly, each one selects FW_RANDOM_DELAY in a predefined range. Then, the forwarder with smallest random delay will cancel the forwarding of others and become the winner who takes
Type-4 nodes are the nodes that are next to the destination node. A type-4 node has to apply the fixed delay to wait for the acknowledgements of the forwarding packets. If it detects that the fixed delay is not long enough, the delay time should be extended.
Because determining the forwarder type of a node must collect the required information during a unit period, the reaction time is not as instant as the former one.
Besides, since the measurement of those RTTs require the transmissions of data greedy UDP throughputs, TCP throughputs, and latencies.
Table 6 lists FloodingRD with different configurations used in these experiments.
The RF daemon is provided as a contrast, which enables all functionalities with fixed parameter settings. The AT_RTT daemon modifies RF to determine the waiting period for an ACK (by the parameter RQ_RETX_TIMEOUT) dynamically. AT_FFD determines the value of FW_FIXED_DELAY by the number of ACK packets from the destination node. The AT_FRD daemon extends the value of FW_RANDOM_DELAY by detecting the existence of other forwarders aside it. The AT daemon enables all functionalities described above.
Symbol Describe
RF Enable forward-random delay and retransmission AT_RTT Decide the waiting period for an ACK dynamically AT_FFD Decide the value of FW_FIXED_DELAY automatically AT_FRD Decide the value of FW_RANDOM_DELAY automatically AT Enable all Auto Tuning functions
Table 6. Types of FloodingRD
Because Auto Tuning can change the value of RQ_RETX_TIMEOUT to fit the requirements of different network environments, the value of this parameter is reduced to be 40 instead of the original value 80. However, this value is similar to that used by AT_RTT, and the performances of daemon AT_RTT are therefore similar to those of daemon RF.
Figure 12 presents the average throughput for greedy UDP connections in a chain network with 2 nodes to 5 nodes. In 2-node cases, only the AT_RTT and AT daemons raise the throughput performance. Because the extra forward delay mechanism only takes place on forwarders, it can not provide any improvements when no forwarders exist in 2-node cases. Generally, the reduction of the values for the FW_RANDOM_DELAY parameter can improve the average throughput more than that for the RW_FIXED_DELAY parameter. Besides, the AT daemon can double the average throughput in 3-node cases. These improvements become less and less significant while the number of nodes increases.
Intuitively, these mechanisms only decrease the latencies; how can they improve
Auto Tuning, the RTT time will be decreased by taking off the extra unnecessary forward delay overhead, and the retransmission timeouts can therefore be reduced further and raises the UDP throughputs.
UDP Throughput
Figure 12. Average UDP throughput to compare functions of Auto Tuning
UDP Throughput
Figure 13. Average Latency under one-way greedy UDP
Figure 13 presents the measured latencies in the same experiment settings with Figure 12. These results prove that Auto Tuning functionalities indeed reduce the latencies. On the other hand, the results also provide an evidence of the relationship
between latency and UDP throughput in our scheme. We can observe that the daemon, which can reduce latency, can raise UDP throughput as well.
The average RTT time is measured on the source node. Figure 14 organizes these data in the same experiment settings and provides another evidence of the relationship between latency and UDP throughput.
Figure 14. Average RTT time at the source under one-way greedy UDP
Figure 15 illustrates the average TCP throughputs in the same topology. Unlike UDP, the throughput of TCP is limited by the protocol design rather than our retransmission scheme. Therefore, the reduction of latencies influences TCP performance slightly.
Figure 15. Average TCP throughput in a chain network
Chapter 5 Implementation
Our system is built on Linux platform. Chapter 3 has described the architecture of our scheme. The related issues and the existing technologies used for implementation are given here.
5.1 Packet Filter
There are lots of existent net filter technologies for the packet redirection.
“Netfilter” in Linux and “divert socket” in FreeBSD are both famous tools to intercept the desired packets. Generally, a net filter is used to build the internet firewalls or to apply the translation of network address and port (NAT). We used netfilter and related technologies such as iptables and libipq library to intercept the packets passed by the kernel.
Netfilter and iptables are building blocks of a framework inside the Linux 2.4.x and 2.6.x kernel. Netfilter can be considered as a set of hooks inside the Linux kernel that allows the kernel modules to register callback functions with the network stack.
Iptables provides a set of commands as an interface between user and netfilter. Users can use iptables commands to tell netfilter what to do with the passing packets.
Netfilter may buffer packet a special queue, and libipq is a development library that
The following example is the configuration that we used on Linux for the field trials.
iptables -N flood_buffer
iptables -A OUTPUT -d 1.0.1.0/24 -j flood_buffer iptables -A flood_buffer -d ! 1.0.1.n -j QUEUE
Our experiments took the subnet 1.0.1 (netmask 0xffffff00) as the original network domain and added a new subnet 1.0.2 for FloodingRD. The number “n” is an identified id of a host. The first rule defines a new queue named flood_buffer. The second rule lets all normal unicast packets be placed in flood_buffer queue. Finally, the last rule inspects the packets in flooding_buffer queue. If a packet is for the local host, it will be passed through to avoid infinite loops. Other packets will be collected into a predefined queue provided by Netfilter, and FloodingRD can receive packets from this queue by the functions defined in libipq library.
To make a NIC receive or send packets with two different network subnets, a common technology, “IP aliasing”, is used to configure the NIC with two IP addresses.
Typical uses of IP aliasing are virtual hosting of Web and FTP servers and reorganizing servers without having to update any other machines. The following commands can make a NIC simultaneously receive packets from 1.0.1.x and 1.0.2.x.
ifconfig wlan0 inet 1.0.1.n netmask 0xffffff00 ifconfig wlan0:0 inet 1.0.2.n netmask 0xffffff00
5.2 Limitations of Current NICs
To be backward with earlier 802.11 products, the broadcast transmission rate is fixed at 2Mbps rather than 11Mbps for unicast packets. This feature limits the maximum throughput of our scheme. To enhance the transmission rate of broadcasts,
some modifications for the drivers of NICs are required. Fortunately, there are still some NICs that provide the ability for configuring the transmission rate for broadcast.
Thanks to the existence of this kind of NICs, we can save efforts to modify the drivers.
Chapter 6 Performance Evaluation
Functionalities of FloodingRD and their efficacy have been demonstrated in Chapter 4. However, these experiment results are generally for a specific purpose.
What we want to present in this chapter are the integral performances, such as the maximum throughput, the latency and the packet loss rate for different traffic types.
At the beginning, an introduction of various performance metrics is given.
Section 6.2 briefs the environment of these experiments. Section 6.3 shows the practical results. Because the circumstances of large scale networks are hard to get by real experiments, simulation is used to help us observe the operation of our scheme in large networks. The related results are given in section 6.4. At the end of the simulation section, there is an evidence for the ability of FloodingRD to get adapt to high mobility environments
6.1 Performance Metrics
While developing a routing protocol, how can we judge it is good or bad? What are the important features of a good routing protocol? Here we define some performance metrics by that we can quantify and measure the efficiency of a routing daemon. These metrics include the average throughput, the packet loss rate, the end-to-end latency as well as the redundancy degree, etc.
6.1.1 Throughput
The traffic generators normally record incoming throughput per second, and
write these records into a log file. By analyzing these log files, we can get the average throughput easily.
6.1.2 End-to-End Latency
Latency is defined as the period from the time a packet is sent out from the source and the time it is received by the destination. A short latency is required by video and audio communication services. The stg/rtg programs, a pair of traffic generators, record the delay time of each received packet. However, this mechanism needs to synchronize the system time on the source node and the destination node.
Even after correcting the system time on hosts by time services, the difference between each host is still too large to provide accurate latency time.
Even after correcting the system time on hosts by time services, the difference between each host is still too large to provide accurate latency time.