CHAPTER 4 EVALUATION
4.3 E XPERIMENTAL RESULTS
4.3.1 Longest Path Reliability
Figure 6(a) shows the longest path reliability with different broadcast data size on the triangle mesh. The reliability of the simple flooding, domain pruning, and Wu and Li algorithms are degraded about 18~20% as the packet size increasing. This is reasonable because a lengthy packet has higher probability to collision with others when transmitting on the air. On the other hand, the degradation of reliabitliy is relative small in ECB and SBA algorithms because of the delay mechanism which result in a staggered transmission and ligher collision probability. Besides, the dynamic probabilisitc algorithm also suffers from the enlargment of packet size. It is also reasonsable because the reliability of this algorithm is much lower than the one of others. Therefore, the collision probability of the dynamic probabilisitc algorithm is also lower than the probability of other algorithms. The reason of the lower reliability coming comes from the aggregation of the increased hop count. Apprarently, the retransmission probabilities for both of the dynamic probabilistic and ECB algorithms is exponentally decreasing as the hop counts. Figure 6(b) verifies our explanation by showing per-hop reliability. The per-hop reliability decreases dramatically for the dynamic probabilistic and ECB algorithms, but it is more steady for other algorithms.
Figure 6(c) and 6(d) show the experimental results of longest path reliability on the 2D mesh and 3D mesh respectively. The results are similar to the case of the triangle mesh. For these reasons, it is recommended to adopt the delay-based algorithm for transmitting a service-discovering broadcast message. It is also recommended to avoid using the probabilistic-based algorithm which leads to the degradation of per-hop reliability.
18
4.3.2 Average Reliability
Figure 7 shows the average reliability, which is the mean reliability of all mesh nodes, for different topologies. According to the results, simple flooding, SBA, and the dynamic probabilistic algorithms are more independent on the topology change.
Thus, the simple flooding and SBA outperform than other algorithms, and the dynamic probabilistic algorithms are worse than others. Due to lighter collision mentioned in the previous section, the average reliability of SBA is slightly better than simple flooding especially for larger size of data. Overall, the SBA shows the best reliability among all algorithms, which is much different from the previous simulations in [21-22]. The reasons are that the delay-based mechanism allieviates the collection problem, and the remaining list after the observation phase enhances reliability.
(a) Longest path reliability on the triangle mesh (b) Per-hop reliability on the triangle mesh
(c) Longest path reliability on the 2D mesh (d) Longest path reliability on the 3D mesh
Figure 6 Details of Reliability
19
.
4.3.3 Forwarding Ratio
The forwarding ratio is the re-transmission ratio of a received broadcast frame.
Figure 8(a) shows the average forwarding ratio for different size of broadcast on a triangle mesh. Apparently, the packet size is not a factor of the forwarding ratio. The results on a 2D and 3D mesh, shown in Figure 8(b) and Figure 8(c) respectively, are also compliant with the same observation. Through different topologies, we find that the average forwarding ratio of simple flooding (100%), dynamic probabilistic algorithm (50%), and ECB (70%) are not influenced by the topology. However, the topologies is the major factor that influences the forwarding ratio of the algorithms using neighbor information. In particular, the Wu and Li algorithm varies most (20%
to 100%) among all algorithms, because a mesh node using Wu and Li is easy to become a gatway node by as judging the disconnectivity of its 1-hop neighbors. Thus, the Wu and Li algorithm would act as flooding in a non fully connected topology.
(a) Average reliability on the triangle mesh (b) Average reliability on the 2D mesh
(c) Average reliability on the 3D mesh
Figure 7 Average Reliability
20
4.3.4 Broadcasting Efficiency
In order to evaluate the efficiency of a broadcast algorithm, we define an index, broadcasting efficiency, which considers both the forwarding efficiency and the reliability factors to express the contribution of each forwarding frame to the reliability. To explain our definition, some variables are defined here:
N(p): number of neighbors for node p;
R(p): observed number of effective broadcasts received by node p;
T(p): expected number of forwarding times issued by node p;
PER: the packet error rate.
The main idea of the index is the result of multiplying the average reliability by a factor which represents the forwarding efficiency. To evaluate the forwarding efficiency, we define the ‘merit’ of a forwarding, i.e., R(p), which counts the number of effective (correctly and non-duplicated) broadcasts. Therefore, of the higher merit an algorithm is, the more efficient it is. By taking the PER into consider, finally, the forwarding efficiency is the number of observed effective broadcasts divided by the
(a) Avg. Forwarding Ratio on the triangle Mesh (b) Avg. Forwarding Ratio on the 2D Mesh
(c) Avg. Forwarding Ratio on the 3D Mesh
Figure 8 Average Forwarding Ratios
21
number of expected received broadcasts for each algorithm, i.e., the number of expected broadcasts is contributed from 𝑇𝑇(𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠) × 𝑁𝑁(𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠) × 𝑃𝑃𝑃𝑃𝑃𝑃 for the broadcast sender, and 𝑇𝑇(𝑝𝑝) × (𝑁𝑁(𝑝𝑝) − 1) × 𝑃𝑃𝑃𝑃𝑃𝑃 for other nodes. Thus, the definitions of forwarding efficiency and broadcasting efficiency are listed as follows:
𝐹𝐹𝐹𝐹𝑠𝑠𝐹𝐹𝐹𝐹𝑠𝑠𝑠𝑠𝐹𝐹𝑠𝑠𝐹𝐹 𝑃𝑃𝐸𝐸𝐸𝐸𝐹𝐹𝐸𝐸𝐹𝐹𝑠𝑠𝑠𝑠𝐸𝐸𝐸𝐸 = ∑𝑠𝑠𝐹𝐹𝑠𝑠𝑠𝑠𝑠𝑠 𝑃𝑃(𝑝𝑝)
𝑇𝑇(𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠) × 𝑃𝑃𝑃𝑃𝑃𝑃 + ∑𝑠𝑠𝐹𝐹𝑠𝑠𝑠𝑠𝑠𝑠 𝑇𝑇(𝑝𝑝) × (𝑁𝑁(𝑝𝑝) − 1) × 𝑃𝑃𝑃𝑃𝑃𝑃 𝐵𝐵𝑠𝑠𝐹𝐹𝐹𝐹𝑠𝑠𝐸𝐸𝐹𝐹𝑠𝑠𝐵𝐵 𝑃𝑃𝐸𝐸𝐸𝐸𝐹𝐹𝐸𝐸𝐹𝐹𝑠𝑠𝑠𝑠𝐸𝐸𝐸𝐸 = 𝐹𝐹𝐹𝐹𝑠𝑠𝐹𝐹𝐹𝐹𝑠𝑠𝑠𝑠𝐹𝐹𝑠𝑠𝐹𝐹 𝑃𝑃𝐸𝐸𝐸𝐸𝐹𝐹𝐸𝐸𝐹𝐹𝑠𝑠𝑠𝑠𝐸𝐸𝐸𝐸 × 𝐴𝐴𝐴𝐴𝐹𝐹. 𝑃𝑃𝑠𝑠𝑅𝑅𝐹𝐹𝐹𝐹𝑅𝑅𝐹𝐹𝑅𝑅𝐹𝐹𝐵𝐵𝐸𝐸
Based on the index, a broadcast algorithm is more efficient if both the average reliability and forwarding efficiency are higher in the meantime. Thus, the higher index value a broadcast algorithm shows, the more efficient the broadcast algorithm is.
The broadcasting efficiency results of each algorithm are drawn in the Figure 9.
We can find that the DP algorithm outperforms than all other algorithms not only for different size of broadcast data but also for different topologies. By checking the Figure 7 and Figure 8, we can know that its advantage comes from the low forwarding ratio which is the result of gateway selection, which effectively reduces the number of forwarders. On the other hand, the efficiency of another gateway selection algorithm, the Wu-and-Li algorithm, degrades to the worst algorithm, the simple flooding, in the 3D mesh topology. The reason is that the Wu-and-Li algorithm statically determines whether it should be a gateway node by checking the local-CDS, but the gateway node is run-time decided by the sender in the DP algorithm. The local-CDS in the 3D mesh topology contains no fully connected local topology, so all mesh nodes become forwarders, which leads to the Wu-and-Li algorithm degrades to the simple flooding algorithm. Figure 8(c), where the forwarding ratio of the Wu-and-Li algorithm is 100%, also verifies the conclusion.
In addition, although the SBA is best at both of the longest path reliability and average reliability as shown in previous section, its broadcasting efficiency is not the best one among all algorithms. Conceivably, the forwarding method is a little more
22
inefficient in SBA, which results in worse broadcasting efficiency than the index of DP. Besides, the results of the simple flooding, dynamic probabilistic algorithm and ECB are all worst and similar when using them under the same topology.