Chapter 4. Network Topology Design
4.7. Performance Evaluation
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
64
hop count limit constraints. Constraints 4.27, 4.28 and 4.29 represent the limits of the number of CRPs and the number of antennas in each CRP. Constraints 4.30 and 4.31 indicate that the pivots nodes should have Q disjoint outgoing paths. Constraint 4.32 indicates the cross network constraint. Constraint 4.33 becomes effective when G’ is a tree-type topology.
In this section, we propose a transformation methodology that transforms the CCN CNTD problem to a BILP problem which can be solved using any binary linear programming algorithm. This transformation methodology is not only applicable to CCN network topology design, but also applicable to other optimization problems. One major benefit of this methodology is that there exists some commercial software such as MatLab such that even non-professional programmers can easily solve the problems. Pseudo code of binary integer linear programming algorithm is listed in Appendix II.
4.7. Performance Evaluation
We conduct several simulation based experiments to evaluate our TDHA against optimal solutions obtained from BILP. The experiment objectives are (1) to analyze the characteristics of CCN topology design problems, (2) the performance of TDHA, (3) the availabilities of multiple path network topologies and (4) depth bound vs. depth weight analysis. Three experimental results analysis, characteristics of CCN network topology problems, the performance of THDA and availabilities of multiple path network topologies, are proposed in this section.
Table 4.2 shows input parameters for test case generation and evaluation metrics. Testing
‧
Cross FT, MPFN and Cross MPFN problems to generate heuristic solutions. Algorithm BILP is applied to solve Simple FT and Cross FT problems to generate optimal solutions. Since BILP needs excessive long computing times to solve MPFN and Cross MPFN problems when the size, of given graph, G, is greater than 100, we used the Topology Random Search Algorithm (TRSA) [22] to find the pseudo-optimal solutions of them. TRSA generates one million solutions randomly and chooses the maximum solution as the pseudo-optimal solution.The pseudo-optimal solutions are used to analyze the chrateristic of CCN CNTD problem and to evaluate the performance of TDHA when optimal solutions are not available.
TABLE 4.2. Parameters for experiment setup
Input parameters Evaluation metrics
Network profit and reliability of pivot nodes are showed in Table 4.3. Unit profit gain is a normalized metric used to examine the characteristics of generated topologies. Deviations of optimal and
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
66
pseudo-optimal profits are applied to evaluate the performance of TDHA against the optimal and pseudo-optimal solutions, respectively.
TABLE 4.3. Evaluation metrics
Metrics Definition
Unit profit gain the total disaster response profit the number of CRPs Deviation of optimal
profit 1 −heuristic solution
optimal solution
Deviation of
pseudo-optimal profit 1 − heuristic solution pseudo optimal solution Reliability of pivot
nodes, Rel(Ф)
Rel (Ф) = Rel (φ1∩φ2∩φ3∩…) ---(4.34)
The reliability of pivot nodes, Rel(Ф), is the probability that all pivot nodes are funtional.
Rel(Ф) is equal to Rel(φ1∩φ2∩φ3∩…) for all pivot nodes φi and applied to evaluated the reliability of the network topology.
TABLE 4.4. Environments of experiments Software
MATLAB R2012a Hardware
CPU Intel® Core™ i3 2.93GHz RAM 4 GB
OS Windows 7
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
67
The specifications of hardware and software are showed in Table 4.4. The test cases are evaluated by simulation using MATLAB on a regular PC.
(a) Experiment 1: Characteristics analysis of CCN network topology problems.
The characteristics of generated topologies are examined by observing the influence of the number of survival nodes |S|, pivot nodes |Ф|, pivot paths |Q| and length bound of pivot paths
|U| on the unit profit gain, which is the profit gain per CRP, of both optimal and pseudo-optimal solutions.
The characteristics of Simple FT and Cross FT are showed in Fig.4.9. The x-axis is the number of survival nodes/number of nodes/solutions and pivot nodes and the y-axis is the unit profit gain. The maximum unit profit gain is 86.2 when |S|=3, |V|=5 and network topology type is Cross FT. The minimum unit profit gain is 76.2 when |S|=2, |V|=200 and network topology type is Simple FT. The unit profit gains of Cross FT are better than Simple FT’s. The unit profit gains of Cross FT and Simple FT are 81.1 to 79.5, respectively. The unit profit gains are 79.4 and 81 when the numbers of survival stations, |S|, are 2 and 3, respectively.
Besides, the profit gain decrease as the value of |V| increases.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
68
Figure 4.9. Characteristics of Simple and Cross FT (optimal solution)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
69
Figure 4.10. Characteristics of MPFN and Cross MPFN (number of pivot nodes/paths) The characteristics of MPFN and Cross MPFN are showed in Fig. 4.10. The x-axis is solutions/number of pivot nodes and the y-axis is the unit profit gains. Shapes ○, □ and + represent |Q| = 1, 2 and 3, respectively. The maximum unit profit gain is 75.1 when |Ф|=1,
|Q|=1 and network topology type is Cross MPFN. The minimum unit profit gain is 63 when
|Ф|=8, |Q|=3 and network topology type is Cross MPFN. Increasing the values of |Ф| and |Q|
will cause the loss of profit. The length of the box in Fig. 4.10 represents the profit loss when the value of |Q| increase. The average length of box in Cross MPFN is longer than that in MPFN.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
70
(b) Experiment 2: Performance analysis of THDA.
Figure 4.11. Performance of TDHA (deviation of optimal profits)
The performances of TDHA in solving Simple FT, Cross FT, MPFN and Cross MPFN are shown in Fig. 4.11. The deviation of optimal profits of Simple FT, Cross FT, MPFN and Cross MPFN are 5.02%, 6.75%, 9.03% and 17.98%, respectively.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
71
Figure 4.12. Performance analysis of TDHA (Simple and Cross FT)
The performance of TDHA in solving Simple FT and Cross FT is showed in Fig. 4.12. The deviation of optimal profits range between 0 and 1. The smaller the value is, the better the performance is. The minimum deviation of optimal profit is 1.71% when |S|=2, |V|=200 and network topology type is Cross FT. The maximum deviation of optimal profit is 10.94% when
|S|=3, |V|=50 and network topology type is Simple FT. The average original deviation of profit of Cross FT is greater than that of Simple FT. The profit deviations of Cross FT and Simple FT are 6.96% and 5.02%, respectively. The profit deviation are 4.46% to 7.369%
when |S|=2 and |S|=3, respectively.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
72
Figure 4.13. Performance analysis of TDHA (MPFN and Cross MPFN)
The performance of TDHA in solving MPFN and Cross MPFN is showed in Fig. 4.13.
Because the heuristic solutions are greater than the pseudo-solutions in most test cases, the values of deviations in Fig. 4.13 are negative. The smaller the the deviation is, the better the performance is. The minimum deviation is -15.16% when | Ф |=6, |Q|=2 and network topology type is Cross MPFN. The maximum deviation is 3.78% when |Ф |=8, |Q|=3 and network topology type is Cross MPFN. The average deviations of MPFN and Cross MPFN are -5.42% and -3.76%. Although, the performance of TDHA in solving Cross MPFN is better than MPFN, the performance of TDHA is more stable in solving MPFN. The average
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
73
deviations are -4.44% and -4.86% when |S| is 2 and 3. The average deviations are -4.98% and -4.43% when |U| is 4 and 6. The number of survival nodes and length bound of pivot paths do not have an obvious influence on the performance of TDHA.
(c) Experiment 3: Reliability analysis of multiple path network topology
Figure 4.14. Reliability of multi-path network topology (box plot)
The effectiveness of multiple path on the network reliability is showed in Fig. 4.14. The x-axis is the failure rate of edges and the number of pivot nodes. The y-axis is the network
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
74
reliability. The lengths of boxes represent the effectiveness of multiple path on the network reliability. The network availabilities are significant improved by using multiple path especially when the number of pivot nodes grows.
Although multiple path can improve the network reliability, it may cause profit degradation.
Therefore, it is important to balance the network reliability and total profit. The relationship between the network reliability and unit profit gain of heuristic solutions are showed in Fig.
4.15. As we can see from Fig. 4.15, the solutions obtained by the TDHA have very high network reliability but with nominal profit degradation.
Figure 4.15. Network reliability and unit profit gain
(d) Experiment 4: Reliability analysis of multiple path network topology
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
75
Figure 4.16. Depth weight based vs. depth bound based
Comparisons of the unit profit gain of depth weight based and depth bound based is showed in Fig. 4.16. The x-axis is the unit profit gain and the y-axis is the average hop count of network topologies. The profit gains of depth weight based are better than depth bound based in most cases. The hop count of depth weight based is also greater than depth bound based in most cases. It means that depth weight based is more aggressive and probe deeper than depth bound based.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
76