To evaluate the performance of our approach, we set up a test architecture, as shown in Fig. 10, which include 1 + 1 dispatchers and two active SIP proxy servers (2 + 0). We also used SIPp [14], which is a free open source test tool, as a SIP traffic generator and analyzer. It includes some SIP user agent test scenarios, which were provided by SIPStone [15], a benchmark tool for evaluating SIP proxy server performance. It establishes and releases multiple calls using INVITE and BYE messages. We used default UAC (user agent client) and UAS (user agent server) scenarios which have already been implemented in SIPp to test the proposed design approach.
The test scenario for call setup has been shown in Fig. 2. And in order to obtain the number of failed calls, we turned off the retransmission mechanism in SIPp while doing the following tests.
The flowing five test cases are evaluated:
Case A All dispatchers and SIP proxy servers work well.
Case B One dispatcher fails.
Case C One SIP proxy server fails.
Case D One dispatcher and one proxy server fail.
Case E Run a background application with high CPU loading in a selected SIP proxy server.
The first four cases are used to evaluate if dispatcher and/or proxy server failures affect load balancing. And the last case is to see if the OSLB can handle a surging load.
Fig. 10. Test architecture for OSLB.
We used a metric, Load Balance Metric (LBM) [16], as follows, to analyze the performance of our load balancing method:
∑ ∑
peak _
is the highest load on any server at the jth sampling point;n
is the number of SIP proxy servers.18
We compare the proposed OSLB with the SIP load balancer [12][13], which is also not client and DNS involved, and its failover detection time is much less than that of the directory server [11]. Table 3 shows the test environment and Table 4 is related test parameters.
Table 3. Test environment.
Environment Value
CPU
(of Virtual machines)
AMD Athlon 64 3000+
Memory
(of Virtual machines)
2 Gigabyte
CPU (of SIPp) Intel Core 2 T5500 Memory (of SIPp) 512 Megabyte
Table 4. Test parameters.
Parameter Value
Call rate (default) 100 calls per second
Call rate (failover) 50, 100, 150, 200 calls per second Call duration 3 minutes
Call limit 10000 calls
Parameters in SIP load balancer
Value Sticky table size 100000 Sticky expire time 32 seconds
In the following, the experimental results of each case are discussed:
Case A: All dispatchers and SIP proxy servers work well.
Fig. 11 shows CPU loading of both SIP proxy servers of OSLB for case A, and Fig. 12 shows CPU loading of both SIP proxy servers of the SIP load balancer for case A. We can see that CPU loading of SIP proxy servers of the OSLB is a little higher than the SIP load balancer. The reason is that AMF, provided by OpenAIS, periodical transmits messages between each server to see if any server is down. And we also used CKPT to transmit data to the dispatcher. However, LBM of OSLB and SIP load balancer are pretty close, LBM of OSLB is 1.05 and LBM of SIP load balancer is 1.04.
Case B: One dispatcher fails
Fig. 13 shows the CPU loading of both SIP proxy servers of OSLB for case B. The active dispatcher failed at the 60th second of the testing period and both SIP proxy servers are not affected by the failure of the active dispatcher. LBM of OSLB remains 1.05 as case A. We cannot evaluate the SIP load balancer in this case since it does not support dispatcher redundancy, and it needs a router or gateway to resolve this problem.
Case C: One SIP proxy server fails.
Fig. 14 shows the CPU loading of both SIP proxy servers of OSLB for case C, and Fig. 15 shows the CPU loading of both SIP proxy servers of the
20
SIP load balancer for case C as well. SIP proxy server 1 failed at the 120th second of the testing duration. From Fig. 14, we found that the CPU loading of SIP proxy server 2 has sudden decreasing after SIP proxy server 1 failed.
The reason is that after the dispatcher detects the failure of SIP proxy server 1, it takes few seconds to remove the failed SIP proxy server from the dispatch list. During this short duration, some requests are still forwarded to the failed SIP proxy server 1 and causing call blocking and call failure problems. After the dispatcher removes failed SIP proxy server from the list, CPU loading of SIP proxy server 2 would arise and successful processing incoming requests.
In Fig. 15, after SIP proxy server 1 failed, the CPU loading of SIP proxy server 2 has higher variation than before. In Fig. 16, we also evaluated the number of failed calls with different call rates when one of the SIP proxy servers fails for case C. We found that our approach encountered much fewer failed calls than the SIP load balancer, and have 82 % improvement. This is because that the SIP proxy server failover time of our approach is lower.
Case D: One dispatcher and one proxy server fail.
The active dispatcher failed at the 60th second, and SIP proxy server 1 failed at the 120th second of the testing duration. Fig. 17 shows the CPU loading of both SIP proxy servers of OSLB for Case D. Its like the combination of Case B and Case C, the CPU loading of SIP proxy servers are not affected by the failure of the dispatcher but would be affected by the failure of SIP proxy server.
Case E: Run a background application with high CPU loading in a selected SIP proxy server.
In this case, we ran a background application with high CPU loading in SIP proxy server 1 for one minute during the test period. From Fig. 18 and Fig. 19, we observed that CPU loading of SIP proxy server 2 of both OSLB and the SIP load balancer arising while SIP proxy server 1 has a high CPU loading background application. And Fig. 20 shows the number of failed calls in this test case for case E. The OSLB stopped sending requests to the overloading SIP proxy server before the call-blocking problem became serious. Therefore the proposed OSLB has smaller number of failed calls than the SIP load balancer, and it has 44 % improvement.
0 20 40 60 80 100
0 20 40 60 80 100 120 140 160
Duration (sec)
CPU loading (%)
OSLB_proxy1 OSLB_proxy2
Fig. 11. Case A: CPU loading of OSLB with all servers work well.
22
Fig. 12. Case A: CPU loading of SIP load balancer with all servers work well.
Fig. 13. Case B: CPU loading of OSLB with the failure of one dispatcher.
0
SIP proxy server 1 failed
Fig. 14. Case C: CPU loading of OSLB with the failure of one SIP proxy server.
SIP proxy server 1 failed
Fig. 15. Case C: CPU loading of SIP load balancer with the failure of one SIP proxy server.
24
Fig. 16. Case C: Number of failed calls under different call rates with the failure of one SIP proxy server.
0
Dispatcher failed SIP proxy server 1 failed
Fig. 17. Case D: CPU loading of OSLB with the failure of one dispatcher and one SIP proxy server.
Fig. 18. Case E: CPU loading of OSLB with one SIP proxy server running a high CPU loading background application.
Fig. 19. Case E: CPU loading of SIP load balancer with one SIP proxy server running a high CPU loading background application.
26 0
50 100 150 200 250 300
1 2 3
Duration (min)
failed call
OSLB SIP LB
Fig. 20. Case E: Number of failed calls with one SIP proxy server running a high CPU loading background application.