Numerical Analysis and Simulation Results
5.2.2 Finding optimal policy by value iteration method
(
1
n v p
q N j
j k ij k
i
∑
=
+
this is because we want to make our reward the bigger the better. But here the value represents the total expected network cost, so we should take minimum instead of taking maximum. In the simulation we assume average life time λ is equal to 30 seconds and the time τ is equal to 2 seconds for smoothing the DCP handover. We also assume that we can get the transition probability from user profile and the connection cost for each link is reserved for mobile operator to define. Therefore, we can calculate the expected total cost
v
i(n) for each state.Set C(L1) C(L2) C(L3)
A 1 1 1 B 3 2 1 C 4 3 8 D 2 7 1
Table 5.4. Different sets of connection cost
5.2.2 Finding optimal policy by value iteration method
In order to simulate our problem, we set the transition probability arbitrarily, P1 = 0.5, P2 = 0.5, P3 = 0.4, P4 = 0.6, P5 = 0.1, P6 = 0.9, and define different sets of connection cost for each link in the table 5.4. In the real case, we can get these probabilities form user profile and the cost of link will be defined by mobile operator. The expected total cost diagram is shown in figure 5.9; for different sets we can find the best policy in the fourth stage through decision matrix shown in figure 5.10.
Figure 5.9_(a) Expected total cost for set A
Figure 5.9_(b) Expected total cost for set B
Figure 5.9_(c) Expected total cost for set C
Figure 5.9_(d) Expected total cost for set D
⎥⎥
Figure 5.10. Decision matrix
Compare the decision matrix (a) with (b), we can find that the optimal policy is different in state 8, state 13, and state 15. When C(L1) =3, C(L2)=2, and C(L3)=1, state 8 will take the decision to alternative state 9, since the expected immediate cost q89 which is equal to 2C(L2)*τ = 16 is smaller than q84 which is equal to [C(L1) +C(L2)]*τ = 20. State 13 will take the decision to alternative state 14 because of the same reason. State 15 will take the decision to alternative state 12, since the expected immediate cost q1512 which is equal to [C(L2) +C(L3)]*τ = 12 is smaller than q157 which is equal to [C(L1) +C(L3)]*τ = 16. There are two more sets (c) and (d), we can explain the decision for each state by the same way.
For different connection link cost we will have different optimal policy, the operator can decide when to take a DCP handover depending on the various connection models, and minimize the network cost for seamless vertical handover.
If changing the average life time λ, it will affect the stage we need to converge to the best policy. It takes at most 10 stages to obtain convergence when the average life time λ = 1. Since the smoothing time τ is equal to one, the expected immediate cost for each alternative state is much closer, and it will take more stages to reach convergence. For most case it only takes four stages to converge, the computation is not an issue. We show this in the figure 5.11.
Figure 5.11. Numbers of stage over λ
Chapter 6 Conclusion
In this work we discuss the problem of minimizing the network cost of seamless handover for real-time service from the network architectural aspect. It identifies that the QCS-QNS negotiation is expected to be the major factor contributing to the latency of vertical handover. To overcome this bottleneck, we introduce the concept of Designated Crossover Point (DCP) to reduce the latency.
By loosely-coupled interworking, VPN connection model, and Mobile IP, we can integrate UMTS and WLAN together and pre-allocate pipes between them to supporting DCP handover which will achieve lower cost for network operator.
Our system architecture bases on the existing 3G UMTS and WLAN infrastructures, therefore, it is a viable scheme.
In our simulation, we can find the dwell timer µ and according to the timer value to perform the DCP handover. Here, we use simplified rule [17], therefore, the timer is either zero or infinity. If considering one UMTS and one WLAN, we just choose one strategy from table 5.1 as our optimal policy. If considering one UMTS and two WLAN, by using the value iteration method of Markov decision process and the VHE user profiles defined by 3GPP, in which we can find the mobility characteristic, we can calculate the network cost and derive the optimal policy that will minimize total expected network cost. That means, the network operator can select the best connection model to supporting the seamless vertical handover for real-time service by spending the least network resource, on the other hand, the user will continue his/her ongoing sessions without interrupting.
Both customers and operators will benefit from our proposed scheme.
Reference
[1] Fodor, G., Eriksson, A., and Tuoriniemi, A, “Providing quality of service in always best connected networks” Communications Magazine, IEEE , Volume:
41 , Issue: 7 , July 2003
[2] Chiussi, F.M., Khotimsky, D.A., and Krishnan, S., “Mobility management in third-generation all-IP networks” Communications Magazine, IEEE , Volume:
40 , Issue: 9 , Sep 2002
[3] ETSI TR 101 957 v1.1.1 (2001-08) “Broadband Radio Access Networks (BRAN); HIPERLAN TYPE 2; Requirements and Architectures for Interworking between HIPERLAN/2 and 3rd Generation Cellular Systems.”
[4] Jaseemuddin, M , “An architecture for integrating UMTS and 802.11 WLAN networks” Computers and Communication, 2003. (ISCC 2003). Proceedings.
Eighth IEEE International Symposium on , 30 June-3 July 2003
[5] P M Feder, N Y Lee, and S Martin-Leon, “A SEAMLESS MOBILE VPN DATA SOLUTION FOR UMTS AND WLAN USERS”Bell Laboratories - Mobility Solutions, Lucent Technologies Inc., USA, UK
[6] Ivan Pepelnjak and Jim Guichard, “MPLS and VPN architectures – a practical guide to understanding, designing and deploying MPLS and MPLS-enabled VPNs”
[7] David Wisely, Philip Eardley, and Louise Burness, “IP for 3G – networking technologies for mobile communications”
[8] V. Fineberg, “A Practical Architecture for Implementing End-to-End QoS in an IP Network”, IEEE Communications Magazine, January 2002
[9] W.T. Chen and L.C. Huang, “RSVP Mobility Support: A Signaling Protocol for Integrated Services Internet with Mobile Hosts,” Infocom 2000.
[10] A. K. Talukdar, B.R. Badrinath, and A. Achaya, “MRSVP: A Reservation Protocol for an Integrated Services Packet Network with Mobile Hosts,”
Dept. Comp. Sci. tech rep. TR-337, Rutgers Univ.
[11] A. De Carolis et al, “QoS-Aware Handover: Secondary Home Agent”, IETF draft-decarolis-qoshandover-02.txt, April 2000.
[12] S. Paskalis, A. Kaloxylos, E. Zernas, and L. Merakos, “RSVP Mobility Proxy,” Internet draft, Dec. 2001.
[13] Q. Shen et al., “On Providing Flow Transparency Mobility Support for IPv6-based Real-Time Services, “ Proc. MoMuC 2000, Tokyo, Japan, Oct.
2000.
[14] Bougant, F.; Delmond, F.; Pageot-Millet, C., “The user profile for the virtual home environment” Communications Magazine, IEEE , Volume: 41 , Issue:
1 , Jan. 2003
[15] 3GPP TS 22.121 V5.3.0 (2002-03), “Technical Specification Group Services and System Aspects; Service Aspects; The Virtual Home Environment,” rel.
5.
[16] Ronald A. Howard, “Dynamic Programming and Markov Process”
[17] Wei-Kuo Liao and Yi-Chung Chen, “Supporting Vertical Handover between UMTS and WLAN for Real-Time Services” 3rd International Conference on Networking ICN'04