Chapter 4 Problem Formulation by MDP in FA Strategy
5.3 Update Rules of Policy with Time-varying MDP Parameters
There are six parameters in our MDP model, and they are λ1,λ2, μ1, μ2,h1 and h2. The six parameters of the actual system vary with time; therefore, we have to adjust them periodically to make our MDP model closer to the actual system and to get the more improved policy. The method how we adjust these parameters is to appreciate the system cost that is induced by rejecting calls (new calls or handoff calls) and the model cost that is one of the computational result in our MDP model, “gain”. The system cost is divided into two parts, and they are
“Block Cost” that is the cost due to blocking new calls and “Drop Cost” that is due to dropping handoff calls. The update rule is derived by appreciating the data of the simulation result. We find that the six parameters may be rational to the either Block Cost or Drop Cost, and it is listed blow.
Table 5.2 Parameters update rules
Parameters Rational Cost (Block Cost or Drop Cost) (+ : positive rational or - : negative rational) Base cell’s arrival rate λ1 Block Cost (+) model cost is defined as Eq. (5.2), and is the sum of the gain of the MDP model per update period.
Before the explanation of the update rules, we have to define the difference of model cost and system cost as Eq. (5.3) below:
1
Furthermore, the adjustment factor of parameters is defined as Eq. (5.4) below:
: base cell's arrival rate
, : neighboring cells' arrival rate
(1 )
: base cell's departure rate
, : neighboring cells' departure rate
(1 )
: base cell's handoff-out rate , : neighboring cells' handoff-in rate
(1 ) update rules to ensure that the model is closer to the actual system. The update period is determined by how fast the system changes.
Chapter 6 Simulator and Results
6.1 Simulator Settings
1. The size of the map for simulation is 12.12 km x 24.25 km. The map are composed of nodes that contain information about :
i. the position : (x , y);
ii. the type of the node : road or not;
iii. the coverage of the cell.
2. There are 98 (14 X 7) cells on the map, and the radius of each cell is 2 km
3. Wrapped-around Map: when mobile reaches the boarder of the map, it will move to the opposite side boarder of the map.
4. There are 50 Channels per cell : the capacity of base cell (the number nominal channels C1) is 50, and the capacity of the neighboring cells C2 is 300.
5. The whole map is spread non-uniform traffic loading
6. Poisson Arrivals on the whole map with arrival rate λ (arrivals / cell / hour). Arrival Rate of mobiles in Base Cell : λ1 ; Arrival Rate of mobiles in Neighboring Cells : λ2.
7. Exponentially distributed service time per mobile with average service time 180 seconds per call. Departure Rate of mobiles in Base Cell : μ1 ; Departure Rate of mobiles in Neighboring Cells : μ2.
8. Handoff rates are determined by the randomized number of mobiles in the base cell, the number of mobiles in the neighboring cells, the speeds of mobiles in the base cell and neighboring cells, and the road topology spread on the base cell and the neighboring cells, and so on. Handoff-in Rate : h1 ; Handoff-out Rate : h2. h1 and h2 are derived by measuring in Base Cell.
9. Mobiles move mainly in one direction and at speeds (20 ~ 90 km/hr) with 5% variance, and they do not move back unless there is no way to move forward, right, or left.
10. We define call failure rate:
Call Failure Rate = Pb + (1-Pb) x Pd , Pb: Call Blocking Rate, Pd: Call Dropping Rate (6.1)
Fig 6.1 Road Topology Example for Simulation
6.2 UML Statechart of Our Model
We use UML(Unified Machine Language) to simulate the environment. The map is transformed from the simulator as illustrated in fig 6.1 above. The OMD (Object Main Diagram) is as illustrated in fig 6.2 below. When the simulation starts, one object of CellsGen generates objects of Map, of Cell, of Channel, and sets all links between Map-Cells, between Cell-Cell, between Cells-Channels. The object of Map generates the map and roads on the map in the beginning and produces objects of Mobile non-uniformly on the roads of the object of Map at the moment when the call comes. Objects of Mobile move along the roads generated by the object of Map, and make the handoff operation from one object of Cell to another when they move across the objects of Cell. The object of Cell has three jobs. First, it gets new calls and handoff calls from objects of Mobile. Secondly, it sets states of the objects of Channel.
Finally, it informs the objects of Cell that are affected by the handoff operation, and keeps the list of using channels, the list of borrowing channels, and the list of borrowed channels. The objects of Channel are passive objects. They just own the records of states of themselves recorded by objects of Cell. The object of MDP owned by the object of Cell make calculation of Policy Iterations to decide the policy.
Fig 6.2 OMD of the Simulator
The State Chart of Mobile are illustrated as fig 6.3 below. When it is constructed, it will own a randomly generated service time which is exponentially distributed, the speed which is randomly selected from 20 km ~ 90 km. Its residual time is determined by the roads and the speed at which the object of Mobile moves.
Fig 6.3 State Chart of Mobile
The State Chart of Map is illustrated as Fig 6.4 below. When it is constructed, it produces the map in the beginning, and generates the inter-arrival time randomly with exponential distribution to determine when the object of Mobile is constructed.
The State Chart of Cell is illustrated as Fig 6.5 below. It receives events evIn which include new call events from objects of Mobile, and handoff-in call events from objects of Cell. It also receives events evOut which include call ending events from objects of Mobile and handoff-out events from objects of Cell. It also periodically updates parameters including Base Cell arrival rateλ1 , Neighboring Cell arrival rateλ2, Base Cell departure rateμ1, Neighboring Cell departure rateμ2, Handoff-out rate h1, and Handoff-in rate h2.
Fig 6.4 State Chart of Map
Fig 6.5 State Chart of Cell
Because there are total 50 × 300 states which is to difficult to compute and not efficient in real-time, we use the aggregation method mentioned to group states into smaller groups. In our model, we choose total 6 × 11 states as shown in Table 6.1 below which is a compromise between computing complexity and the difference of the result derived. After the offline policies are determined, the values of all states are derived. With the values of the states, we will then determine the online policies by One-Step policy when events evIn occur.
Note that the last column and row (with gray background) of the table is made of only one single state. Because no matter there is a new call or a handoff call arrives in that state, it will not be accepted due to unavailable of the channel. And the information of adjacent cells’ state will be update periodically. When the policy is derived, each cell’s base station can make call admission control according to nine actions listed in Table 6.2 below.
Table 6.1 Aggregation of total states
Cell’s States (after aggregation)
Base Cell’s Group
0~9 10~19 20~29 30~39 40~49 50
1 2 3 4 5 6
Adjacent Cells’ Group
0~29 1 0 1 2 3 4 5
30~59 2 6 7 8 9 10 11
60~89 3 12 13 14 15 16 17
90~119 4 18 19 20 21 22 23
120~149 5 24 25 26 27 28 29
159~179 6 30 31 32 33 34 35
180~209 7 36 37 38 39 40 41
210~239 8 42 43 44 45 46 47
240~269 9 48 49 50 51 52 53
270~299 10 54 55 56 57 58 59
300 11 60 61 62 63 64 65
Table 6.2 Alternatives of base cell’s states.
6.3 Simulation Results
Fig 6.6 Call Block Rate of different method under different loads Alternatives New Call Handoff Call
0 block drop
BDCL Block Rate(Policy pdated per 15 sec)
0.0000
Call blocking Rate is illustrated above as Fig 6.6, there are three methods of BDCL, which are BDCL without call admission control, call admission control by MDP, call admission by Guard Channel Policy. They are simulated under different loads and cost (block cost-drop cost), 50% (5-30), 70% (5-30), 90% (5-30), 100% (5-30), 150% (5-40) ,150% (5-45), 200% (5-50), 200% (5-60). We also simulated it with update periodically and without it. The result shows that Block Rate of our MDP method no matter with periodic policy update or not is lower than that of Guard Channel of BDCL when traffic load is under 100%. However, Block Rate of our MDP method without periodic policy update is higher than that of Guard Channel policy of BDCL when traffic load is greater than 100%, but Block Rate of our MDP method with periodic policy update is lower than that of Guard Channel policy of BDCL. We can conclude that our MDP method with periodic policy update gets better efficiency than Guard Channel policy of BDCL.
Fig 6.7 Call Drop Rate of different method under different loads BDCL Drop Rate(Not Updated)
Load50 Load70 Load90 Load100 Load150(5-40) Load150(5-45) Load200(5-50) Load200(5-60) Load(%)
Prob
Normal DR Policy DR Guard DR
Call Dropping Rate are illustrated above as Fig 6.7. The result shows that call drop rate of our MDP method without periodic policy update is almost the same as that of Guard Channel policy of BDCL. However, call drop of our MDP method with periodic policy update is higher than that of Guard Channel policy of BDCL. We can conclude that our method with periodic policy update gets worse efficiency than Guard Channel policy of BDCL and our method without periodic policy update.
Fig 6.8 Call Failure Rate of different method under different loads
Call Failure Rate is illustrated above as Fig 6.8. The result shows that call failure rate of our MDP method without periodic policy update is higher than that of Guard Channel policy of BDCL under all traffic loads. However, call failure rate our MDP method with periodic is lower than that of Guard Channel policy under traffic load 100%, but it is higher under traffic
BDCL Failure Rate(Not Updated)
Load50 Load70 Load90 Load100 Load150(5-40) Load150(5-45) Load200(5-50) Load200(5-60) Load(%)
load higher than 100%. We conclude that the efficiency of our method gets better under traffic 100% than Guard Channel policy of BDCL.
Fig 6.9 Map of road topology in Cell 16, Cell 57 and Cell 62
Fig 6.10 Parameters update for load 70 Cost 5-30 of period 15 sec
Cell 16 Cell 57 Cell 62
Fig 6.11 Parameters update for load 70 Cost 5-30 of period 60 sec
Fig 6.12 Parameters update for load 70 Cost 5-30 of period 120 sec
Fig 6.13 Parameters update for load 70 Cost 5-30 of period 180 sec
Fig 6.14 Parameters update for load 70 Cost 5-30 of period 240 sec
Fig 6.15 Parameters update for load 70 Cost 5-30 of period 300 sec
Fig 6.16 Parameters for load 70 Cost 5-30 without periodic update
As illustrated above in Fig 6.10, Fig 6.11,…, Fig 6.16, they are curves of six parameters of Cell 16, of Cell 57 and of Cell 62 with time varying under different policy update periods, which are 15 seconds, 60 seconds, 120 seconds, 180 seconds, 240 seconds, 300 seconds and no policy update. Cell 16, as illustrated above in Fig 6.9, is the cell with the most crowded road topology among those three cells; Cell 57 is the medium one; Cell 62 is the one with the least crowded road topology. The above Figures show that the six parameters of more crowded road topology vary more severely with time and vary more frequently with more frequent policy update.
Fig 6.17 Average cost of Load 70 Cost 5-30 with different update periods
Fig 6.18 Average cost of Load 90 Cost 5-30 with different update periods
Fig 6.19 Average cost of Load 100 Cost 5-30 with different update periods
Fig 6.20 Average cost of Load 150 Cost 5-40 with different update periods
Fig 6.21 Average cost of Load 150 Cost 5-45 with different update periods
Fig 6.22 Average cost of Load 200 Cost 5-50 with different update period
Fig 6.23 Average cost of Load 200 Cost 5-60 with different update period
As illustrated above in Fig 6.17 and Fig 6.18, the result shows that the average cost is not lower with more periodic policy update. In Fig 6.17, it is simulated under traffic load 70% for different policy update periods, and the average cost of update period 120 seconds is the lowest one among those of other update periods. In Fig 6.18, it is simulated under traffic load 90% for different policy update periods, and the average cost of update period 180 seconds is the lowest one among those of other update periods. It means that the more frequent policy update does not get the better efficiency.
From Fig 6.19, Fig 6.20,…, Fig 6.23 illustrated above, they are simulated under traffic loads of 100%, 150% and 200% for different policy update periods. Those figures show that the average cost without policy update is lower under traffic loads over 100%. The efficiency is better if we do not update policy under over-loaded traffic. Moreover, From Fig 6.17 to Fig 6.23, we find that the average cost with more frequent policy update does not adjust so severely with great dumping. The curve of those average costs is smoother with more frequent policy update.
Chapter 7 Conclusion
In the thesis, we simulated the call admission control problem in cellular networks with BDCL strategy by One-Step Policy under the environment of road topology. The result shows that the efficiency of call failure rate is better under traffic load of less than 100% than that of Guard Channel Policy of BDCL. However, if the traffic load is over 100%, the efficiency gets worse than Guard Channel Policy of BDCL. We suppose that our method would preserve some channels to borrow before the system reaches the convergence, but this phenomenon causes the waste of channels and therefore, the efficiency gets worse than Guard Channel Policy of BDCL.
As we mentioned, the borrowing takes costs. In the aspect of periodic policy update of six parameters, the result shows that the efficiency of call failure rate with policy update is better than that without policy update. However, it causes the efficiency of call dropping rate worse.
This is the trade-off between update and no update.
References
[1] S. Tekinay and B. Jabbari, “Handover and Channel Assignment in Mobile Cellular Networks,” IEEE Commun. Mag., vol. 29, no. 11, Nov. 1991.
[2] E. C. Posner and R. Guerin, “Traffic Policies in Cellular Radio that Minimizing Blocking of Handoff Calls,” in Proc. 11th Teletraffic Cong. (ITC 11), Kyoto, Japan, Sept.
1985.
[3] S. Choi and K. G. Shin, “Adaptive Bandwidth Reservation and Admission Control in QoS-Sensitive Cellular Networks,” IEEE Trans. Parallel and Distributed Sys., vol. 13,
no. 9, pp. 882–897, Sep. 2002.
[4] R. Ramjee, D. Towsley, and R. Nagarajan, “On Optimal Call Admission Control in Cellular Networks,” Wireless Networks Journal, vol. 3, no. 1, pp. 29-41, March 1997.
[5] R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, 1960.
[6] D. P. Bertsekas, Dynamic Programming and Optimal Control, vol. 2, 2nd edition, Athena Scientific, Belmont, Massachusetts, 2000.
[7] D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, NJ, 1987.
[8] D. Hong and S. S. Rappaport, “Traffic Model and Performance Analysis for Cellular Mobile Radio Telephone Systems with Prioritized and Nonprioritized Handoff Procedures,” IEEE Trans. on Vehicular Tech., vol. 35, no. 3, pp. 77-92, Aug. 1986.
[9] B. L. Miller, “A Queuing Reward System with Several Customer Classes,”
Management Science, vol. 16, no. 3, pp. 234-245, 1969.
[10] M. L. Puterman and M. C. Shin, “Modified Policy Iteration Algorithms for Discounted Markov Decision Problems,” Management Science, vol. 24, no. 11, pp. 1127-1137, Jul.
1978.
[11] M. L. Puterman and M. C. Shin, “Action Elimination Procedures for Modified Policy Iteration Algorithms,” Operations Research, vol. 30, no. 2, pp. 301-318, 1982.
[12] E. L. Porteus, “Overview of Iterative Methods for Discounted Finite Markov and Semi-Markov Decision Chains,” in Recent Developments in Markov Decision Process, R. Hartley et. al. (eds.), New York: Academic Press, 1980.
[13] D. Collins and C. Smith, 3G Wireless Networks, McGraw-Hill, 2001.
[14] A. J. Viterbi, CDMA: Principles of Spread Spectrum Communication, Reading, Mass.:
Addison-Wesley, 1995.
[15] T. J. Ott and K. R. Krishnan, “State Dependent Routing of Telephone Traffic and the Use
of Separable Routing Schemes,” in Proc. 11th Teletraffic Cong. (ITC 11), Kyoto, Japan, Sept. 1985.
[16] V. G. Lazarev and S. M. Starobinets, “The Use of Dynamic Programming for Optimization of Control in Networks of Communications of Channels,” Eng. Cyber., vol. 15, pp. 107-116, 1977.
[17] S. Zachary, “Control of Stochastic Loss Networks, with Applications,” J. Roy. Statist.
Soc. Ser. B, vol. 50, pp. 61-73, 1988.
[18] A. Jayasuriya, D. Green and J. Asenstorfer, “Modeling Service Time Distribution in Cellular Networks Using Phase-Type Service Distributions, ” in Proc. IEEE Int. Conf. on Communications (ICC 2001), vol. 2, pp. 440-444, Helsinki, Finland, June 2001.
[19] F. Barcelo and J. Jordan, “Channel Holding Time Distribution in Cellular Telephony,”
The 9th International Conference on Wireless Communications (Wireless’97), vol. 1, pp. 125-134, Alberta Canada, 9-11 July, 1997.
[21] Chia-Fu Chen, “Markov-Decision-Based Call Admission Control with Neighboring State Information in Cellular Networks, ” in NCTU, Dec 2006.
[22] M. Zhang and T.-S. Yum. “Comparisons of channel assignment strategies in cellular mobile telephone systems.” IEEE Trans. on Vehicular Technology, Vol. 38, no. 4, pages 211{215, Nov., 1989.
[23] Kshirasagar Naik and David S.L. Wei and Stephan Olariu, “Channel Assignment in Cellular Networks with Synchronous Base Stations, ” PEWASUN’ 05, October 10–13, 2005, Montreal, Quebec, Canada.