Sensitivity Analysis

In this section we evaluate some sensitive factors that might contribute either negative or positive impact to outgoing call predictors. The diversity of users connectivity, users

call-ing population, regularity users callcall-ing behaviour, and users plurality on interconnectivities behaviour are the factors that we are going to explore in this coming subsection.

Impact of calling diversity

To better understand the context of each observation user, it would be useful to have infor-mation of users calling diversity. We use the concept of Entropy measurement to quantify the diversity of users calling behaviour. The main objective of this measurement is to be able to distinguish the users based on the callee population that user used to contact with. So that, we will be able to see the impact of callee population on prediction accuracy, where the user with high entropy clarifies he has many callees that he used to contact with, otherwise he has only a few callees that he usually to interact. To measure the U serEntropy, let begin with some formal definitions. Let U be a user and C_U is set of callee that at least received one call from the user U . For a callee cϵCU, let to define Zc as the number calls that callee c received from the user U and Z_U is the total calls user U made. Therefore, the probability a callee c receive the call from user U is Pc(U ) = _|Z^|Zc|

U| , where Pc(U ) is the total fraction of all receiving calls to callee c from the user U . So, the computation of U serEntropy accordingly to the following equation:

U serEntropy(U ) =−∑

cϵCu

P_c(U )logP_c(U ) (4)

For concrete illustration of the difference levels of users calling diversity, see Fig. 5.4.. It shows the difference between three users with difference levels of entropies. The diverse of shape of point represents the difference callees and the number of point represents frequency call from each user to the callees. Fig. 5.4(a). illuminates the user with low level entropy which he only interacts with one callee and Fig. 5.4(b). illustrate the user with medium entropy. The user with high entropy, where he has many callees, is indicated by Fig. 5.4(c)..

With simple word, a user will have high entropy if he made many calls to many difference callees. Conversely, the user will have low entropy if he interacts frequently focus on few callees.

To illustration the impact of increasing of user calling diversity on all prediction methods, we randomly select 25 users from reality mining dataset as a sample. Fig. 5.5. shows the diversity level of observation users, where the horizontal axis represents the observation users

Figure 5.4: Illustration of user calling diversity

and their frequency calls, and vertical axis represents the level of user entropy. The user entropy is not affected by the frequency call, but it strongly influenced by interaction of user to many callees. The user with highest entropy (0.705) represented by user33, while the lowest entropy (0.345) represented by user60. According to the level entropy on Fig. 5.5., those sample users are divided to 3 categories users: first, low−level are the users which their entropy value place in between 0.3 0.5. Second, medium− level where the entropy value of user be in between 0.5 0.6, finally the users who have entropy value up to 0.6 is categorized as high− level.

Figure 5.5: Entropy level each of observation user

Fig. 5.6. shows the prediction accuracy comparison under difference level of user entropies with difference setting value of λ ((a)λ=21 days, λ=30 days, and λ=60 days). In Fig. 5.6(a), CPL and CPL(-d) models perform better accuracy than our proposed models on low-level

and middle-level data. But on high level-data, our proposed models perform as well as CPL and CPL(-d) model. CPL and CPL(-d) still outperform than our proposed models when the observation day (λ) is increased to 30 days in low-level data (Fig. 5.6(b)), but only CPL(-d) model perform better than others model for middle-level data. Fol high-level data, our proposed models achieve better accuracy than existing models. Meanwhile, our proposed models PGF and PR, outperformed than other models in every level of data. For the middle-level users, all models getting decrease in average 10% except CPL and CPL(-d), where both of these model decline significantly (approximately 40% for CPL and 30% for CPL(-d)) and getting even worst on high-level user. Once again, this result implies inability of CPL or CPL(-d) in case for handling a lot of prediction. For PGF and PR in case for high− level users, even getting more decreasing, but they still outperformed than other models. Both of these models almost reach 60% of accuracy. Overall, PGF and PR are still reliable in many situations of datasets.

Figure 5.6: Prediction accuracy for difference categories of users with difference setting of observation day(λ): (a)λ=21 days, (b)λ=30 days, and (c)λ=60 days

Impact of the regularity of users routine

In studying the properties of user calling behaviour that related to accuracy portion, we explore the attribute of the regularity of user schedule. The regularity of user schedule is calling density of a user to specific callee at exactly time slot in hour of a day time. To be more clearly, let begin with an example. Suppose you used to call your colleges during day time and your parents every night before going to bed, so the density of your calling activity will be high during day time and low at night time. Since the number of callees is heavily at the day time, the regularity of your calling activity will be detected which we define this situation as irregular schedule calling time. Otherwise, your night time we define your regular schedule calling due to vary specific callee that you used to contact with. To measure the regularity users calling schedule again we use the concept of entropy. Given time-slot the historical call log of the user and number of time-slot t (where t = 1, 2, 3, , 24), the userSchedule defines as in the following equation:

U serEntropy(U ) =−∑

cϵCu

P_c^t(U )logP_c^t(U ) (5)

where P_c^t(U ) is the probability of user U call callee c (cϵC_U), whereC_Uis all callee who receive at least one call during observation day) at given slot-time t. To the best of description of user schedule, we select one user from each category (user60 from low− level, user75 from medium− level, and user96 from high − level) that mentioned in previous subsection as a sample which shown in Fig. 5.7.

Fig. 5.7. gives vary clear description of three differences users in calling activities. Where, user60 we define as the user who has more regularity in calling behaviour during 2 pm until the entire night time and has very heavy calling activity during 9 oclock to 12 oclock. U ser75 seem has not so regular schedule in calling activity since his entropy value distributes evenly in the whole day. Meanwhile, user96 seem has tightly call activity during 11 oclock in evening until 6 oclock in morning and he has less call activity in slot time of 11am and 3 pm.

To clarify the accuracy prediction in each time slot, Fig. 5.8. show the accuracy of all model prediction on each time slot of user96. In general, we can say that almost all accuracy of all the models seem reasonable, where when the entropy value is high the accuracy down,

Figure 5.7: Entropy value of three selected users (user60, user75, and user96) in each time-slot

otherwise the accuracy is increasing. Overall, the model of PGF, PR and GA look dominant in every slot time.

Figure 5.8: Accuracy prediction of all model of users user96 in each time-slot

Impact of the variety of communication

As we mentioned in the beginning of this chapter the Reality Mining dataset consist of three difference types of user communication: V oice− call, short − message, and data − transfer, where the volume portion of voice− call type higher than two other types. In this subsection we want to find out the impact of using variety of communication ways to the outgoing

call prediction. According to the result that we show on Fig. 5.9., short − message and data− transfer give a positive contribution on outgoing call prediction, even very slightly on PGF, PR, and GA. Otherwise, with the increasing of prediction volume by considering short massage and data transfer, give a negative impact for CPL and CPL(-d). Again, we prof that CPL and CPL(-d) get negative impact on dealing with bigger data prediction.

Figure 5.9: Impact of the variety of communication to outgoing call prediction. Short − message and data− transfer give a positive impact to our proposed model, but inversely proportional to the existing model