Chapter 4 Numerical Illustration
4.4 Use Mean Absolute Error and Kendall’s test to understand the performance of
According to table 4.7 and table 4.8, we understand the distinct performance of different decision-makers’ preference utilities will vary under six MCDM methods and change whether considering context effect or not. The next step is to understand which MCDM method has more similar result to subjective preference and how much the context effect will influence decision-makers’ preference. In this research, we use mean absolute error (MAE) and Kendall’s test to analyze the result.
4.4.1 Mean absolute error (MAE)
Table 4. 9 Table of the result of MAE under linear utility and prospect utility
Compare Linear Utility Prospect Utility
Rank
Rank with and without context effect 0.4 0.6
Rank with context effect
PLP 0 0
Pair 0 0
Table 4.9 shows the result of the correlation between MCDM methods and subjective preference and the difference after adding the context effects. We use mean absolute error (MAE) to do a pairwise comparison. Firstly, we list six MCDM methods (, which are PLP, APH, TOPSIS, TOPSISr, Pair, and VIKOR) to find out which MCDM method can come out more similar to subjective preference and how it changes under different utility. We find that the ranks, which comes out through AHP, PLP, Pair method, are closer to the rank comes out through decision-makers' subjective preference. Because PLP, Pair, and AHP methods are kinds of interactive methods, our research concludes that the interactive methods could help decision-makers to come out the more appropriate rank of their preference. Moreover, comparing the linear utility and prospect utility, we found that the prospect utility has much higher MAE. It shows that using TOPSIS, TOPSISr, and VIKOR methods under prospect utility cannot perspective demonstrate decision-makers' preference and find out the optimal selection. Secondly, we compare how the rank will change after adding context effects under difference utilities. Under linear utility, the change range is smaller than the range under prospect loss. Lastly, we list PLP and Pair methods to find the correlation between MCDM methods and decision-makers' subjective preference. The reason why we choose PLP and pair methods is that these two methods are interactive methods, which using the preferences utilities to generate the rank of alternatives (, rather than using criteria). Though AHP has similar characteristic to Pair methods, only Pair method allows one to represent such subjective
assessments and to process them by analyzing, quantifying and identifying the inconsistencies.
After the calculation, we find that under both MCDM methods, the rank is similar to decision-makers’ subjective preference after adding context effects. And, both optimal selection’s ranks are similar under linear utility and prospect utility.
4.4.2 Kendall’s test
Table 4. 10 Table of the result of Kendall’s test under linear utility and prospect utility Compare Linear Utility Prospect Utility
tau p tau p
Rank with and without
context effect 0.91 0.00** 0.87 0.00**
Note:* shows the case when the statistical p-value is not greater than 0.05
** shows the case when the statistical p-value is not greater than 0.01
For here, we use Kendall's test to get the correlation between MCDM methods and subjective preference and the result shows at Table 4.10. In Table 4.10, the value of tau demonstrates the measure of correlation, and the statistical p-value (two-tailed p) demonstrates the significant difference. We use ** to shows the case when the statistical p-value is not greater than 0.01, which also means the MCDM and the subjective preference has highly significantly consistency. In this experience, the rank comes out through AHP, PLP, Pair method, is as same the rank comes out through decision-makers' subjective preference. It means that AHP, PLP, and Pair methods can completely show decision-makers' subjective preference. However, if the statistical p-values are higher than 0.05, it means that the rank comes out through MCDM methods, which are TOPSIS,
TOPSISr, and VIKOR methods, don't have consistency with the rank comes out through decision-makers' subjective preference. Moreover, because the value of tau under prospect utilities are lower than linear utilities, the rank under prospect utility is more inconsistent than the rank under linear utility. Secondly, when testing the consistency between the rank of subjective preference before and after the context effect, the statistical p-values are all lower than 0.01 under both using linear or prospect utility. However, the value of tau when using linear utility is similar to when using prospect utility. Thus, we find that no matter using linear utility or prospect utility, it will not affect the consistency of the rank after adding context effects' utilities. Lastly, as same as using MAE, under both Pair and PLP methods, the rank is highly consistent with decision-makers' subjective preference after adding context effects.
4.5 Extension situation—the utility under different value of
hATT,
COM
h, and
hSIMAt 4.1.4, we define that
hATT,
hCOM, and
hSIM are 0.1. In this part, we define them in different ways.4.5.1 Define
hATT,
hCOM, and
hSIM as 0, 0.1, or 0.5When
hATT,
hCOM, and
hSIM are defined as 0, 0.1, or 0.5, there will be twenty-seven different kinds of combination. After we use R-programming to come out the utility of each alternatives, we find that some utilities will be negative. The reason is that the compromise effect will cause negative effect on the total utility. If the beta of
hCOM is too big, it will cause the total utility of that alternative to be negative (the example show at table 4.11). According to the theory of AHP (Pair), two alternatives both should be positive when using AHP(Pair) to compare (Millet and Schoner, 2005). When we use negative utilities to run AHP (Pair) analysis, the errors will show up. As a result, we remove the options which beta equals 0.5 and define
hATT,
hCOM, and
hSIMas 0, 0.1.Table 4. 11 The Example of negative utility after adding context effect
i Utility without context effect
Utility of Context Effect
Utility with context effect Compromise
Effect Attraction Effect Similarity Effect
Total
COM
h =0.5
hATT=0.1
hSIM=01 0.508158508 0 0.658573803 0.086956522 0.074553032 0.582711541 2 0.671328671 -0.517065564 0.929181276 7.62647E-17 -0.165614654 0.505714017 3 0.249417249 -0.367767474 0.146266978 0.043478261 -0.164909213 0.084508036 4 0.076923077 -0.76122288 0 0.043478261 -0.376263614 -0.299340537 5 0.466200466 -0.126209243 0.350629245 9.15176E-17 -0.028041697 0.438158769 6 0.666666667 -1 0.558856808 1 -0.344114319 0.322552347 7 0.596736597 -0.593658625 0.515107873 1 -0.145318525 0.451418071 8 0.552447552 -0.093664343 0.541404788 1.52529E-17 0.007308307 0.559755859 9 0.438228438 -0.391563329 0.616602638 0.326086957 -0.101512705 0.336715733 10 0.757575758 -0.676131208 1 0 -0.238065604 0.519510154
4.5.2 Define
hATT,
hCOM, and
hSIM as 0 or 0.1When
hATT,
hCOM, and
hSIM are defined as 0 or 0,1, there will be eight different kinds of combination. After using R-programming to come out the ranks, we use Mean absolute error (MAE) and Kendall’s test to analyze the result.4.5.2.1 Mean absolute error (MAE)
Table 4. 12 Table of the result of MAE under linear utility and prospect utility
Situation I II III IV V VI VII VIII
Rank with and without context effect
under subjective preference 0 0.8 0 0.4 0.2 0.4 0.2 0.4 Rank with and without context effect
under PLP 0 0.8 0 0.4 0.2 0.4 0.2 0.4
Rank with and without context effect
under Pair 0 0.8 0 0.4 0.2 0.4 0.2 0.4
Rank with context effect
under Subjective Preference and PLP 0 0 0 0 0 0 0 0 Rank with context effect
under Subjective Preference and Pair 0 0 0 0 0 0 0 0 Prospect Utility
Rank with and without context effect
under subjective preference 0 0.2 0.4 0.4 0.6 0.6 0.4 0.6 Rank with and without context effect
under PLP 0 0.2 0.4 0.4 0.6 0.6 0.4 0.6
Rank with and without context effect
under Pair 0 0.2 0.4 0.4 0.6 0.6 0.4 0.6
Rank with context effect
under Subjective Preference and PLP 0 0 0 0 0 0 0 0 Rank with context effect
under Subjective Preference and Pair 0 0 0 0 0 0 0 0
Table 4.12 shows the result of the correlation between MCDM methods and subjective preference under without and with the context effects. Moreover, the table also shows how the ranks vary after adding context effect. Both under linear utility and prospect utility, the rank between subjective preference and both MCDM methods, namely PLP and Pair are the same. And the rank of subjective preference, PLP, and pair all become different after adding context effect. Furthermore, we may wonder how the context effects vary the rank after adding them into utilities. Thus, we use significance
testing, we found that compared to attraction effect and similarity effect, the compromise effect will cause a bigger change of the rank under both linear and prospect utility.
4.5.2.2 Kendall’s Test
Table 4. 13 Table of the result of Kendall’s Test under linear utility and prospect utility
Situation I II III IV V VI VII VIII With and without context effect
under subjective preference 1 ** 0.82 ** 1 ** 0.91 ** 0.96 ** 0.91 ** 0.96 ** 0.91 **
With and without context effect
under PLP 1 ** 0.82 ** 1 ** 0.91 ** 0.96 ** 0.91 ** 0.96 ** 0.91 **
With and without context effect
under Pair 1 ** 0.82 ** 1 ** 0.91 ** 0.96 ** 0.91 ** 0.96 ** 0.91 **
With context effect under Subjective Preference and
PLP 1 ** 1 ** 1 ** 1 ** 1 ** 1 ** 1 ** 1 **
With context effect
under Subjective Preference and Pair 1 ** 1 ** 1 ** 1 ** 1 ** 1 ** 1 ** 1 **
Prospect Utility tua p tua p tua p tua p tua p tua p tua p tua p With and without context effect
under subjective preference 1.00 ** 0.96 ** 0.91 ** 0.91 ** 0.87 ** 0.82 ** 0.91 ** 0.87 **
With and without context effect
under PLP 1.00 ** 0.96 ** 0.91 ** 0.91 ** 0.87 ** 0.82 ** 0.91 ** 0.87 **
With and without context effect
under Pair 1.00 ** 0.96 ** 0.91 ** 0.91 ** 0.87 ** 0.82 ** 0.91 ** 0.87 **
With context effect under Subjective Preference and
PLP
1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 **
With context effect
under Subjective Preference and Pair 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 ** 1.00 **
Note:* shows the case when the statistical p-value is not greater than 0.05
** shows the case when the statistical p-value is not greater than 0.01
Table 4.13 also shows the result of the correlation between MCDM methods and subjective preference under without and with the context effects. We use Kendall's test to figure out the correlation. Similar to the result of Table 4.14, no matter we use linear utility or prospect utility to find the decision-makers' preference, the rank has no difference between subjective preference or MCDM methods. As we mentioned before, the value of tua demonstrates the level of relevant and the statistical p-value (two-tailed p) demonstrates the significant difference. In this Table, we use ** to shows the case when the statistical value is not greater than 0.01. As we can see, all the statistical p-values are not greater than 0.01, which means the MCDM and the subjective preference has highly significantly consistency. However, we can use the value of tau to determine the level of relevant. The bigger the value of tau, the higher level of relevant. We use significance testing to find out which context effects affect the rank before and after adding context effect. As the result under MAE method, compared to the attraction effect and similarity effect, the compromise effect will affect the level of relevant most under both linear and prospect utility.