Product Candidates Ranking

As dissimilarity describes the disagreement between any two group members [3], a decision alternative with high dissimilarity is not easy to be consensus. In this study, the dissimilarity (DS) of product p is defined as: characterizations, w w₁+ ₂ 1. At the end of proposed mechanism, a list was presented to consumer as the recommendation from decision group.

4.3 Experiment

4.3.1 Experiment Process

To implement the proposed mechanism, Facebook was selected as data source and the experiment process is shown in Figure 4.4. For the purpose of collecting basic data required, a group of social network users were invited to be participants. Snowball sampling is a feasible way when studying social network issues [2], so it was used to

44

construct the experiment. By using 3 (S) stages 3 (K) names snowball sampling 40 participants were invited in each social network (group). The characteristics of these social networks are summarized in Table 4.5.

Start

(Product Candidates Ranking) Decision makers evaluation End

Figure 4.4 Experiment process for social recommendation mechanism

Table 4.5 Characteristics of the three networks

ATTRIBUTES SOCIAL NETWORKS

STUDENT OFFICE WORKER RANDOM GROUP

Number of participants 40 40 40 invited to be decision-makers. In the experiment, they can issue a decision problem and evaluate the effectiveness of decision alternatives. The decision-makers were asked to issue 2 product purchasing problems (one for mobile phone and one for digital camera) together with their criteria of product selection during the experiment, and these problems were delivered to the decision group members selected by system. When a decision problem was presented to decision group, members can express their design by QOC schema. During the experiment, a product list containing 40 products selected from Amazon top rated items was presented to decision group, and the

45

percentage of the products they ever used/surveyed was recorded to calculate expertise level.

Every recommended product was asked to provide a hyperlink containing related information, so that the click stream data can be collected to compare with other methods. Besides, every recommended product was evaluated manually by the decision makers to see if they are satisfied with the alternatives presented. To avoid information overloading, the first two product candidates of each method were selected.

Since there is a strong tendency for users to spend a greater length of time reading articles of interest to them [32, 56], these data were collected to evaluate the effectiveness of proposed mechanism. The click count and stay time of each page linked to alternatives were recorded, and satisfaction was rated on a 5-point Likert Scale for each alternatives presented to decision makers: Very Useful, Useful, Neither Useful nor Useless, Useless and Very Useless by a rating score of 5, 4, 3, 2 and 1. The related settings of this experiment are listed in Table 4.6.

Table 4.6 Experiment settings of social recommendation mechanism

ITEM SETTING

Type of support Product candidate list recommendation based on provided criteria Participant sampling Snowball sampling

No. of participants Office worker:3 (out of 40) Random member: 3 (out of 40)

Provided criteria Digital camera: camera type, resolution, price Mobile phone: screen size, price, size and weight

Benchmark method

Minimum regret for recommendation strategy Average satisfaction for recommendation strategy

Maximum satisfaction without social impact for decision group selection Evaluation method Clickstream: browsing time on the product pages provided

Perceived helpfulness: questionnaire survey by 5-point Likert scale

4.3.2 Benchmark Methods

To compare proposed mechanism with others, three methods were selected as benchmark.

 Minimum regret: this method was used as baseline benchmark. All the experiment process was the same as proposed mechanism except the products were

46

recommended for minimized regret. the selection rule was changed from equation (4.9) to :

 

1 ( ) ( ) ( ) 2 (1 )

p p p p

RS w min EC i R i C i w DS (4.10)

 Average satisfaction: the decision group member were select as above method, and the selection rule was changed from equation (4.9) to :

 

1 ( ) ( ) ( ) 2 (1 )

p p p p

RS w avg EC i R i C i w DS . (4.11)

 Maximum satisfaction: the decision group members were selected by considering the result of social profile analysis only, that is, the social impact was not included.

4.4 Result and Discussion

In the experiment clickstream data of every alternative presented was collected, and the average stay time of different methods and groups on every alternative is plotted in Figure 4.5. As shown in the figure, the proposed mechanism attracted decision makers to spend more time on the alternatives than other methods. Moreover, as shown in Figure 4.6, the average usefulness level of alternatives generated by proposed mechanism is also higher than other methods. To further examine if there are significant differences in average stay time and average usefulness level, a statistical method is required.

47

Figure 4.5 Average stay time for different groups and methods

Figure 4.6 Average usefulness level for different groups and methods

Two-way analysis of variance (ANOVA) is a statistical analysis in which two independent factors are examined with regard to their impact on a dependent variable.

48

In this work two-way ANOVA was used to test the impact of method used and user group on average stay time and average usefulness level. As seen from Table 4.7, the method used in the experiment has impact on the average stay time as the test result is significant at 0.05 (as 0.00<0.05). In contract, the user group has no impact as 0.961>0.05. For the same reason, based on Table 4.8 the satisfaction can only be influenced by method used during the experiment. Post hoc tests such as Tukey's test most commonly compare every group mean with every other group mean. Knowing that the methods used in the experiments could affect stay time and usefulness level, Tukey’s test was used to see if there is a significant difference between different methods. From Table 4.9 and Table 4.10, there are significant differences between proposed mechanism and other benchmark methods, and the average stay time and average usefulness level are higher than other methods.

Table 4.7 Tests of between-subjects effects for average stay time

Dependent Variable: Average Stay Time

SOURCE TYPE III SUM OF SQUARES DEGREE OF FREEDOM MEAN SQUARE F SIG.

Group 0.030 2 0.015 0.039 0.961

Method 518.683 3 172.894 451.907 0.000

Table 4.8 Tests of between-subjects effects for average usefulness level

Dependent Variable: Average Usefulness Level

SOURCE TYPE III SUM OF SQUARES DEGREE OF FREEDOM MEAN SQUARE F SIG.

Group 1.037 2 0.519 1.648 0.193

Method 647.652 3 215.884 685.997 0.000

49

Table 4.9 Multiple comparisons of average stay time

(I) METHOD (J) METHOD MEAN DIFFERENCE (I-J)

Min Regret

*. The mean difference is significant at the .05 level.

Table 4.10 Multiple comparisons of average usefulness level

(I) METHOD (J) METHOD MEAN DIFFERENCE (I-J)

Min Regret

*. The mean difference is significant at the .05 level.

4.5 Chapter Summary

In this chapter, a personalized while socialized recommendation was proposed. QOC representation schema was used to describe the design logic of product candidates.

Decision group selection mechanism and recommendation conflict resolution within

50

decision group were proposed and served as tools to select adequate products for decision maker. An empirical study further proved the feasibility and effectiveness of proposed mechanism. This chapter successfully introduced the social impact theory and design rationale into the development of social network-based decision support mechanism. Besides, this study also extended the concept of decision support system development to utilize social network platform. From the viewpoint of practice, this work showed a feasible way to develop a social network-based decision support system together with the related techniques for semi-structured decision problems. By dividing the system framework into modules, those who are interested in developing such kind of applications can further improve the system by plugging in new modules as needed.

51