Comparisons with Other Recommendation Methods

We compare our method with the following five well-known model based recommendation methods.

 SVD (Sarwar et al., 2000) is a matrix factorization algorithm that factorizes a user-item rating matrix into the user preference matrix and the user-item preference matrix.

The preference matrices are used to identify a set of like-minded users and to predict the rating of an item for a target user.

 SVD++ (Koren., 2008) enhances the SVD method by examining implicit user feedbacks on items. The method constructs a binary matrix indicating whether a user

has rated an item. The binary matrix is regarded as the implicit user feedback and is combined with the explicit ratings to estimate latent preferences.

 PMF (Mnih & Salakhutdinov, 2008) is a probabilistic extension of SVD. The authors presumed the preferences (ratings) of users on items are determined by a probabilistic linear model that incorporates a Gaussian prior distribution to minimize the root mean squared error of the predicted ratings. The evaluation shows that the method is effective when ratings are sparse.

 LDA (Xie & Gao, 2014) applies the latent Dirichlet allocation to learn user preferences from ratings. Different to our method, the learning process does not consider the precedence between items. Comparing this method helps realize the benefit of our pairwise learning.

 fastMF (He et al., 2016) improves matrix factorization approaches by assigning weights to missing ratings. Many recommendation systems discard missing ratings of items by assuming that users do not notice the items. However, if the items are popular items, the missing ratings may be a hint that the users do not like the items.

The fastMF assigns different weights to items according to item popularity. Also an alternating least squares technique is adopted to enhance the efficiency of the matrix factorization.

Table 3. The performance of Compared Methods

To ensure the comparisons are fair, the compared methods were implemented with public domain packages³. Besides, the same 5-fold cross validation is applied to the methods to obtain their recommendation performance. Regarding to the number of latent topics, we evaluated the methods by settings K at 30. We also test their performance under the K suggested by the authors.

As shown in Table 3, our method dominates the compared methods in terms of R-precision, P@K, and MRR. Normally, P@K decreases as K increases. This is because the items are ranked according to their relevance to user preferences. A large K thus includes false item suggestions that affect the precision performance. The P@K scores of the compared methods are low. In other words, the methods are not able to place items relevant to user preferences at the top of the recommendation list. As a result, their MRRs are all inferior. By contrast, our precision performances are good that the MRR scores are almost double to theirs.

The SVD, SVD++, PMF, and fastMF methods are all matrix factorization approaches that their preference learning aims at minimizing the difference between the predicted ratings and the ratings given by users. As shown in Table 3, SVD++, PMF, and fastMF performance better than SVD does. This is because the methods enhance SVD by means of various item information (e.g., the popularity of items to weight missing ratings).

Nevertheless, their performances are still inferior to ours. Essentially, the goal of recommendation systems is to rank items according to user preference (Pessiot et al., 2007). Rather than approximating the ratings given by users, our preference learning

3 For PMF, SVD, and SVD++, we used the Surprise matrix factorization package (http://surprise.readthedocs.io/en/stable/index.html). FastMF was based on the authors’

code released on https://github.com/hexiangnan/sigir16-eals.

examines the precedence of items to obtain user preferences. The superior performance of our method validates the value of our pairwise learning. The LDA method and our method are graphical model approaches that our method further considered item precedence to acquire user preferences. The improvement of our method over LDA again demonstrates the benefit of our pairwise learning.

Except fastMF, all the methods produce low diversity scores. As shown in Table 1, the users in the evaluation dataset only rated at least 20 items out of 1,682 items. The rating sparsity phenomenon severely affects the ability of the methods to recommend items received little or no ratings. Consequently, the methods’ diversity performances are poor. As mentioned above, fastMF assigns weights to missing ratings. The method thus is able to recommend different items and achieves good diversity performance.

In summary, our method is superior to the state-of-art methods in R-precision, P@K, and MRR. Our method was able to recommend items that has potential to be rated high scores. By increasing the probability of items exposure, users’ satisfaction is increase and items provider achieve higher loyalty from their users.

5 DISCUSSIONS AND IMPLICATIONS

In this paper, we have developed an effective recommendation method that combines pairwise learning with latent Dirichlet allocation. Instead of approximating the ratings of items made by users, the method examines the precedence between items to learn user preferences. In addition, pair selection strategies are designed to select representative item precedence pairs. The experiment results based on a huge dataset show that the strategies make our preference learning effective and efficient. Moreover, the proposed pairwise

LDA recommendation method outperforms the state-of-the-art recommendation methods in terms of R-precision, precision at k, and MRR.

Our study reveals several potential research directions. First, the experiment result shows that the recommendations based on primitive LDA is comparable to those of the sophisticated matrix factorization methods. While the research of recommendation systems primarily focused on enhancement of matrix factorization, the result suggests that graphical models that associate user behavior (e.g., ratings) with their preferences in probability theory are promising. Second, by considering pairwise learning, our method enhances LDA significantly. The superior performance of our method also suggests recommendation system researchers adopt learning to rank techniques to improve recommendation performance. This suggestion also corresponds to Pessiot et al. (2007)’s assertion that the core of recommendation systems is to rank items according to user preference, rather approximating the ratings given by users. Regarding the precedence pair selection strategies, our study indicates that not only popular items but also item pairs that reveal subtle preference differences of users are informative to recommendation systems. Notably, the strategies help our recommendations concentrate on popular items.

As the revenue of e-commerce is normally based on the sales of popular items. Our evaluation results also encourage researchers to investigate popular item recommendations.

6 LIMITATIONS AND FUTURE WORKDS

Our research is subject to the following limitations. First, as our preference learning is based on the precedence of items, we cannot learn the preference of a user if the user gave all the rated items the same rating. In the future work, we will investigate the missing

not at random (Marlin & Zemel, 2009) which examines the preference of users against non-rating items. We would give different weights (e.g., ratings) to items that are ignored by a user according to the item popularity. This way, item precedence pairs could be generated to derive user preferences. Second, while our method is good at recommending popular items, the diversity of the recommendations is low. This is because the Gibbs sampling is a frequency based inference algorithm that unpopular items may receive a low recommendation probability. As mentioned above, we will investigate a weighting scheme to assign weights to missing items. The weight scheme will also revise the weight of popular and unpopular items so as to increase the chance of suggesting unpopular items.

Moreover, as the preferences of users may be affected by friends, social influence will be incorporated into the preference inference procedure to increase the recommendation diversity.

7 REFERENCES

 Adomavicius, G. and Y. Kwon (2012). "Improving aggregate recommendation diversity using ranking-based techniques." IEEE Transactions on Knowledge and Data Engineering 24(5): 896-911.

 Aggarwal, C. C. (2016). Recommender systems, Springer.

 Alpaydin, E. (2014). Introduction to machine learning, MIT press.

 Blei, D. M., et al. (2003). "Latent dirichlet allocation." Journal of machine Learning research 3(Jan): 993-1022.

 Breese, J. S., et al. (1998). Empirical analysis of predictive algorithms for

collaborative filtering. Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc.

 Deshpande, M. and G. Karypis (2004). "Item-based top-n recommendation

algorithms." ACM Transactions on Information Systems (TOIS) 22(1): 143-177.

 Fürnkranz, J., et al. (2008). "Multilabel classification via calibrated label ranking."

Machine learning 73(2): 133-153.

 Gemman, S. and D. Geman (1984). "Stochastic relaxation, Gibbs Distributions, and The Bayesian of Images." IEEE Trans. Pattern Anal. Mach. Intelligence 6.

 Gemulla, R., et al. (2011). Large-scale matrix factorization with distributed stochastic gradient descent. Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM.

 Griffiths, T. L. and M. Steyvers (2004). "Finding scientific topics." Proceedings of the National academy of Sciences 101(suppl 1): 5228-5235.

 Guo, G., et al. (2013). A Novel Bayesian Similarity Measure for Recommender Systems. IJCAI.

 He, X., et al. (2016). Fast matrix factorization for online recommendation with implicit feedback. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, ACM.

 Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai, Montreal, Canada.

 Koren, Y. (2008). Factorization meets the neighborhood: a multifaceted

collaborative filtering model. Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM.

 Koren, Y., et al. (2009). "Matrix factorization techniques for recommender systems." Computer 42(8).

 Lai, H.-J., et al. (2003). Customized internet news services based on customer profiles. Proceedings of the 5th international conference on Electronic commerce, ACM.

 Lam, S. K. and J. Riedl (2004). Shilling recommender systems for fun and profit.

Proceedings of the 13th international conference on World Wide Web, ACM.

 Larson, R. R. (2010). "Introduction to information retrieval." Journal of the American Society for Information Science and Technology 61(4): 852-853.

 Li, S., et al. (2015). Deep collaborative filtering via marginalized denoising auto-encoder. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, ACM.

 Li, Z. and C. Xu (2013). Tag-based top-N recommendation using a pairwise topic model. Information Reuse and Integration (IRI), 2013 IEEE 14th International Conference on, IEEE.

 Lin, K.-Y., et al. (2013). Data selection techniques for large-scale rank SVM.

Technologies and Applications of Artificial Intelligence (TAAI), 2013 Conference on, IEEE.

 Lops, P., et al. (2011). Content-based recommender systems: State of the art and trends. Recommender systems handbook, Springer: 73-105.

 Marlin, B. M. and R. S. Zemel (2009). Collaborative prediction and ranking with non-random missing data. Proceedings of the third ACM conference on

Recommender systems, ACM.

 Mnih, A. and R. R. Salakhutdinov (2008). Probabilistic matrix factorization.

Advances in neural information processing systems.

 Pessiot, J.-F., et al. (2007). "Learning to rank for collaborative filtering."

 Rendle, S., et al. (2009). BPR: Bayesian personalized ranking from implicit feedback. Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence, AUAI Press.

 Resnick, P., et al. (1994). GroupLens: an open architecture for collaborative

filtering of netnews. Proceedings of the 1994 ACM conference on Computer supported cooperative work, ACM.

 Sarwar, B., et al. (2000). Application of dimensionality reduction in recommender system-a case study, Minnesota Univ Minneapolis Dept of Computer Science.

 Sarwar, B., et al. (2001). Item-based collaborative filtering recommendation algorithms. Proceedings of the 10th international conference on World Wide Web, ACM.

 Sharma, A. and B. Yan (2013). Pairwise learning in recommendation: experiments with community recommendation on linkedin. Proceedings of the 7th ACM Conference on Recommender Systems, ACM.

 Shi, Y., et al. (2010). List-wise learning to rank with matrix factorization for collaborative filtering. Proceedings of the fourth ACM conference on

Recommender systems, ACM.

 Tan, T. F. and S. Netessine (2009). "Is Tom Cruise threatened? Using Netflix Prize data to examine the long tail of electronic commerce." Wharton Business School, University of Pennsylvania, Philadelphia.

 Voorhees, E. M. (1999). The TREC-8 Question Answering Track Report. Trec.

 Wang, J., et al. (2006). Unifying user-based and item-based collaborative filtering approaches by similarity fusion. Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, ACM.

 Xie, W., et al. (2014). "A probabilistic recommendation method inspired by latent dirichlet allocation model." Mathematical Problems in Engineering 2014.

 Yao, W., et al. (2015). Collaborative Topic Ranking: Leveraging Item Meta-Data for Sparsity Reduction. AAAI.

 Yu, K., et al. (2004). "Probabilistic memory-based collaborative filtering." IEEE

Transactions on Knowledge and Data Engineering 16(1): 56-69.

 Zhang, B.-T. and Y.-W. Seo (2001). "Personalized web-document filtering using reinforcement learning." Applied Artificial Intelligence 15(7): 665-685.