第五章 結論
5.3 未來研究方向
5.3 未來研究方向
本研究將增強式學習結合網站瀏覽推薦，有別於以往推薦之研究，我們將學 習應用在資訊提供型網站而非交易型網站。而對於研究過程中仍有許多改善之處，
未來可參考以下幾個方向進行相關研究：
1. 改變資料來源種類
本研究之目標網站除了屬於資訊提供型網站之外，也限制於學術網站，
使用者為學生及教職人員，類型單純，未來可選擇不同類型的資訊提供型網 站，並在較複雜的網站之下進行研究。
2. 選擇大型網站
本研究僅採用學校單一系所的網站記錄，其資料量有限，未來可以將學 習方式使用在大型網站，觀察在大型網站中的學習效果是否有差別；另外，
在資料量變大的情況下，可以嘗試將學習間隔時間縮小，例如：由一天變為 兩小時，以增加學習次數。
3. 增加狀態種類
實驗中，由於顧慮到未經學習的狀態會導致突然出現推薦不良的情況，
因此將學習的狀態侷限在一定數量之下，如此一來，可能導致不同瀏覽情況 被視為相同狀態；未來研究若能提升網站的學習次數，可同時增加學習狀態 的種類，將網站瀏覽的情況切得更細，以符合確切使用者需求；而增強式學 習學習過程中能夠簡單、快速地建立新的狀態以供學習，未來亦可讓網站遇 到不同於以往的瀏覽情況時，自動增加學習狀態。
54
4. 固定學習行動
本研究之增強式學習的行動為調整各參數權重的相對值，因此讓推薦效 果不穩定，未來的學習可以考慮使用固定權重作為可選擇的行動，並觀察調 整是否會變得更穩定。
5. 調整學習期間
觀察本研究實驗之短期與長期學習結果，為期一年學習的路徑減少程度 並沒有比單月學習好，顯示出學習時間並沒有與調整效果成正比，未來研究 可朝時間的調整進行，找出效果最適合學習的時間長度。
6. 加入模糊理論
未來可以將模糊理論應用在本研究提出之推薦模式，使增強式學習中的 狀態換成模糊狀態，因此可將所有狀態分成許多狀態區塊，提高傳統增強式 學習的學習效率。
55
參考文獻
[1] Bar-Hillel, A., Di-Nur, A., Ein-Dor, L., Gilad-Bachrach, R., Ittach, Y. (2007).
Workstation Capacity Tuning using Reinforcement Learning. In Proceedings of
the 2007 ACM/IEEE conference on Supercomputing, Reno, Nevada, November
12-16, 2007, pp. 1-11
[2] Branavan, S.R.K., Chen, H., Zettlemoyer, L., and Barzilay, R. (2009).
Reinforcement Learning for Mapping Instructions to Actions. In Proceedings of
Association for Computational Linguistics, pp. 82–90
[3] Brusilovsky, P. (1996). Methods and Techniques of Adaptive Hypermedia. User
Modeling and User-Adapted Interaction Journal, 6, pp. 87-129
[4] Bu, X., Rao, J., and Xu, C.Z. (2009). A Reinforcement Learning Approach to Online Web Systems Auto-Configuration. In Proceedings of International
Conference on Distributed Computing Systems, Washington, DC, USA, IEEE
Computer Society, June 2009, pp. 2-11.
[5] Catledge, L. and Pitkow, J. (1995). Characterizing Browsing Behaviors on the World Wide Web. Computer Networks and ISDN Systems, 27(6)
[6] Chou, P.H., Li, P.H., Chen, K.K., Wu, M.J (2010). Integrating Web Mining and Nneural Network for Personalized E-Commerce Automatic Service. Expert
Systems with Applications, 37(4), pp. 2898-2910
[7] Cooley, R., Mobasher, B., Srivastava, J. (1997). Web Mining: Information and Pattern Discovery on the World Wide Web. In Proceedings of International
Conference on Tools with Artificial Intelligence, Newport Beach, California,
U.S.A., November 4-7, 1997, pp. 558-56756
[8] Diligenti, M., Gori, M., and Maggini, M. (2005). Learning Web Page Scores by Error Back-Propagation. In Proceedings of International Joint Conferences on
Artificial Intelligence, Edinburgh, Scotland, UK, July 31 - August 05, 2005, pp.
684-689
[9] Estivill-Castro, V. and Houle, M. (2001). Robust Distance-Based Clustering with Applications to Spatial Data Mining. Algorithmica, 30(2), pp. 216–242,
[10] Etzioni, O. (1996). The World-Wide Web: Quagmire or Gold Mine?
Communication of the ACM, 39, pp. 65-68
[11] Fang, Y.P. and Ting, I.H. (2010) Applying Reinforcement Learning for the AI in a Tank-Battle Game. Journal of Software, 5(12), pp. 1327-1333
[12] Ipek, E., Mutlu, O., Mart´ınez, J.F., Caruana, R., (2008), Self-Optimizing Memory Controllers: A Reinforcement Learning Approach. In Proceedings of
International Symposium on Computer Architecture, June 2008, Washington, DC,
USA, pp. 39-50
[13] Koh, C.E. and Balthazard, P. (1997). Electronic Commerce and the World Wide Web: a Framework of Business Web Use and a Study of Business Web Practices.
In Proceedings of Decision Sciences Institute, San Diego, CA, pp. 599-601
[14] Kosala, R. and Blockeel, H. (2000). Web Mining Research: A Survey. ACM
SIGKDD Explorations, 2 (1), pp. 1–15
[15] Labroche, N., Monmarch´e, N., and Venturini, G. (2003). Antclust: Ant Clustering and Web Usage Mining. In Proceedings of the Genetic and
Evolutionary Computation Conference, July 12-16, 2003, Chicago, USA, pp.
25-36
57
[16] Lee, J.W. (2001). Stock Price Prediction Using Reinforcement Learning. In
Proceedings of IEEE International Joint Conference on Neural Networks,
Washington D.C., July 15-19, 2001, pp. 690–695
[17] Mahmood, T. and Ricc F. (2007). Learning and Adaptivity in Interactive Recommender Systems. In Proceedings of International Conference on
Electronic Commerce, Minneapolis, USA, August 19-22, 2007, pp. 75-84
[18] Mahmood, T. and Ricc F. (2009). Improving Recommender Systems with Adaptive Conversational Strategies. In Proceedings of the 20th ACM conference
on Hypertext and hypermedia, Torino, Italy, June 29 - July 01, 2009 pp. 73-82
[19] Perkowitz, M. and Etzioni, O. (1997). Adaptive Web Sites: an AI Challenge. In
Proceedings of International Joint Conferences on Artificial Intelligence,
Nagoya, Japan, August 23-29, 1997, pp. 16-23
[20] Perkowitz, M. and Etzioni, O. (1998). Adaptive Web Sites: Automatically Synthesizing Web Pages. In Proceedings of Association for the Advancement of
Artificial Intelligence, Madison, Wisconsin, July 26–30, 1998, pp. 727–732
[21] Pierrakos, D., Paliouras, G., Papatheodorou, C., Spyropoulos, C.D. (2003). Web Usage Mining as a Tool for Personalization: A survey. User Modeling and
User-Adapted Interaction, 13, pp.311-372
[22] Rojanavasu, P., Srinil, P., and Pinngern, O. (2005). New Recommendation System Using Reinforcement Learning. In Proceedings of the Fourth
International Conference on eBusiness, Bangkok, Thailand, November 19-20,
2005, 13(SP3)58
[23] Saremi, H.Q. and Montazer, G.A. (2006). Web Usability: A Fuzzy approach to the Navigation Structure Enhancement in a Website System, Case of Iranian Civil Aviation Organization Website. Transactions on Engineering, Computing
and Technology, pp. 123-128
[24] Schwarzkopf, E. (2001). An Adaptive Web Site for The UM2001 Conference. In
Proceedings of the UM2001 Workshop on Machine Learning for User Modeling,
Sonthofen, Germany, July 13-17, 2001, pp. 77–86
[25] Srivastava, J., Cooley, R., Deshpande, M., Tan, P-T. (2000). Web Usage Mining:
Discovery and Applications of Usage Patterns from Web Data. In Proceedings of
SIGKDD Explorations, 1(2), pp. 1-12
[26] Sutton, R. S. and Barto, A.G. (1998). Reinforcement Learning: An Introduction.
MIT Press, Cambridge, MA
[27] Taghipour, N., Kardan, A., Ghidary, S.S. (2007). Usage-Based Web Recommendations: a Reinforcement Learning Approach. In Proceedings of ACM
Conference on Recommender Systems, October 19-20, 2007, New York, USA, pp.
113-120
[28] Teíeni, D., Feldman, R. (2001). Performance and Satisfaction in Adaptive Websites: an Experiment on Searches within a Task-Adapted Website. Journal of
the Association for Information Systems, 2(3), pp. 1-30
[29] Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. PhD Thesis, University of Cambridge, England
59
網站
[30] 黃欣(2011)，網路贏過報紙成為主要新聞來源。 2013 年 1 月。取自：
http://finance.sina.com/bg/juhengwang/20110320/2122249778.html。
[31] 資 策 會 (2012) ， 2011 全 球 上 網 行 為 大 揭 密 。 2013 年 1 月 。 取 自 ： http://mag.udn.com/mag/digital/storypage.jsp?f_ART_ID=391283。