Experiment 1-1: parameter selection for TP-V method

This experiment aims to determine the value of parameter

δ

_V in the TP-V method. TP-V method uses

δ

_V to adjust the relative weight of self task profile and collaborative topic-variation profile. When

δ

_V is set to 1, the TP-V method is equivalent to TP method, which takes only the self task profile into account. When

δ

V is set to 0, it is equivalent to the collaborative topic-variation profile. The experiment was conducted by systematically adjusting the value of

δ

_V in an increment of 0.1. The precision metric was chosen as the performance measure to evaluate the effectiveness of the methods. The optimal parameter values with the best results (the highest precision values) were chosen as the parameter settings of the proposed equations. Table 7.1 shows the performance of TP-V with different

δ

_V value in terms of precision under various top-N supported documents.

Observation: Table 7.1 shows that the average precision value of TP-V method with δ

V= 0.7 exceeds those with the other values. Meanwhile, while setting

δ

_V= 0.7 in the given equation of TP-V method, it can achieve the best performance under Top-5,

documents retrieved

of number

relevant are

that documents retrieved

of number

= precision

documents relevant

known of

number

retrieved are

that documents relevant

of number

= recall

recall precision

recall precision

F

× +

×

×

= (1+ ₂ ²)

β

β

β

Top-10, Top-15, or Top-20 document support. The TP-V method is increasing dramatically from

δ

_V= 0 to 0.7 and is decreasing slightly from

δ

_V= 0.7 to 1, as shown in Fig. 7.1. The experimental result reveals that self task profile is more important than the collaborative topic-variation profile in the TP-V method.

Table 7.1 Effectiveness of TP-V method under various δ

_V

values

δ

_V

Top-N 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Top-5 0.046 0.185 0.2 0.215 0.338 0.338 0.354 0.369 0.369 0.338 0.354

Top-10 0.062 0.138 0.192 0.292 0.323 0.362 0.385 0.408 0.392 0.385 0.385

Top-15 0.056 0.138 0.215 0.262 0.308 0.354 0.369 0.385 0.359 0.359 0.359

Top-20 0.065 0.146 0.208 0.258 0.304 0.362 0.354 0.358 0.327 0.323 0.319

Average 0.057 0.152 0.204 0.257 0.318 0.354 0.365 0.380 0.362 0.351 0.354

Fig. 7.1 Result of knowledge support of TP-V under various δ

_V

value 7.2.2. Experiment 1-2: parameter selection for TP-D method

This experiment aims to determine the value of parameter

δ

_D in the TP-D method. TP-D method uses

δ

_D to adjust the relative weight of self task profile and collaborative document profile. When

δ

_D is set to 1, the TP-D method is equivalent to the TP method, which takes only the self task profile into account. When

δ

_D is set to 0, it is equivalent to the collaborative document profile. The experiment was also conducted by systematically adjusting the value of

δ

_D in an increment of 0.1. The

precision metric was chosen as the performance measure to evaluate the effectiveness

of the methods. The optimal parameter values with the best results (the highest precision values) were chosen as the parameter settings of the proposed equations.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

Table 7.2 shows the performance of TP-D method with different

δ

_D value in terms of precision under various top-N supported documents.

Observation: Table 7.2 shows that the average precision value of TP-D method with δ

D= 0.5 has the best performance (i.e., precision value) than the other values.

Interestingly, the result shows that if we set

δ

_D= 0.5,

δ

_D= 0.8,

δ

_D= 0.9, or

δ

_D= 1, they all have similar results. Thus, the curve of TP-D method shown in Fig. 7.2 is smooth and steady from

δ

_V= 0.4 to 1. The result indicates the collaborative profile of

TP-D method has no significant influence to this experiment. We may take a further

analysis in the Experiment 2.

Table 7.2 Effectiveness of the TP-D method under various δ

_D

values

δ

_D

Top-N

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Top-5 0.138 0.231 0.292 0.323 0.323 0.338 0.338 0.338 0.354 0.354 0.354 Top-10 0.185 0.262 0.285 0.346 0.369 0.385 0.385 0.392 0.385 0.385 0.385

Top-15 0.169 0.231 0.303 0.354 0.379 0.369 0.354 0.354 0.359 0.359 0.359

Top-20 0.162 0.25 0.308 0.331 0.346 0.331 0.319 0.319 0.315 0.315 0.319

Average 0.163 0.243 0.297 0.338 0.354 0.356 0.349 0.351 0.353 0.353 0.354

Fig. 7.2 Result of knowledge support of TP-D under various δ

_D

value 7.2.3. Experiment 2: comparisons of self profile adaptation methods

This experiment aims to compare the performance of task-relevant document support between the four methods: PP, PP-T, PP-P, and TP under various top-N supported documents. PP method is the baseline method as described in Section 7.1.1, which solely adjusts task profile based on the document profiles of documents accessed, but PP-T and PP-P methods consider the time factor and topic profiles respectively. The TP method is the self profile adaptation method which adjusts task

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

profile based on the document profile, topic profiles and time effect. Herein, Table 7.3 shows the performance of the four methods in terms of precision, recall and F-measure under various top-N documents.

Observation 1: Table 7.3 and Fig. 7.3 shows that the average values of precision,

recall, and F-measure of proposed PP-T, PP-P, and TP methods are better than those of the baseline method, PP method, under various top-N retrievals. The result reveals that it is effective to consider the topic profiles and the time factor during the profile adaptation process.

Observation 2: Fig. 7.3 shows the curve of each method under various top-N

supported documents. Notice that TP method is better than the PP-T and PP-P methods under various top-N supported documents. Consequently, the result implies that the TP method is more effective than the other three methods, by considering both the topic profiles and the time factor during the profile adaptation process.

Table 7.3 Comparison between self profile adaptation methods

PP method PP-T method PP-P method TP method

method

Top-N Pre. Re. F. Pre. Re. F. Pre. Re. F. Pre. Re. F.

Top-5 0.215 0.030 0.051 0.246 0.035 0.060 0.215 0.022 0.040 0.354 0.041 0.073

Top-10 0.215 0.055 0.085 0.269 0.068 0.105 0.262 0.055 0.090 0.385 0.092 0.145

Top-15 0.215 0.083 0.115 0.277 0.105 0.146 0.287 0.097 0.140 0.359 0.132 0.186

Top-20 0.204 0.102 0.129 0.277 0.138 0.175 0.277 0.125 0.166 0.319 0.155 0.200

Average 0.212 0.067 0.095 0.267 0.086 0.121 0.260 0.075 0.109 0.354 0.105 0.151

Fig. 7.3 Trends of retrieval effectiveness of the four methods under various top-N document support

0.2 0.3 0.4 0.5

PP PP-T PP-P TP

PP 0.215 0.215 0.215 0.204

PP-T 0.246 0.269 0.277 0.277

PP-P 0.215 0.262 0.287 0.277

TP 0.354 0.385 0.359 0.319

Top-5 Top-10 Top-15 Top-20

7.2.4. Experiment 3: comparisons of various methods

This experiment aims to compare the performance of task-relevant document support between the four methods: PP, TP, TP-V (with

δ

_V= 0.7), and TP-D (with

δ

D= 0.5) under various top-N supported documents. PP method is the baseline method as described in Section 7.1.1, which solely adjusts task profile based on the document profiles of documents accessed. The TP method is the self profile adaptation method which adjusts task profile based on the document profile, topic profiles and time effect. The TP-V method and TP-D method further consider the effect of collaborative profiles generated from similar workers. The parameters

δ

_V and

δ

_D are used to adjust the relative importance of the worker's self task profile and the collaborative profile in TP-V method and TP-D method, respectively. According to the result of experiment 1-1 and 1-2, the

δ

_V= 0.7, and

δ

_D= 0.5 can achieve the best performance. Herein, Table 7.4 shows the performance of the four methods in terms of precision, recall and F-measure under various top-N documents.

Observation 1: Table 7.4 shows that the average values of precision, recall, and

F-measure of proposed TP-V, and TP-D methods are far better than those of the baseline method, PP method, under various top-N retrievals. The result reveals that it is effective to consider the collaboration from similar workers during the profile adaptation process.

Observation 2: Fig. 7.4 shows the curve of each method under various top-N

supported documents. The TP-V method performs slightly better than the TP and

TP-D methods. Collaborative adaptation of task profiles based on similar workers’

topic variations is effective to improve the quality of document retrieval.

Table 7.4 Comparison between methods

PP method TP method TP-V method TP-D method

method

Top-N Pre. Re. F. Pre. Re. F. Pre. Re. F. Pre. Re. F.

Top-5 0.215 0.030 0.051 0.354 0.041 0.073 0.369 0.045 0.079 0.338 0.040 0.070

Top-10 0.215 0.055 0.085 0.385 0.092 0.145 0.408 0.098 0.153 0.385 0.091 0.143

Top-15 0.215 0.083 0.115 0.359 0.132 0.186 0.385 0.144 0.200 0.369 0.135 0.190

Top-20 0.204 0.102 0.129 0.319 0.155 0.200 0.358 0.181 0.229 0.331 0.160 0.206

Average 0.212 0.067 0.095 0.354 0.105 0.151 0.380 0.117 0.165 0.356 0.107 0.152

Fig. 7.4 Trends of retrieval effectiveness of the four methods under various top-N document support

7.2.5. Case inspections

The overall experimental results demonstrate that the TP-V method performs slightly better than the other methods with

δ

_V= 0.7; that is, the self-adapted profile gets higher weight than the collaborative profile in the TP-V method. We conduct further inspections of each case and find that some cases have significant improvement of retrieval effectiveness. Fig. 7.5 shows three cases (case 1, 2, and 3) with large degree of topic-needs variation and one normal case (case 4) with small degree of topic-needs variation.

Fig. 7.5 Four experimental cases

0.2 0.3 0.4 0.5

precision value PP

TP TP-V TP-D

PP 0.215 0.215 0.215 0.204

TP 0.354 0.385 0.359 0.319

TP-V 0.369 0.408 0.385 0.358 TP-D 0.338 0.385 0.369 0.331 Top-5 Top-10 Top-15 Top-20

Case 1 - ml

0 0.2 0.4 0.6 0.8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

TP-V TP-D

Case 2 - noin

0 0.1 0.2 0.3 0.4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

TP-V TP-D

Case 3 - jessie

0 0.2 0.4 0.6 0.8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

TP-V TP-D

Case 4 - nancy

0 0.2 0.4 0.6 0.8

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ value

precision value

TP-V TP-D

Observation 1: Fig. 7.5 shows that the precision value of the four cases using

proposed TP-V and TP-D methods under various δ values. For the TP-V method, cases 1, 2, and 3 perform better under δ = 0.4 or δ = 0.5; that is, the collaborative profile gets higher weight than the self-adapted profile (δ = 1). The normal case (case 4) has better performance under δ = 0.7 for TP-V method, and has better performance under δ = 0.3~1 for TP-D method. The result reveals that it is more effective to give higher weight on the collaborative profile during the profile adaptation process for cases with large degree of topic-needs variation, but more effective to give higher weight on the self-adapted profile for the normal case.

Fig. 7.6 Trends of retrieval effectiveness of the three cases (case 1, 2,3) under various top-N document support

Observation 2: Fig. 7.6 shows the average retrieval effectiveness of the three cases

(case 1, 2, and 3) under various top-N document supports. The curve shows that TP-V method performs better than PP, TP and TP-D methods under various top-N supported documents. Consequently, the TP-V method considering collaborative adaptation based on topic-needs variation is more effective than the other three methods, especially for the cases which have large degree of topic needs variation.

7.2.6. Discussions

The overall experimental results demonstrate that the proposed novel adaptive

task-profiling technique is effective. Generally, the more recent the document accessed

the more important it is to reflect a work's current task needs. Thus, the time factor is important for profile adaptation process. Experimental result shows that the proposed

adaptive task-profiling techniques perform better than the baseline method - PP

method which does not consider topic taxonomy and time effect.

0.2 0.3 0.4 0.5 0.6

precision value PP

TP TP-V TP-D

PP 0.333 0.333 0.356 0.3

TP 0.4 0.4 0.467 0.433

TP-V 0.467 0.433 0.556 0.55

TP-D 0.333 0.4 0.489 0.45

Top-5 Top-10 Top-15 Top-20

Moreover, the proposed profiling technique adopts a novel collaborative profile adaptation approach to adjust task profiles. We analyze the variations of workers' task needs on the topic taxonomy to identify workers with similar variations of topic needs over time. Similar workers' variations of topic needs are used to predict the target worker's future variations of topic needs, and are used to adjust the target worker's task profile. Accordingly, the TP-V and TP-D methods are proposed to examine the effect of the profiling via collaboration. The result is interesting, and we summarized the results with the associated discussions below.

Generally, the TP-V, TP-D, and TP methods have similar results. That is, the similar workers' variations of topic needs or document access can be adopted to predict the target worker's future topic needs but can only slightly improve the retrieval effectiveness of self task profile (TP method). However, for workers with large degree of topic-needs variation, collaborative adaptation of task profile based on similar workers’ topic-needs variation can gain significant improvement of the retrieval effectiveness. From the result demonstrated in Section 7.2.5, we can infer that the collaborative profile is more important than the self task profile for the cases with large degree of topic-needs variation. In future work, we plan to improve the

TP-V and TP-D methods by adjusting the δ value according to the degree of

task-needs variation.

8. Conclusion and future research issues

In this work, we propose a self profile adaptation approach to model the task needs of the workers by considering the knowledge activities of workers and the effect of time factor. Moreover, the variations of topic needs over time are also measured by the proposed approach. With the help of topic needs variation, we also propose approaches to find the similar workers by identifying similar topic needs variation over time, and to predict potential task needs with combination of the collaborative profile, i.e., the topic-variation profiles in TP-V method and the document profiles in

TP-D method. In addition, an interface is implemented to show the detailed

information after each operation of proposed methodology mentioned in this work, and two experiments are conducted. From the experimental results, we find that the proposed enhancement of adaptation approach is effective. The TP method which represents the proposed event-based self profile adaptation approach also performs well in the experiments. However, during the period of conducting experiments, we also find there are some limits in our methodology. As a result, these issues are addressed as follows to be further investigated in the future.

(1)Improvement of profiling approach: In this work, our proposed self profile

adaptation approach is event-based approach in which a document access triggers the adaptation process. An event-based approach may result in overreaction because workers may not always access the documents they really need. In order to ease the situation of overreaction, a transaction-based profiling approach, which considers documents accessed within a period as a transaction, is worth to be explored. The time weight used to reflect the significance of time variance during the execution of task can be computed by adopting the time cancroids of transactions.

(2)Refinement of topic taxonomy: The topic taxonomy used in this work is inherited

from previous research [19], which consists of thirty-six topics. Nevertheless, we find some cases conducting the opposite result to that in experiment 1-1. By using the interface to observe the details, we find that the similar workers identified by our proposed approach can not provide enough support for them, since their relevant topics are not contained in the topic taxonomy. For this reason, the refinement of topic taxonomy continuously with emerging topics is important to model workers’ task needs.

Reference

[1] A. Abecker, A. Bernardi, K. Hinkelmann, O. Kuhn, M. Sintek, "Context-aware, proactive delivery of task-specific information: the KnowMore Project,"

Information Systems Frontiers, 2(3/4), pp.253-276, 2000a.

[2] A. Abecker, A. Bernardi, H. Maus, M. Sintek, C. Wenzel, "Information Supply for Business Processes: Coupling Workflow with Document Analysis and Information Retrieval," Knowledge Based Systems, 13(1), pp.271-284, 2000b.

[3] R. Baeza-Yates, B. Ribeiro-Neto, "Modern Information Retrieval," New York: The ACM Press, 1999.

[4] M. Balabanovi'c, "An Adaptive Web Page Recommendation Service," Proceedings of the First International conference on Autonomous Agents, Marina del Rey, CA, pp.378-385, February, 1997.

[5] N.J. Belkin, W.B. Croft, "Information Filtering and Information Retrieval: Two Sides of the Same Coin?" Communications of the ACM, 35(12), pp.29-38, 1992.

[6] P. D. Bra, G. J. Houben, F. Dignum, "Task-Based Information Filtering: Providing Information that is Right for the Job," Proceedings of INFWET97, http://wwwis.win.tue.nl/infwet97/proceedings/task-based.html, 1997.

[7] J.S. Brown, P. Duguid, "Organization Learning and Communities of Practice,"

Organization Science, 2(1), pp.40-57, 1991.

[8] A. Celentano, M. G. Fugini, S. Pozzi,"Knowledge-based document retrieval in office environment: The Kabiria system," ACM Transactions on Information Systems, 13(3), pp.237-268, 1995.

[9] T. H. Davenport, L. Prusak, "Working Knowledge: How Organizations Manage what they know," Boston MA: Harvard Business School Press, 1998.

[10] J. Davies, A. Duke, A. Stonkus, "OntoShare: Using Ontologies for Knowledge Sharing," Proceedings of the 11th International WWW Conference WWW2002, Hawaii, USA, 2002.

[11] K. D. Fenstermacher, C. Marlow, "Supporting Consultants with Task-Specific Information Retrieval," Proceedings of The American Association of Artificial Intelligence, Orlando Florida: AAAI Press, 1999.

[12] K. D. Fenstermacher, "Process-Aware Knowledge Retrieval," Proceedings of the 35th Hawaii International Conference on System Sciences, Big Island, Hawaii, USA, pp.209-217, 2002.

[13] G. Fischer, J. Ostwald, "Knowledge Management: Problems, Promises, Realities, and Challenges," IEEE Intelligent Systems, 16(1), pp.60-73, 2001.

[14] D. Goldberg, B. M. Nichols, Oki, D. Terry, "Using Collaborative Filtering to Weave an Information Tapestry," Communications of the ACM, 35(12), pp.61-70, 1992.

[15] P. H. Gray, "The Impact of Knowledge Repositories on Power and Control in the Workplace," Information Technology & People, 14(4), pp.368-384, 2001.

[16] J. Hahn, M. Subramani, " A Framework of Knowledge Management Systems:

Issues and Challenges for Theory and Practice," Proceedings of the 21st International Conference on Information Systems, Brisbane, Australia, pp.302-312, 2000.

[17] A. Kankanhalli, F. Tanudidjaja, J. Sutanto, C.Y. Tan (Bernard), "The Role of IT in Successful Knowledge Management Initiatives," Communications of ACM, 46(9), pp.69-73, 2003.

[18] J. Koh, Y.-G. Kim, "Knowledge Sharing in Virtual Communities: An e-Business Perspective," Expert Systems with Applications, 26, pp.155-166, 2004.

[19] D.-R. Liu, I.-C. Wu, and K.-S. Yang (2005), "Task-based K-Support System:

Disseminating an Sharing Task-relevant Knowledge," Expert Systems with Applications, 29(2), pp.408-423, 2005.

[20] D. W. McDonald, M.S. Ackerman, "Expertise Recommender: A Flexible Recommendation System and Architecture," Proceedings of the ACM 2000 Conference on Computer Supported Cooperative Work, Philadelphia, PA, pp.231-240, 2000.

[21] S. E. Middleton, N. R. Shadbolt, D. C. Roure, "Ontological User Profiling in Recommender Systems," ACM Transaction on Information Systems, 22(1), pp.54-88, 2004.

[22] J. Mostafa, S. Mukhopadhyay, W. Lam, M. Palakal, "A Multi-level Approach to Intelligent Information Filtering: Model, System and Evaluation," ACM Transactions on Information Systems, 15(4), pp.368-399, 1997.

[23] S. Mukhopadhyay, J. Mostafa, M. Palakal, "An Adaptive Multi-level Information Filtering System," Proceedings of the Fifth International Conference on User Modeling, pp.21-28, 1996.

[24] I. Nonaka, "A Dynamic Theory of Organizational Knowledge Creation,"

Organization Science, 5(1), pp.14-37,1994.

[25] J. Park, S. Hunting, "XML Topic Maps: Creating and using Topic Maps for the Web," Boston MA: Addison-Wesley, 2003.

[26] M. Pazzani, J. Muramatsu, D. Billus, "Syskill & Webert: Identifying Interesting Web Sites," Proceedings of the Thirteen National Conference on Artificial Intelligence, Portland, Oregon: AAAI Press., pp.54-61, 1996.

[27] M. Pazzani, D. Billsus, "Learning and Revising User Profiles: The Identification of Interesting Web Sites," Machine Learning 27, pp.313-331, 1997.

[28] P. Resnick, H.R. Varian, "Recommender Systems," Communications of the ACM, 40(3), pp.56-58, 1997.

[29] C. J. v. Rijsbergen, "Information Retrieval," Butterworths, London, 1979.

[30] J. J. Rocchio, "Relevance Feedback in Information Retrieval," in: G. Salton (Eds), The SMART Retrieval System: Experiments in Automatic Document Processing, Englewood Cliffs, NJ: Prentice Hall, pp.313-323, 1971.

[31] T. Rodden, "A survey of CSCW Systems," Interacting with Computers, 3(3), pp.319-353, 1991.

[32] G. Salton, C. Buckley, "Term Weighting Approaches in Automatic Text Retrieval,"

Information Processing & Management, 24(5), pp.513-523, 1988.

[33] G. Salton, C. Buckley, "Improving Retrieval Performance by Relevance Feedback,"

Journal of the American Society for Information Science, 41(4), pp.288-297, 1990.

[34] F. Sebastiani, "Machine Learning in Automated Text Categorization: A Survey,"

Technical report, Istituto di Elaborazione dell'Informazione, C.N.R., Pisa, Italy, 1999.

[35] A. Sieg, B. Mobasher, R. Burke, "Inferring User's Information Context: Integrating User Profiles and Concept Hierarchies,", Proceedings of the 2004 Meeting of the International Federation of Classification Societies, Chicago, USA, 2004.

[36] P. L. Wang, D. Soergel, "A Cognitive Model of Document Use during a Research Project. Study I. Document Selection," Journal of the American Society for Information Science and Technology, 49(2), pp.115-133, 1998.

[37] P. L. Wang, M. D. While, "A Cognitive Model of Document Use during a Research Project. Study II. Document at Reading and Citing stage," Journal of The American Society for Information Science and Technology, 50(20), pp.98-114, 1999.

[38] E. Wenger, R. McDemott, W. Snyder, "Cultivating Communities of Practice,"

Harvard Business School Press, 2002.

[39] D. H. Widyantoro, T. R. loerger, J. Yen, "Learning User Interest Dynamics with a Three-Descriptor Representation," Journal of the American Society for Information Science and Technology, 52(3), pp.212-225, 2001.

[40] I.H. Witten, A. Moffat, T.C. Bell, Managing Gigabytes: Compressing and Indexing Documents and Images, 2^nd edn., Morgan Kaufmann Publishers, Los Alto, USA, 1999.

[41] I-C. Wu, D.-R. Liu, W.-H. Chen, "Task-stage Knowledge Support Model: Coupling User Information Needs with Task Stage Identification," Proceeding of the IEEE 2005 International Conference on Information Reuse and Integration (IRI 2005), Las Vegas, USA, Aug. 2005, pp. 19-24.

[42] M. H. Zack, "Managing Codified Knowledge," Sloan Management Review, 40(4), pp.45-58, 1999.

在文檔中 透過分析資訊需求變動修正工作特徵檔以提供工作相關資訊 (頁 38-0)