DOI:10.6245/JLIS.2016.421/699
70 Journal of Library and Information Science 42（1）：70 – 71（April, 2016）
Interactive Information Retrieval in the Work
Context: the Challenge of Evaluation
Pia Borlund
Royal School of Library and Information Science, University of Copenhagen, Denmark
E-mail: pia.borlund@hum.ku.dk
This extended abstract addresses the challenge of how to evaluate interactive information retrieval (IIR) that takes place in work settings, as defined, or characterized, by the European Network for Work Information research network (ENWI). ENWI shares a common interest in the study of workplace information practices (http://www.enwi.org). The overall objective of ENWI is to better understand people and their work tasks, and how information is part of people’s work in order to support their information practices and knowledge work (Borlund, Mandl, & Womser-Hacker, 2013). Borlund and colleagues (2013) further explain how the study of information practices in the work environments goes beyond search activities and social studies. As viewed by ENWI, information practices integrate the study of people, social structures, technology and their interaction. Therefore, search technology and its integration into work tasks and the construction of information ecologies around retrieval technology to facilitate knowledge work require a holistic approach (Borlund, Mandl, & Womser-Hacker, 2013). In other words, it calls for realistic evaluation of IIR in the work context.
Conducting holistic and realistic IIR evaluation in the work context is an immense challenge. It includes capturing and measuring recognition, expression, and satisfaction of information needs, which is further complicated because IIR of today, in both job-related and non-work contexts, takes place in seamless task switching and mobile IT environments on various platforms, including via apps (Borlund, 2016). This obviously raises the question of how to conduct realistic IIR evaluation. An evaluation approach which, as said, needs to have a holistic perspective, as an holistic insight is needed to understand how all related information practices, including IIR, take place and how they can be supported best. This insight is also valuable and applicable when designing more traditional information retrieval (IR) experimental evaluations. As Belkin succinctly posits: “We need to have experimental conditions in which system functionalities can be evaluated with respect to the ways in which they support different behaviors, and sequences of behaviors, and in which the
圖書館學與資訊科學 42（1）：70 – 71（民一○五年四月） 71
independent variables are situational characteristics which can be manipulated and whose effects can thus be investigated” (Belkin, 2008, p.50).
In a recent keynote presentation, Borlund (2016) put forward the call and invitation for constructive ideas regarding how to evaluate IIR in a realistic and/or naturalistic way. To inspire and motivate, she gave the example of how a research group at University of California, Berkeley (Gao et al., 2016) has developed a wristband that monitors health condition via sweat. Borlund expressed the desire for a similar type of wristband that sensors and logs IIR no matter the time, place, use of IT platforms or different browsers, search engines or apps. To be followed up with interviews about type, complexity, purpose, and satisfaction of information needs. So far, a suggestion to rely on Living Lab methodologies (e.g., Bergvall-Kåreborn, Holst & Ståhlbröst, 2009) has been made. All suggestions are welcome and appreciated, and the invitation is hereby repeated.
References
Belkin, N.J. (2008). Some(what) grand challenges for information retrieval. ACM SIGIR forum, 42(1), 47-54. doi:10.1145/1394251.1394261
Bergvall-Kåreborn, B., Holst, M. & Ståhlbröst, A. (2009). Concept design with a living lab approach. Proceedings of the 42nd Hawaii International Conference on System Sciences, pp.1-10. doi: 10.1109/ HICSS.2009.123
Borlund, P. (2016). Interactive information retrieval: An evaluation perspective. Proceeding of the CHIIR '16
Conference on Human Information Interaction and Retrieval, Carrboro. p.151. doi: 10.1145/
2854946.2870648.
Borlund, P., Mandl, T., & Womser-Hacker, C. (2013). The second workshop of the European Network for Work Information (ENWI). ACM SIGIR forum, 47(2), 74-77. doi:10.1145/2568388.2568401
Gao, W., Emaminejad, S., Hnin Yin Yin Nyein, Challa, S., Chen, K., Peck, A.. . Javey, A. (2016). Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis. Nature, 529 (7587), 509-514. doi:10.1038/nature16521
