CONCEPTUAL FRAMEWORK

In the following we define two terms, “a view”and “naïve ontology,” to describe our intention to improve our methodology in capturing the micro level processes of exploratory search processes as well as in expanding our understanding of knowledge-building behavior of searchers.

A View as a Unit of Analysis

We used the concept of “a view” to develop the methodology for this study. Kwasnik [2] defined the notion of “a view”as “what a person articulates as seeing at one time [in his/her physical as well as cognitive world], that is, a span of attention”and proposed to use it as a unit of analysis in browsing. By expanding physical eye movements into the internal states of cognitive and affective worlds, a view could represent a synthesized chunk of external and internal behavior at each point of move in exploratory search processes. We thus introduce the notion of “view”as the unit of analysis for holistically capturing and analyzing exploratory search processes.

Naïve Ontology for Designing Interface for Domain Novices

The increasing availability of and accessibility to Internet resources by the general public allows domain novices to interact with unfamiliar document collections. In response to such trends, the main role of ontology seems to be shifting from a search aid to a navigation tool. “Naïve ontology”in this study is a conceptual framework for the type of information access interface that allows domain novices to refine their knowledge interactively by acquiring chunk by chunk information as they encounter in digital environments. Specifically, this is a framework for designing a kind of navigation tool that supports seamless switching between searching and browsing to acquire and build basic domain knowledge that enables domain novices to explore domain collections. We introduce the concept of naïve ontology to advocate the needs for developing a new type of information access interface intended to support domain novices’exploratory searches.

METHODOLOGY

We invited a total of fifteen novice searchers (seven undergraduate and eight graduate; ten female and five male students) to participate in a series of three experiments. We used an eye tracker to record searchers’eye movements

and a HyperCam to capture screen shifts and talk aloud protocol. We showed the recorded eye movements superimposed on the screen shifts during the post search interviews to elicit searchers’views on each move during the exploratory search processes. We used a slightly different procedure in each of three experiments, as depicted in Figure 1.

Pre-search Activities

Three female college students participated in the first experiment. Each participant completed a short questionnaire that asked the frequency of their Internet use, favorite browsers and search engines, topics chosen at the college entrance examination, whether planning to get a teacher’s certificate, and topics of interest in history and geography, in addition to demographic data. Each participant was then given two vignette scenarios prepared by the author. The first scenario asked the participant to conduct a Web search to prepare for teaching a middle school history class on a particular topic; the second scenario asked the participant to conduct a Web search to prepare for visiting a world heritage site.

In the second experiment, we invited four female participants in two pairs and conducted the same questionnaire. Then we asked each pair to discuss their topics of interest in the domains of history and geography for about ten minutes to let them choose two topics for a Web search. Each participant had either a history or a geography topic. This modification in the procedure was introduced since each participant in the first experiment was tense and nervous because s/he was required to wear unfamiliar

devices, sit alone in a laboratory environment, and be observed by two researchers and a research assistant. This was also because one participant was unable to make progress in her exploratory search because of the confusion caused by encountering incomprehensible information.

In the third experiment, we invited three female and five male students. We conducted the same questionnaire with eight participants in four pairs. We then asked them to discuss their topics of interest in the domain of history and to choose two topics for a Web search, one for each participant in each pair. In addition, we gave a vignette scenario that asked the participants to conduct a search to prepare for teaching a middle school history class on any topic. The modification was introduced to capture self generated and imposed exploratory search processes.

In all three experiments, participants were asked what and how much they already knew about the topic of the search.

Recording of Exploratory Search Processes

In the first and second experiments, screen shifts and mouse movements were captured using a HyperCam. Participants’

eye movements were captured using the EMR NL8 eye tracker which required fixing searchers jaw on the stand so they could not move their head.

In the third experiment, we captured participants’responses to a talk-aloud protocol that incorporated screen shifts and mouse movements using a HyperCam. This modification was possible because we used a new eye tracker VOXER which allowed searchers to move their head to enable them to talk freely. Each session was conducted over 15-20 minutes.

Pre-Search Activities

Recording SLP

Post-Search Interview

Short questionnaire Pair of participants discuss topics

SLP by a pair of participants

Screen shifts & mouse movements (HyperCam) Eye-movements (EMR-NL8B)

Bookmark useful site (Browser)

Show eye-movements while articulating cognitive & affective states Record & transcribe interview

Constant-comparative method (ATLAS.ti) Develop a taxonomy of knowledge modification Short questionnaire

Two vignette search scenarios

SLP by single

participant Experiment 1

(three participants) Experiment 2 (four participants))

Short questionnaire Pair of participants discuss topics

Screen shifts, mouse movements

& talk aloud (HyperCam) Eye-movement (VOXER) Bookmark useful site (Browser)

Experiment 3 (eight participants)

Figure 1: Procedure.

Post-search Interview

After each exploratory search session, participants were interviewed. During the interview, we showed the video recorded eye movements superimposed on the screen to elicit the searcher’s view at each move. Specifically, we stopped the video at each move (typing, clicking, browsing, copying, pasting, bookmarking, and stopping), and asked the searcher reasons for the move as well as accompanying thoughts and feelings at that moment.

Data Analysis

Both pre search activities and post-search interviews were voice recorded and transcribed in detail. For the third experiment, talk aloud protocols during the exploratory search processes were also transcribed and compared to the post search interviews to increase reliability.

The transcribed interview data were coded for knowledge acquisition using ATLAS.ti , software for qualitative data analysis, following a bottom up strategy with the constant comparative technique.

RESULTS

In the post search interview, participants articulated what they thought, felt, expected, and intended to do at each point of move, either voluntarily or in response to the interviewer’s inquiries. Analysis of the interview data lead us to develop a taxonomy of knowledge modification during the participants’exploratory search processes (Table 1). A detailed description of the taxonomy is reported elsewhere [3,4]. This taxonomy helps to increase our understanding on exploratory search processes in terms of how searches knowledge is modified while searching for

information, which should eventually lead us to develop a naïve ontology.

IMPLICATIONS

The reported pilot study provides us with some methodological implications for capturing exploratory search processes. The study also provided us with useful information for developing naïve ontology for designing an information access interface intended for domain novices.

Methodological Implications

We perceived that three new methods implemented in this study were helpful in increasing the reliability of a descriptive study of exploratory search processes. They are:

use of a “view”in data collection; showing eye movements in post search interview; and pairing of participants.

The use of an eye tracker in data collection and a “view”as the unit of analysis for exploratory search processes provided us with a coherent picture of searchers thoughts, feelings, and physical movements. First of all, we were convinced that the use of a “view”as a unit of analysis helped us capture the dynamically changing knowledge structure of domain novice searchers together with the ir externally observable behavior during their exploratory searches. In the post search interviews, participants reports of what they thought, felt, and had done related what they saw on the screen with their eye movements. They said the eye movements shown on the screen pointed to the exact places that they were looking at and that watching their own eye movements helped them recall their search processes.

Thus, the method of using an eye tracker to collect data on exploratory Web search processes and showing the

Type Definition Example of Expressions*

Adding Acquire novel information to increase knowledge

I didn’t know that the origin of Bento is in Edo culture. I have never thought about it.

Correcting Clean up a mis-understanding

I did not expect Johannes Brams to be classified here. I thought Brams is in a bit earlier in chronology, early romantic music…

Limiting Narrow down the scope of the concept

When I examined the text, it was about the Renaissance era which is quite different from the industrial revolution I was looking for… I thought industrial arts may have influenced over civil life.

Relating Relate a concept with another concept

It said that the admission ticket for Akropolis Museum and Partenon Temple are unified. So I thought the days for both of them is closed are the same. ..

Specifyin g

Increase the depth of meaning of the concept by

increasing specificity

It was written in the Christian calendar for the beginning and the end of Taisho era

Transform ing

Understand a concept in a different framework

I did not know the experiential learning programs are offered by the [Edo Tokyo Museum]… I thought it’s a pure museum

Table 1. Taxonomy of Knowledge Modification (*translated from Japanese).

recorded eye movements during the post search interview may be recommended for future studies. They will not only identify what searchers are looking at rather than what data are shown on the screen, but will also increase the reliability of data on internal processes of searchers, which eventually increases the reliability of research findings.

The method of pairing participants is helpful to elicit natural conversation during exploratory search processes as well as during post search interviews. This procedure may be particularly useful in Japan where people are shy about expressing their thoughts and feelings in front of strangers.

In a series of three experiments, we used slightly different data collection procedures. In the first experiment, we invited three participants individually and asked each of them to conduct searches. During the post search interviews, participants explained their behavior, but seemed hesitant to articulate their thoughts and feelings. In the second and third experiments, we invited participants in pairs by asking each of them to bring an intimate friend as a partner to facilitate verbal communication. During exploratory search processes, one of the pair conducted the search and the other gave suggestions. At the post search interview, the participant who conducted the search articulated what s/he did, thought, and felt, while the other participant of the pair shared her/his own perspectives. We perceived that the paring of participants helped them articulate their thoughts and feelings more freely not only during the post search interviews but also during

exploratory search processes, and provided more detailed data on their search processes compared to the first experiment .The presence of an intimate friend seems to relax participants even in the unfamiliar lab environment and leads them to a more naturalistic articulation than during one-to-one interviewing with the researcher.

Naturally generated free flowing conversation between the pair during the search processes for the second experiment led us to introduce a voice recording of think aloud protocol at the search processes in the third experiment and triangulate it with post search interview data to improve the reliability of the interview data.

Implications for Interface Design

Elicitation and analysis of novice searchers’exploratory search processes lead us to make design suggestions for an information access interface intended for domain novices as synthesized in Table 2. Some of the functionalities proposed in Table 2 have been reflected in recently developed navigation tools, such as the Flamenco system [5] and Junii+ [6] that seamlessly combine browsing and searching. As demonstrated in these tools, an information access interface to support exploratory search processes is expected to be naturalistic for novice searchers to enable them to follow their own “view”of the world developed through everyday life experiences without being disturbed by the mandatory use of unfamiliar ontology intended for domain experts.

Aspect Implications

Content

provide high-quality pictures and visual images

present correlations among people and incidents using graphics provide dictionary of historical figures and events

present the content in a simple manner use colors to categorize texts

provide glossary of technical terms

Browsing

enable to browse time and space simultaneously or interchangeably enable to browse time and space from broad to specific

enable to trace and overview historical transitions of people, culture, religions, and political rulers of each geographic area or country

enable to choose pictures from among thumbnails

Navigation

provide metadata on genre of contents

allow switching between different calendar systems provide links to contemporaneous figures and incidents

provide historical and geographic frameworks people and incidents

provide links to free online translation services or enable to switch languages Table 2. Implications for Interface Design.

CONCLUSION

We conducted a series of three experiments in a case study in order to seek for optimal method for identifying characteristics of exploratory search processes, using the topic of history and geography as domain instances and invited fifteen students as novice domain searchers. We used a slightly different procedure for each experiment to seek an optimal method.

The results we obtained have useful implications for design of naïve ontology as the bases for an information access interface intended for novice domain searchers. The study also provided us with implications for three new methods for capturing exploratory search processes: the use of a

“view”as a unit of analysis that may enables us to capture both physical and cognitive processes together in a reliable manner; the use of an eye tracker in capturing the exploratory search processes and showing eye movements superimposed on screen shifts to participants in post search interviews which seemed to help participants recall their internal processes; and pairing of participants, which seemed to help them freely articulate their thoughts and feelings, leading to elicitation of a rich and thick description of the processes.