Photo Editors’ Search Tactics and Moves for
Image Retrieval
Tsai-Youn Hung
Associate Researcher,
National Museum of Taiwan Literature, Tainan, Taiwan (R.O.C.)
Email: [email protected]
〦⼫⥱ņKeywordsŇ
ŘImage RetrievalřSearch TacticsřSearch MovesřSearch Patterns.
μAbstractν
The objective of this study is to investigate
photo editors’ search tactics and moves
during their searching for images. This
study involves 30 photo editors from
newspaper and magazine companies using
the Associated Press Photo Archive
Database to retrieve specific, general, and
subjective photos. The results show that
task type affected the choice of which tactics
and moves to employ. More subjective
tasks lead to more complex moves. Browse,
enlarge/caption, enlarge, alter, and caption
were top five common tactics used across
the three searches. The results indicate that
image searching is an intensively visual
activity and browsing is the key move for
image searching. This study suggests that a
more responsive and powerful interface
should be implemented to allow searchers to
have more control when querying and using
browsing features.
INTRODUCTION
The tremendous growth in the amount of digital images is driving the need of more effective methods
for storing, searching, and retrieving such image data. The need could be far greater as the number of images now available on the Internet far exceeds any single picture collection. Over the decade, researchers have developed well established techniques and tools making significant progress in improving access (Datta et al., 2008). For example, allowing users to form queries by sketching a query images or by filling-in an image example to retrieve objective image. However, it focuses on the technical aspects of image indexing and retrieval. It lacks a theoretical background for the design of the image systems (Enser, 1995; Rasmussen, 1997).
In designing systems that provide access to image resources, it is important to have information on the users themselves. In the late 1990s, a few user-oriented studies in image retrieval were reported and typically these were based on analyzing user queries received by a manually operated archive to identify image attributes which were employed by users to search images (Enser & McGregor, 1992; Ornager, 1997; Keister, 1994; Markey, 1988; Hastings, 1995; Jorgensen, 1994, 1996). The results of these studies were significant and shed light on the image indexing problem. However, most of these studies were conducted in non-digitized collections with written or verbal queries. Users’ feedback, while
interacting with systems, was not considered in these studies. Thus, these studies cannot explain users’ information searching behavior as to how users formulate their search terms and how they interact with systems on the way to find visual information. As more and more users obtain visual information from image retrieval systems, there is a need for understanding their searching behavior. This study is based on the idea to investigate photo editors’ image searching behavior.
LITERATURE REVIEW
Many studies of online search strategies have focused on text-based retrieval activity and identified some distinctive process models. However, there is comparatively less focus on search strategies for image retrieval.
Batley (1988) was the first researcher to examined forty-one subjects’ visual information search strategies by giving them different types of tasks. The results show that the use of browsing increased as the specificity of given tasks decreased. Keywords were used mostly in searching for images for specific tasks. Abstract or subjective tasks mostly led to random browsing. The authors concluded that information need and individual characteristics might determine visual information search strategies.
A similar study was done by McDonald and Tait (2003). They examined subjects’ ability to construct visual queries of three different tasks with a content-based image retrieval system. The results indicate that a relationship between tool choice and search task. For images that the subjects had previously seen, they used the navigation and sketch tools most frequently. For images that contained particular objects, subjects chose to browse the collection serially with the most frequency.
Over the past years a considerable number of studies have been conducted on information retrieval studies for information over the Internet. Many of these
studies have only concentrated on text information retrieval not image based information. Hsieh-Yee (1998) examined search tactics employed by users for searching for images on the Web. The results show that some Web users’ search tactics were similar to those for online searching searches, but several tactics were unique to Web searching, such as, switching search engines, selecting items for possible hyperlinks, and checking Websites. Goodrum et al. (2003) examined and categorized the search moves made by eighteen graduate students searching images on the Web. The results show that, on average, subjects input two queries, spend twenty minutes searching per image, and change their initial queries frequently. Most of the time was spent on browsing surrogates and websites, and inspecting images for relevance. More recently Fukumoto (2006) analyzed twenty undergraduate students’ image searching behavior by using search engines on the Internet. The results reveal that subjects tended to follow a set search strategy. Many of the moves indicate that subjects’ behavior differed depending on if the task set was presented as a closed or open task.
Markkula and Sormunen (1998) investigated journalists’ searching behavior and the criteria they applied in selecting photos. They found that journalists’ typical queries were mainly single word or single phrase queries and based on proper names of people, geographical places or buildings. Browsing was the primary strategy used by the journalists.
Frost et al. (2000) developed a prototype image retrieval system to investigate art history specialists and generalists’ search patterns. The results show that browsing is a useful strategy for image searching both for specialists and generalists.
Most recently Westman et al. (2008) studied journalism professionals and non-professionals’ search strategies for image retrieval by using a prototype multimodal image retrieval system which includes text, color, sketch, quality, and category
combinable query modes. The results show that task type affected the use of query modes, e.g., conceptual tasks led to employ text and category only. User background also affected the types of queries used.
The results of these previous studies show that a visual information retrieval session just like textual information retrieval session involves not only searching and matching but also other processes including formulating queries, interpreting retrieved items, changing strategies, etc. People engage in multiple types of strategies within an image searching episode, and change strategies during the course of a search. Context and task type affect the use of strategies.
Compared to prior text-based information retrieval (Chen & Dhar, 1991; Wildemuth et al., 2004; Nahl & Tenopir, 1996), it shows that the strategies users employed for image retrieval is different from that used for text-based information retrieval. It is believed that because of the obvious differences in nature between visual information and text. Visual information is different from text in representing and expressing information. Words cannot entirely represent or interpret an image. A same image might mean something different to different people, even the same person at a different time.
These previous studies investigated users’ search strategies employed during the course of a search; however, the interaction with systems was not examined in details. Journalism is one of the domains in which people utilize many digital photos. This study is to examine the mirco-dynamics of image searching process of photo editors. The aim is to gather and analyze empirical data on photo editors’ search tactics and moves as they proceed and interact with image systems to solve given tasks from the start point to the end of a search.
METHODOLOGY
1. Participants
Thirty Photo editors were recruited from newspaper and magazine companies from the New Jersey, New York, and Philadelphia areas. All the participants had prior experience in conducting image database searches.
2. Database
System
This study used the Associated Press Photo Archive database system. The Associated Press Photo Archive has been widely used in the fields of history, journalism, political science, and art.
It contains over 700,000 photos and dates back over 150 years. Each picture is accompanied by a 50-to-75-word caption that fully describes the person or event in its surrounding context. Each picture is also indexed by location, photographer, date taken, date submitted, and subject keywords. Searchers can use this database not only to search for factual photos but also subjective and emotional photos.
3. Setting
The experiments of this study were conducted in 2005 at participants’ workplaces. An appointment was made with each participant before the researcher arrived at the participant’s workplace. The researcher brought a laptop and connected to the Associated Press Archive database through the Internet connection of the participants’ workplaces.
4. Tasks
The tasks of this study were that the participants were asked to search specific, general, and subjective photos from the Associated Press Archive database. The search time was not limited, and each participant had to select up to five photos for each task searched. The three types of tasks-- specific, general, and subjective, were created based on Shatford’s (1986) image analysis (Table 1).
Table 1! Shatford’s image analysis
Specific General Subjective Finding images of an individually
named person, group, thing, event, location, or action.
Finding images of a kind of person, group, thing, event, place, condition, or action.
Finding images which have emotional or abstract concepts.
Table 2! Coding scheme for search tactics and moves
Moves to enter queries I. Beginning a session
Specify—To search one term or concept that is as specific as the information desired. Exhaust—To include more terms or concepts of the query in a search formulation. II. Reducing the size of a set
Intersect 1—To intersect new search terms by using “AND” or “NOT” operator. Intersect 2—To change operators from “OR” to “AND”.
Add 1—To add at least one term of the query from a previous search formulation to reduce the size of a set by using the “AND” operator.
Reduce 1—To remove one term of the query from a previous search formulation which used the “OR” operator. Refine—To add definers without changing other original terms.
Proximity—To locate on term within a certain distance of another. III. Enlarging the size of a set
Reduce 2—To remove at least one term of the query from a previous search formulation which used the “AND” operator.
Add 2—To add at least one term of the query from a previous search formulation to enlarge the size of a set by using the “OR” operator.
Expand 1—To expand new search terms by using the “OR” operator. Expand 2—To change operators from “AND” to “OR”.
IV. Increasing both precision and recall
Alter—To use new search terms to replace all search terms from the previous search formulation.
Vary—To substitute one’s search terms in any of several ways; the number of terms does not change but at least one new term was substituted.
Parallel—To substitute search terms to other conceptually parallel terms.
Trace— To examine information already found in order to obtain additional terms for the search. Correct— To watch for and correct spelling and factual errors in one’s search topic.
V. Others
Repeat—To repeat the previous search formulation without changes. Moves to select relevant images Browse—To browse thumbnails displayed in one screen.
Enlarge—To enlarge thumbnails.
Enlarge&Caption—To enlarge thumbnails and read the caption. Caption—To read image captions without enlarging the photos.
5. Topics
Under each task, a topic was provided. The selection of topics was considered based on the distribution of photos across the database and the subject of topic. It is believed that the distribution of relevant photos could affect individual’s searching behavior, such as, relevance judgment, time spent on
searching, and search moves, etc. Topic subject was also considered as a selection factor, because topic subject was known to affect search process and results. Thus, to avoid this, topics should be selected which would be accessible with general knowledge and non-technical subject areas. Based on the considerations, three topics— US presidential election
(specific), Terrorist (general) and Peace (subjective) — were chosen from the top downloaded topics of photos for the year of 2004 by the users of the Associated Press Archive database.
6. Text
Queries
Under each topic, a text query related to the topic was provided. The three text queries were selected from LexisNexis Academic database, which covers full-text of more than 50 major English-language newspapers from the U.S. and around the world, and more than 400 magazines and journals and over 600 newsletters.
7. Online Search Process
During each participant’s online searching, a user guide of how to use the Associated Press Photo Archive database was provided. CyberCam, a screen capture software system, was used to capture every search move of the participants on the screen. Each participant had to conduct one image search for each of the given topics. Participants were divided into three groups of ten subjects. Group one performed the specific search first; group two performed the general search first; and group three performed the subjective search first. This was to eliminate the learning and fatigue effects of using the photo archive.
DATA ANALYSIS
The collected data for each of the three tasks were analyzed separately, so that comparisons could be made. The transaction logs were coded using a coding scheme adapted from Bates’ (1979a) search tactics, and Fidel’s (1985) online searching moves. Table 2 is the coding scheme used to identify search tactics and moves in this study. The coding scheme can be categorized into two groups: moves to enter queries (i.e. tactics for query formulation) and moves to select relevant images (i.e. searching tactics by interacting with interface to find relevant images).
RESULTS and DISCUSSION
To identify the search tactics and moves employed by the searchers, each search statement was categorized using the adapted coding scheme. A tactic here is defined as a strategy that is consciously applied to solve an information-searching problem; a move refers to a change of shift from one state to another.
1. Search
Tactics
The frequency with which each tactic was used in the three searches is displayed in Table 3. The searchers employed tactics 2848 times in the three searches, 986 times (34.6%) in the specific search, 952 times (33.4%) in the general search, and 910 times (32%) in the subjective search. The most common tactic used across the three searches was “Browse”. This tactic was used 1647 times (57.83%). It indicates that searchers spent most of their time browsing trying to find relevant photos. In the specific search, searchers employed the “Browse” tactic 560 times which accounted for 56.7% of the total, while in the general and subjective searches, searchers used “Browse” tactic 551(57.8%) and 536 (58.9%) times respectively. Though in number they were not significantly different, there was somewhat of a difference when using the “Browse” tactic in the three searches. In the specific search, searchers heavily used the “Browse” tactic, but the browsing time at each page was shorter than those in the general and subjective searches. In the general search, the browsing time of each page was comparatively longer; furthermore, some searchers browsed previous pages. The searchers spent the longest time on browsing each page when conducting the subjective search task. The reason that the “Browse” tactics was heavily used could be that the searchers not only needed to browse the retrieved sets, but also needed to use the “Browse” tactic to compare the retrieved sets to avoid selecting similar photos. For news or magazine photo editors, they seldom put similar photos in a section of an article.
The second common tactic was “Enlarge/Caption”. The searchers employed the tactic to enlarge photos first and then read the captions. The tactic was used 471 times accounting for 16.54% of all tactics. In the specific search, the searchers used it 144 times representing 30.57% of the 471 times used, while in the general and subjective searches, the searchers used it 184 (39.07%) and 143 (30.36%) times respectively. There was no significant difference between the three searches. The searchers heavy use of the “Enlarge/ Caption” tactic might be accounted for by noting that each person may have his/her own interpretation of a photo. The interpretation could be completely different than other people. Even though the photo looked relevant, the searchers needed to know what was really happening in the photo and what its background was. To make sure that their interpretations of the photo matched the photo’s description provided by the database, the searchers needed to enlarge the photos and compare the enlarged photos with the attached caption. The number shows that searchers in the general search used the “Enlarge/Caption” tactic more. It explains that when searching for photos which contain general meaning, the searchers need textual information to make relevance judgments.
The third common tactic was “Enlarge”. Searchers used this 134 times accounting for 4.71% of the total number of tactics. It was used 84 times during the specific search, 20 times during the general search, and 30 times during the subjective search. The searchers in the specific search used the “Enlarge” tactic 84 (62.09%) times. The reason could be that photos retrieved in the specific search have more direct meaning. The searchers only needed to enlarge the photos, and not to check the accompanying caption. This also explains why searchers used the “Enlarge/Caption” tactic less in the specific search, whereas in the general search the searchers used the “Enlarge/Caption” more, but used the “Enlarge” tactic less.
The fourth common tactic was “Alter”. The searchers used this tactic to replace all search terms from the previous search formulation with new terms. The “Alter” tactic occurred 122 times accounting for 4.28% of the total number of tactics used. In the specific search it was used 30 times (24.59%), in the general search it was used 45 times (36.89%), and in the subjective search it was used 47 times (38.52%). The result shows that the searchers used this tactic less frequently in the specific search than in the general and subjective searches, suggesting that when conducting the general and subjective searches, the retrieval sets were not satisfactory, and the searchers needed to use a variety of new terms to revise their search queries. The cause of the dissatisfaction could be that the searchers were not successful with their selection of search terms to match the relevant photos.
The fifth common tactic was “Caption”. The tactic is defined as the searchers reading the captions without enlarging the photos. The searchers used the tactics 71 times across the three searches. The tactic was used 19 times (26.76%) in the specific search, 27 times (38.03%) and 25 times (35.21%) in the general and subjective searches. The caption tactic was used more often in the general and subjective searches. For image searching, the attachment of textual information to a photo is considered as a very valuable resource to help searchers understand the relationships of elements in the photos, especially for those photos where searchers may not be familiar with the context. Searchers used the “Caption” tactic more often when they were conducting the general and subjective searches. This suggests that the searchers could make relatively quick judgments of relevance or suitability for specific photos without checking with the captions. Nevertheless, in the general and subjective searches, some searchers skipped the “Enlarge” tactic and directly clicked on the “Caption” button, trying to comprehend what the photos were about. This strongly suggests that searchers rely more on textual
description to make relevance judgments when searching images which contain general and abstract concepts.
The top 5 search tactics were used 2445 times. This is 85.85% of the total 2848 tactics used. In addition to the top 5 common tactics, there were some differences across the three searches in terms of using other tactics. For the initial querying tactics—
“Exhaust” and “Specify”, each searcher started querying by using either the “Exhaust’’ tactic which is to include most or all concepts of the query in the initial search formulation or “Specify” which is to search one term or concept in the initial search formulation. The result shows that 20 searchers in the specific search began their searches with the“Exhaust” tactic, whereas only 7 searchers in the
Table 3 Frequency of tactics employed by search tasks
Search Tactics Specific Task General Task Subjective Task Total % of total tactics
Browse 560 551 536 1647 57.83% Enlarge&Caption 144 184 143 471 16.54% Enlarge 84 20 30 134 4.71% Alter 30 45 47 122 4.28% Caption 19 27 25 71 2.49% Specify 9 23 26 58 2.04% Vary 27 14 15 56 1.97% Refine 20 19 12 51 1.79% Add2 16 17 11 44 1.54% Exhaust 20 7 4 31 1.09% Add1 12 8 7 27 0.95% Intersect 1 7 6 13 26 0.91% Trace 3 4 16 23 0.81% Repeat 5 6 8 19 0.67% Reduce 2 9 5 4 18 0.63% Parallel 5 4 4 13 0.46% Correct 4 2 3 9 0.32% Intersect 2 4 3 1 8 0.28% Reduce 1 2 4 1 7 0.25% Expand 1 2 1 3 6 0.21% Proximity 1 2 1 4 0.14% Expand 2 3 0 0 3 0.11% Total 986 952 910 2848
general search and 4 searchers in the subjective search started with the “Exhaust” tactic. Conversely, in the specific search, there were only 9 searchers who began their searches with the “Specify” tactic, whereas 23 searchers in the general and 26 searchers in the subjective began their initial search with the “Specify” tactic. This indicates that the searchers had sufficient terms to formulate their initial queries when conducting the specific search, whereas the searchers in the general and subjective searches had fewer term
to start their search. Most searchers started their searches with the term “terrorist” in the general search, and “peace” in the subjective search. This can be inferred that the searchers had difficulties in interpreting images containing general or subjective meaning, such as the general concept “terrorist” and the subjective concept “peace” into broad, narrow, or synonym terms. The finding can explain why the “Trace” tactic was only used 3 times in the specific search, but 16 times in the subjective search. When
the searchers were not able to generate more terms in their initial queries, they used the “Trace” tactic to find more terms during browsing the retrieval sets either from the photos or from other textual information. For example, searcher 10 changed her search term from “Pray, peace” to “Dalai Lama” after she browsed a photo in which the Dalai Lama was praying and meditating. Searcher 17 changed his search term from “Peace symbol” to “Dove” after he browsed a photo in which a dove shape was cut in a field. Searcher 18 changed his search term from “Pope” to “Vigil” because he saw people gathering to hold a vigil for the Pope. The ‘‘Trace’’ tactic was used often when the searchers were searching for images containing emotional or subjective meaning. However, in the general search, the searchers didn’t use the “Trace” tactic as often as in the subjective search. It could be explained that the searchers did use the tactic trying to find relevant terms but didn’t try many proper or good terms during browsing.
Since the searchers didn’t have sufficient terms to formulate their query in the general and subjective searches, they didn’t employ the “Vary” tactic as often as in the specific search. The tactic indicates to substitute one’s search terms in any of several ways use; the number of terms without changing but at least one new term was substituted. This tactic occurred in the specific search 27 times, buy only occurred 14 times in the general search and 15 times in the subjective search.
Fidel (1984) proposed two types of “querying tactics”—conceptual and operational tactics. A conceptual tactic is defined as one that modifies a retrieved set by changing the meaning of the concept it represents. For example, when a broader or a narrower term is used, or when a term combines a concept from an additional facet. An operational tactic is defined to be one that uses the system features in order to modify a retrieved set without changing the conceptual meaning it represents. For example, to
limit the retrieval to documents published in certain year, or to add synonyms and variant spelling. To investigate if there are differences in using operational and conceptual tactics among the three searches, the querying tactics used in this study were categorized based on Fidel’s definition. Among the tactics, only “Intersect 1”, Expand 1”, and “Alter” tactics were associated with changing the meaning of search statements. Thus, they were categorized as the conceptual tactics. The rest of the tactics were categorized as operational tactics.
The 30 searchers employed a total of 446 querying tactics in the three searches. Of these, 292 (65.47%) were operational and 154 (34.53%) were conceptual. In general, the searchers used more operational tactics to formulate and reformulate their search queries. Comparing the three searches, the searchers in the specific and general searches heavily used the operational tactics (specific = 74%; general = 62.9%), whereas in the subjective search the searchers used operational and conceptual tactics more evenly. The result indicates that when conducting the specific and general searches, the searchers concentrating on using the operational tactics; nevertheless, the searchers in the subjective search needed to employ both the operational and conceptual tactics.
2. Search
Moves
To analyze searchers’ search moves, each move and subsequent moves were coded in alphabetical codes. A move here refers to a transition move from one state to another. In this study, 8 move codes were used to represent the individual steps taken within the course of a search. The 8 move codes are: enter queries (q), search (s), browse (b), move to next page (n), enlarge photos (e), enlarge and read the caption (e©), read captions (c), and select photos (se). After coding, the frequency of moves generated by the 30 searchers in these three searches was summarized respectively. Table 4 shows the total frequency of individual moves made by the searchers in the three searches.
Table 4! Frequency of moves employed by search tasks
Move Specific Task General Task Subjective Task
b 560 551 536 s 173 169 172 n 284 262 277 e 84 20 30 e© 144 185 144 se 149 123 106 c 19 27 25 q 179 170 176 Total 1592 1507 1466
Table 5! Transition frequency between two moves in the specific search
b s n e e© se c q B 30 b 233 65 113 59 15 75 s 134 39 n 230 16 6 7 10 2 13 e 39 10 5 3 23 1 3 e© 48 8 1 9 68 10 se 102 20 4 7 4 3 9 c 7 3 1 8 q 179
Note: “B “stands for Begin.
Table 6! Sequential search moves for searcher 13 in the subjective search
BÆspecificÆsÆbÆe©ÆnÆbÆtraceÆsÆbÆeÆbÆe©ÆbÆcÆalterÆsÆbÆe©ÆnÆbÆtraceÆsÆb Æe©ÆbÆseÆbÆ nÆbÆe©ÆseÆb
The 30 searchers made a total of 4565 moves, 1592 moves in the specific search, 1507 in the general search and 1466 in the subjective search. The average number of moves a searcher made per query reflects the degree of interaction during a search. The searchers made an average of 11.94 moves in the specific search, with a median of 8.34; 12.77 moves in the general search with a median of 8.34; and 10.49 moves in the subjective search, with a median of 7.78. There was no significant difference in the frequency of average number of moves; however, the searchers made more “Select” moves in the specific search than
in the general and subjective searches. It may indicate that it is easier to make a relevance decision in the specific search than in the general and subjective searches.
To see transition patterns, the sequential transitions between two moves were calculated. Table 5 is an example of sequential transitions between two moves occurring in the specific search. It clearly shows that transitions occurred not only straightforward proceeding but also in reverse shifting. For example, the sequential move from “q” to “s” occurred 179 times, and the reverse move from “s” to “q” occurred
39 times. The move from “b” to “n” occurred 233 times, and from “n” to “b” occurred 230 times; from “e” to “b” occurred 39 times, and from “b” to “e” occurred 65 times.
In the query state, moves were further coded to see the transitions within the query state based on the adapted coding scheme. Table 6 is the sequential search moves for searcher 13 from the start point to the end of a search.
The discrepancies were that in the state of querying, the searchers in the specific search were engaged in more query reformulations and more transition moves and were most likely to use an “Alter”, a “Vary”, or a “Refine” move to reformulate their queries. In the general search, the searchers either used ”Alter”, or “Refine” or “Add 2” move to reformulate their queries and proceed to the “search” state, and in the subjective searches, the searchers either used “Alter”, or “Trace” to move on the “search” state.
In the “search” state, most searchers moved on to the “browse” state across the three searches, and only few searchers moved back to the “query” state in the general and subjective search, whereas quite a few searchers moved back to the “query” state to reformulate their queries in the specific search. It indicates that the searchers had more candidate terms to use when querying for specific search.
In the “browse” state, the most dominant move was to proceed to the “next page” state used across the three searches, and the searchers immediately moved back to the “browse” state.
In the “enlarge & caption” state, most searchers in the three searches moved on to the “select” state. From the “select” state, most searchers moved back to the “browse” state.
In the “enlarge” state, the searchers moved on to the “browse” state in the specific search , while in the general search the searchers moved on to the “select”
state, and in the subjective search the searchers either moved on to the “browse” state or to the “select” state in the subjective search.
In the “caption” state, the searchers moved on to the “select” state in the specific search, while in the general search the searchers moved on to the “browse” state, and in the subjective search the searchers either moved on to the “select” state or to the “browse” state.
In the “select” state, the searchers in the specific search did not select the photos after enlarging photos but moved on to the browse state a total of 39 times, while in the general search the searchers moved on the “select” state after enlarging photos, and in the subjective search the searchers either moved on to select the photos or continue to browse. On the contrary, the searchers in the specific search selected the photo after reading the caption, while in the general search the searchers moved to the “browse” state after reading the caption, and in the subjective search the searchers either moved on to select the photos or to browse other photos. The decision on selecting a photo was dependent on various previous moves among the specific, general, and subjective searches.
The commonality was that there was a dominant path across the three searches that moved to the “browse” state. It also indicates that browsing was the key move for image searching. The searchers spent nearly half of their time browsing.
To further investigate the pattern of longer sequential transitions, the Markov model was used. This model was first used by Penniman (1975) and modified by Chen and Cooper (2002). The Markov model assumes that users’ information searching behavior can be classified into a relatively small number of well-defined groups called states, and the chance of moving from a state to any other states is dependent on the previous state. There is a definite set of a probabilities governing how a search might move.
The limitations of the higher order sequential dependencies come from the fact that the chance observing the length of sequences decreased geometrically with the length of the chain. Thus, in this study, the sequential dependencies were identified up to the third order.
In the first-order sequential dependencies, the transition patterns show that most transitions forwarded to the “browse” state across the three searches. The difference was that all the transitions in the specific search were all linear transitions, while in the general and subjective searches some reverse transitions occurred. Also in the subjective search, the high probability of transition from “caption” to “browse” shows that after reading the caption, the next move was not to select the photo but proceed to browse other photos, which was different from the patterns, from “Enlarge& Caption” to “select”, in the specific and general searches.
In the second-order sequential dependencies, the commonality among the three searches was the existence of reiterative transition, i.e., “bÆ nÆ b” and most transitions forwarded to the “browse” state. The difference was when making the selection decision, the
searchers in the specific search shifted forward to the “select” state directly from the “browse” state, while in the general and subjective searches, before making the selection decision, the searchers would enlarge the photo and read the captions, or in the general search the searchers would read the caption first and then select the photo.
In the third-order sequential dependencies, the commonality among the three searches was the reiterative and loops that occurred between states, for example, sÆbÆnÆb and nÆbÆnÆb. The difference was searchers in the specific search concentrated on employing the moves, such as “selection”, “enlarge” and “enlarge & caption” interacting with browsing to find relevant photos, while the searchers in the general and subjective searches, often repeatedly used the browsing move after “query” and “search” states. It seemed that the searchers had difficulties in finding targeted photos. The pattern supports Batley’s (1988) finding that use of browsing increased as the specificity of given tasks decreased. General or subjective searches mostly lead to random browsing. Table 7 shows the patterns of sequential dependencies in decreasing probabilities.
Table 7! Probability of Sequential Dependencies in Decreasing Order by Search Tasks Sequential
Dependencies Specific Task General Task Subjective Task
nÆb (0.81) nÆb (0.95) sÆb (0.92)
sÆb (0.78) sÆb (0.83) nÆb (0.88)
SeÆb (0.69) SeÆb (0.68) SeÆb (0.61)
e©Æse (0.47) bÆn (0.46) cÆb (0.56)
First-order
eÆb (0.46) e©Æse (0.42) bÆn (0.48)
eÆnÆb (1.00) bÆnÆb (0.94) seÆnÆb (1.00)
bÆnÆb (0.86) e©ÆnÆb (0.94) bÆnÆb (0.87)
SeÆnÆb (0.8) SeÆnÆb (0.88) e©ÆnÆb (0.86)
e©ÆnÆb (0.75) cÆseÆb (0.7) nÆe©Æse (0.75)
Second-order
bÆseÆb (0.71) e©ÆseÆb (0.66) e©ÆseÆb (0.6)
e©ÆbÆnÆb (0.88) sÆbÆnÆb (0.97) sÆbÆnÆb (0.89) nÆbÆnÆb (0.87) nÆbÆnÆb (0.96) nÆbÆnÆb (0.87) eÆbÆnÆb (0.86) bÆe©ÆnÆb (0.94) eÆbÆnÆb (0.83) seÆbÆnÆb (0.81) e©ÆbÆnÆb (0.93) e©ÆbÆnÆb (0.83) Third-order
Knowing the order of sequential dependencies can help us predict a searcher’s next move based on his or her previous moves. It also reveals how users interact with the system. For example, in the specific image search, the probability of moving on to “caption” from any current stage is extremely low. It might suggest that “caption” is not very helpful for searching very specific images. However, “caption” was a very important move for the general and subjective searches to locate relevant images. The understanding of users’ sequential moves for different types of image searches can help image system designers develop interfaces that will better support users’ search moves.
CONCLUSION
Five common tactics—“browse”, “enlarge/caption”, “enlarge”, “alter”, and “caption” were frequently used by the searchers in this study. Except for the “alter” tactic, four of them are searching tactics. This indicates that when searching for visual information, searchers intend to use searching tactics. This could be due to the querying difficulties. The use of searching tactics can compensate for the querying difficulties. Though there were common tactics used by the searchers across the three searches, there were differences in the use of tactics in each search. For example, the “enlarge” tactic was applied frequently in the specific search, while the “alter” and “caption” tactics were applied more often in the general and subjective searches. It suggests that the specific search was more visually-oriented. The searchers relied more on the visual attributes of photos. Enlarging photos is very useful and helpful in examining photos’ visual attributes and making the relevance decisions. However, in the general and subjective searches, the searchers needed to rely more on textual information to make relevance decisions. The searchers applied the “caption” tactic to read the textual information of the photos to help them understand the context of the photos. The searchers
were not successful in querying. The results suggest that search type affects searchers’ image searching behavior in terms of the search tactics and search moves employed. Thus, image retrieval systems should be designed accordingly to support for different types of image searches. Image retrieval systems should have a responsive and possibly user sensitive thesaurus system displayed to the searcher for navigation and support for searchers’ formulation and reformulation tactics as well.
Browsing was the most heavily used among the tactics across the three searches. Browsing is the key move for image searching. Almost 60% of transitions were browsing across the three searches. The finding is consistent with those found in previous studies (Batley, 1988; Goodrum, et al., 2003; Markkula & Sormunen, 1998; Jorgensen & Jorgensen, 2003, 2005). Browsing is a well-established retrieval strategy, and is particularly useful when searchers have ill defined goals or simply want to find out what is available or as a source of inspiration for future query development (Marchionini, 1995). In this study, through browsing the searchers could easily identify interesting items, especially for searching general and subjective images. It suggests that browsing could be an effective way for searchers to gain intellectual access to images without having an extensive knowledge of terms and concepts in the subject domain of the image collection. Thus, a more powerful interface should be designed that allow users to have more control when using browsing features. For instance, users not only can browse through retrieved images but also can choose various display options such as image sizes, resolutions, relevance display, or hyperlink capabilities to link to similar images. In this study, searchers kept clicking the “next page” button to browse subsequent pages. It is not an efficient use of time to click the button repeatedly. A more controllable interface should be implemented. In addition, searchers heavily used the browsing tactic can be indicative of searchers not being
able to formulate queries, thus there should be a relevance feedback mechanism enabling searchers to refine a query by specifying examples of relevance items while browsing. According to Chang and Rice (1993), browsing varies according to the specificity of the search criteria of goal: specific, ill defined, and undefined. It would be important to study if the purpose and level of browsing are different in the specific, general, and subjective searches in order to implement an interface which can sufficiently support browsing.
This study focuses on the journalism field, thus the findings might not be applied to a different user group or the examination of different types of images. Only 30 photo editors participated in this study, for generalization to a larger population and for more sophisticated analyses, further research with a larger sample size is suggested. In addition, only one database and three text queries were used in this study, it is dangerous to generalize from a small number of instances. Further studies on other databases and more text queries would be recommended.
REFERENCES
Bates, M. (1979a). Information search tactics. Journal
of the American Society for Information Science, 30(4), 205-214.
Batley, S. (1988). Visual information retrieval: Browsing strategies in pictorial databases. Proceedings
of 12th International Online Information Meetings,
373-381.
Chang, S., & Rice, R. (1993). Browsing: A multidimensional framework. In M. Williams (Ed.),
Annual Review of Information Science and Technology, 28 . Medford, NJ: Learned Information.
Chen, H.-M., & Cooper, M. (2002). Stochastic modeling of usage patterns in a Web-based information system. Journal of the American
Society for Information Science and Technology, 53(7), 536-548.
Chen, H., & Dhar, V. (1991). Cognitive process as a basis for intelligent retrieval systems design.
Information Processing and Management, 27(5),
405-432.
Datta, R., Joshi, D., Li, J. & Wang, J. Z. (2008). Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40, 2.
Enser, P. (1995). Pictorial information retrieval.
Journal of Documentation, 51(2), 120-170.
Enser, P., & McGregor, C. (1992). Analysis of visual information retrieval queries. Report on Project
G16412 to the British Library Research and Development Department. London: British Library.
Fidel, R. (1985). Moves in online searching.
Online Review, 9(1), 61-74.
Fidel, R. (1984). Online searching styles: A case study based model of searching behavior. Journal
of the American Society for Information Science, 35,
(4), 211-221.
Frost, C. O. et al. (2000). Browse and search patterns in a digital image database. Information Retrieval,
1, 287-313.
Fukumoto, R. (2006) An analysis of image retrieval for metadata type image database. Image
Processing and Management, 40, 723-728.
Goodrum, A., Bejune, M., & Siochi, A. (2003). A state transition analysis of image search patterns on the Web. Proceedings of Image and Video
Retrieval, Second International Conference, CIVR 2003, 281-290.
Hastings, S. K. (1995). Query categories in a study of intellectual access to digitized art images.
Proceedings of the 58th Annual Meeting of the American Society for Information Science, 32, 3-8.
Hsieh-Yee, I. (1998). Search tactics of Web users in searching for texts, graphic, known items and subjects: A search simulation study. Reference
Librarian, 60, 60-83.
Jorgensen, C., & Jorgensen, P. (2005) . Image querying by image professionals. Journal of the American
56(12), 1346-1359.
Jorgensen, C. , & Jorgensen, P. (2003). Image querying by image professions. Proceedings of the
66th Annual Meeting of the American Society for
Information Science and Technology, 40, 349-356.
Jorgensen, C. (1998). Attributes of images in describing tasks. Information Processing &
Management, 34(2/3), 161-174.
Jorgensen, C. (1996). Indexing images: Testing an image description template. Proceedings of the
59th Annual Meeting of the American Society for Information Science, 33, 209-213.
Keister, L. (1994). User types and queries: Impact on image access systems. In Fidel, R. et al. eds. Challenges in Indexing Electronic Text and Images.
Medford, NJ: Learned Information for the American Society for Information Society, p. 7-22.
Marchionini, G. (1995). Information seeking in
electronic environments. Cambridge University
Press, Cambridge UK.
Markey, K. (1988). Access to iconographical research collections. Library Trends, 37(2), 154-174.
Markkula M., & Sormunen E. (1998). Searching for photos -Journalists’ practices in pictorial IR. The Challenge of Image Retrieval, A Workshop and Symposium on Image Retrieval, Electronic Workshops in Computing, Newcastle upon Tyne, 5.6 February 1998. The British Computer Society. McDonald, S., & Tait, J. (2003). Search strategies
in content-based image retrieval. Proceedings of
the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 80-87.
Nahl, D., & Tenopir, C. (1996). Affective and cognitive searching behavior of novice end-users of a full-text database. Journal of the American Society for
Information Science, 47, 276-286.
Ornager, S. (1997). Image retrieval: Theoretical analysis and empirical user studies on accessing information in images. Proceedings of the 60th
Annual Meeting of the American Society for Information Science, 34, 202-211.
Penniman, W. D. (1975). A stochastic process analysis of on-line user behavior. Proceedings of
the American Society for Information Science, 12,
147-148.
Rasmussen, E. M. (1997). Indexing images. Annual
Review of Information Science and Technology, 32,
169-196.
Shatford, S. (1986). Analyzing the subject of a picture: A theoretical approach. Cataloging &
Classification Quarterly, 6(3), 39-62.
Westman, S., Lustila, A., & Oittinen, R. (2008). Search strategies in multimodal image retrieval.
Proceedings of Information Interaction in Context,
13-20.
Wildemuth, B. (2004). The effects of domain knowledge on search tactic formulation. Journal of
the American Society for Information Science and Technology, 55(3), 246-258.
