In this study, we explored how human factors (including cognitive, affective, and skill factors) influence search behaviors, strategies and
performance in a variety of multimedia search engines. The findings support the ideas that (a) students with different thinking style levels (an affective factor) perform various search strategies and behaviors and present different search performance in text search tasks, (b) successful image searches are strongly dependent on reading ability (a cognitive factor) rather than on Internet experience (a skill factor), (c) the strongest predictor of landmark search performance is environmental cognition (a cognitive factor) followed by spatial ability and geographical knowledge (two cognitive factors), and (d)
metacognitive strategies (a cognitive factor) are important factors influencing video search success and learning effectiveness, but verbal-imagery cognitive style (an affective factor) does not exert a strong influence on video searches.
Therefore, the development of cognitive abilities is very important for students to be able to search the web for information effectively.
This study offered a summary of human factors, different types of search engines, and search behaviors and strategies that can serve as indicators of how students interact with and respond to search engine interfaces. The results indicate that a positive relationship was found between text search performance and the total number of keywords, while a negative relationship was found between video search performance and the total number of keywords. However, no relationship was found between image search performance and keyword-based behavior indicators, and positive relations were found between image search performance and two behavior indicators of
search outcome evaluation. We believe that the differences lie in the distinction between text, image and video searches. Because text search makes it easy for users to make use of related ideas to formulate keywords, the users can broaden or narrow their search scopes by constantly changing the keywords. Moreover, text search has a huge database; therefore, it only takes a few searches to retrieve the correct documents or answers. However, given that images generally have abstract and complex concepts, it is hard for users to translate their visions into literal descriptions in order to formulate keywords. Therefore, the search engines may not always provide the
corresponding images. Under this circumstance, it is vital for users to compare and evaluate the relevance between the images and the task description. This way, users can obtain better search performance if they are willing or
competent enough to view more pages of image search results. As for video searches, videos, like images, are infused with complex concepts. Sometimes it is hard for people to describe the video content that they want with accurate keywords. In addition, not every topic is suitable to be made into a video. So, the database of videos is not as large as the image and text databases. If users fail to use accurate keywords, they will not be able to find relevant videos.
Even though users change keywords frequently, it does not help much in resolving this issue.
The use of computer technologies for problem-solving is rapidly becoming a required daily life skill for students and non-students alike. This
transformation is affecting education in terms of knowledge transfer and
construction because students are increasingly required to take the initiative to seek and construct their own knowledge pools. Accordingly, learning
effectiveness is increasingly impacted by information collection, analysis, assessment, and integration. Visual stimuli, in particular, promote learner interest and attention, support learner efforts to construct new knowledge, and facilitate memory. However, much of the information on websites is uploaded by Internet users, leading to poor quality. To achieve autonomous learning, educators can focus on teaching Internet search and website information assessment skills, as well as on helping students to acquire the knowledge contained in various types of search engines. For example, computer teachers can introduce how web search results can be ranked and remind students that the most useful/correct knowledge and quality information is not necessarily placed at the top. Moreover, teachers should provide different guidance to different students based on the students’ individual differences and encourage them to leverage their characteristics and expertise to find good and readable information on the Internet in order to address all types of tasks and problems in real life.
Even though our four-phase search tasks may appear simple to execute, we observed sharp distinctions between students at various cognitive ability levels. For example, a lack of reading ability affects students’ abilities to search for images effectively. Therefore, how to modify instructional approaches for students with specific characteristics, such as good versus poor reading skills, improve student’s cognitive abilities (including reading, spatial, metacognitive abilities, etc.), and build up their visual literacy are all missions for teachers.
We found that most participants surfed the web following the sequence of search result lists and became bored or frustrated after viewing a small
number of links. Only good readers were capable of selecting satisfactory
materials when facing numerous retrieved outcomes; hence, the ranking and clustering functions of search engines seemingly need improvement. In particular, on meta search engines, the volume of search results is greater than other general search engines. The sequence and classification of search results can correspond to the individual differences of users even more.
Otherwise, users may fail to find the information that they want when facing massive search results. The authors suggest that search engine designers create interfaces or algorithms that (a) present large bodies of search results in ways that are easier for users to comprehend by, for example, classifying search results according to correlation; (b) help users narrow their searches to reduce information complexity according to their individual information needs or cognitive abilities, for example, AI technologies can be used to differentiate individual abilities (e.g., reading, spatial or visual literacy) and Internet usage habits (e.g., viewing result pages and keyword usage), to provide appropriate auxiliaries; and (c) predict users’ search intentions, for instance, after users have chosen their first keywords, instead of forcing them to filter large amounts of search results, search engines could be designed to recommend related information or search results.
Currently, search engines are trending towards hybrid or multimedia search engines. Thus, it is possible for users to input not only keywords but also images, videos and audio in order to search for information by comparing more sets of metadata. The search results are likely to be more diversiform, and the search processes of searchers may be different. Search engine designers need to consider how users construct mental representations when performing multimedia searches, how complex information negatively affects
performance, and how to alleviate users’ working memory loads. For example, search engines can provide more personalized functions in terms of inquiry strategies, filtering techniques, and multiple media indexing or provide some functions that are enabled or disabled by users’ inclinations. However, it remains to be examined whether these new functions support greater search result accuracy or simply impose additional cognitive burdens. Future research can further investigate whether there are other cognitive abilities affecting the search performance of multimedia search engines, leveraging this research as a cornerstone.
This study acknowledges at least one study limitations. The author simply adopted the model driven methodology to explore search behaviors and performance in accordance with individual differences measured through various questionnaires. Future research can further adopt the data driven methodology to investigate individual characteristics in accordance with search behavior patterns collected through artificial intelligence technologies.
Thus, the results of these two methods can be cross-referenced and it can enhance accuracy of this study.
References
Akturk, A. O., & Sahin, I. (2011). Literature review on Metacognition and its measurement. Procedia-Social and Behavioral Sciences, 15, 3731–3736.
Allen, B. L. (1998). Designing information systems for user abilities and tasks: An experimental study. Online & CD-ROM Review, 22(3), 139– 153.
Allen, B.L. (2000). Individual differences and the conundrums of user-centered design: Two experiments. Journal of the American Society for Information Science, 51(6), 508–520.
Anderson, R. C. (1977). The notion of schemata and the educational enterprise. In R.
C. Anderson, R. J. Spiro, & W. E. Montague (Eds.), Schooling and the acquisition of knowledge (pp. 415–431). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ayersman, D. J. (1995). Effects of knowledge representation format and hypermedia instruction on metacognitive accuracy. Computers in Human Behavior, 11(3–4), 533–555.
Best, J. B. (1989). Cognitive psychology. New York: Academic Press.
Bilal, D. (1998, October). Children’s search processer in using World Wide Web search engines: An exploratory study. Paper presented at the Sixty-First ASIS annual meeting, Pittsburgh, PA.
Bilal, D. (2000). Children’s use of the Yahooligans! Web search engine. I. Cognitive, physical, and affective behaviors on fact-based tasks. Journal of the American Society for Information Science, 51(7), 646–665.
Bilal, D. (2001). Children’s use of the Yahooligans! Web search engine. II. Cognitive and physical behaviors on research tasks. Journal of the American Society for
Information Science, 52(2), 118–136.
Bilal, D. & Kirby, J. (2002). Differences and similarities in information seeking on the web: Children and adults as web users. Information Processing and Management, 38(5) , 649–670.
Boutell.Com (2007). How many websites are there? Retrieved July 3, 2012, from the http://www.boutell.com/newfaq/misc/sizeofweb.html
Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: analysis of a complex cognitive skill. Computers in Human Behavior, 21(3), 487–508.
Brown, A. L. (1987). Metacognition executive control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp.65–116). Hillsdale, NJ: Erlbaum Associates.
Butler, D. (2006). The web-wide world. Nature, 439, 776–778.
Byström, K., & Järvelin, K. (1995). Task complexity affects information seeking and use. Information Processing & Management, 31(2), 191–213.
Carroll, J. (1993). Human cognitive abilities: A survey of factor-analytical studies.
NY: Cambridge University Press.
Cheng, C. -C. (2005). Measuring reading difficulties of vocabulary, meanings and sentences. Paper presented at the Sixth Chinese Lexical Semantics Workshop, Xiamen, China.
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152.
Chitu, A. (2008, April 28). Improving Google Image Search using implicit PageRank.
Retrieved January 25, 2010, from
http://googlesystem.blogspot.com/2008/04/improving-google-image-search-using.htm l
Chiu, S. -T. (2006). The effects of metacognition and concept mapping on opened and closed tasks searching outcomes (Unpublished master’s thesis). National Chiao Tung University, Hsinchu, Taiwan.
Choi, Y. (2010). Effects of contextual factors on image searching on the Web.
Journal of the American Society for Information Science, 61(10), 2011–2028.
Clark, J. M., & Paivio, A. (1991). Dual Coding Theory and Education. Educational Psychology Review, 3(3), 149–210.
comScore (May 4, 2006). 694 million people currently use the Internet worldwide according to comScore Networks. Retrieved January 8, 2011, from
http://www.comscore.com/Press_Events/Press_Releases/2006/05/comScore_Launche s_World_Metrix
Cook, M. P. (2006). Visual representations in science education: The influence of prior knowledge and cognitive load theory on instructional design principles. Science Education, 90(6), 1073–1091.
Cunningham, S., & Masoodian, M. (2006). Looking for a picture: An analysis of
everyday image information searching. Paper presented at the 6th ACM/IEEE-CS Joint Conference on Digital Libraries, New York, NY.
Debowski, S.（2002）. Developing effective electronic information seekers. Australian Journal of Management, 27(1), 21–29.
Derry, S. J. (1999). A Fish called peer learning: Searching for common themes. In A.
M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp.
197–211). Mahwah, NJ: Lawrence Erlbaum Associates.
DeVries, R., Zan, B., Hildebrandt, C., Edmiaston, R., & Sales, C. (2002). Developing constructivist early childhood curriculum: Practical principles and activities. NY:
Teachers College Press.
Di Vesta, F. J. (1987). The cognitive movement and education. In J. A. Glover & R. R.
Ronning (Eds.), Historical Foundations of Educational Psychology (pp. 203–233).
NY: Plenum Press.
Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69(2), 145–186.
Downs, R. M., & Stea, D. (1977). Maps in minds: Reflections on cognitive mapping.
NY: Harper & Row.
Duffy, T. M., & Jonassen, D. H. (1992). Constructivism: New implications for
instructional technology. In T. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 1-16). Hillsdale, NJ: Lawrence Erlbaum Associates.
Eliot, J., & Smith, I. M. (1983). An international directory of spatial tests. Windsor, Berkshire: NFER-Nelson.
Evans, W. G. (1980). Environmental cognition. Psychological Bulletin, 88(2), 259–287.
Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering education. Engineering Education, 78(7), 674–681.
Ford, N., Miller, D., & Moss, N. (2005). Web search strategies and human individual differences: Cognitive and demographic factors, Internet attitudes, and approaches.
Journal of the American Society for Information Science and Technology, 56(7), 741–756.
Ford, N., Wood, F., & Walsh, C. (1994). Cognitive styles and searching. Online &
CD-ROM Review, 18(2), 79–86.
Freeman, A. (2001). The eyes have it: Oral miscue and eye movement analysis of the reading of fourth grade Spanish/English bilinguals (Unpublished doctoral
dissertation). University of Arizona, Tucson, Arizona.
Fukumoto, T. (2006). An analysis of image retrieval behavior for metadata type image database. Information Processing & Management. 42(3), 723–728.
Gagne, E. D. (1985). The cognitive psychology of school learning. Boston, MA: Little, Brown.
Golledge, R. G. (1999). Human wayfinding and cognitive maps. In R. G. Golledge (Ed.), Wayfinding behavior: Cognitive mapping and other spatial processes (pp.
1–45). Baltimore: Johns Hopkins University Press.
Golledge, R. G., & Stimson, R. J. (1987). Analytical behavioral geography. NY:
Groom Helm.
Goodman, K. S. (1986). What’s whole in whole language. Portsmouth, NH:
Heinemann.
Google (n.d.). Google inside search. Retrieved January 13, 2013, from
http://support.google.com/websearch/bin/answer.py?hl=en&hlrm=zh-Hant&answer=1 06230&ctx=cb&src=cb&cbid=-ni6lpk9btibq
Hart, R. A., & Moore, G. T. (1973). The development of spatial cognition: A review.
In R. M. Downs & D. Stea (Eds.), Image and environment (pp. 246-288). Chicago:
Aldine Publishing Company.
Hegarty, M., Montello, D. R., Richardson, A. E., Ishikawa, T., & Lovelace, K. (2006).
Spatial abilities at different scales: Individual differences in aptitude-test performance and spatial-layout learning. Intelligence, 34(2), 151–176.
Hölscher, C., & Strube, G.. (2000). Web search behavior of Internet experts and newbies. Computer Networks, 33(1–6), 337–346.*
Hou, S., & Ramani, K. (2004). Dynamic query interface for 3D shape search. Paper presented at the DETC’04 ASME 2004 Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Salt Lake City, UT.
Hoz, R., Bowman, D., & Kozminsky, E. (2001). The differential effects of prior knowledge on learning: A study of two consecutive courses in earth sciences.
Instructional Science, 29(3), 187–211.
Hsieh, P. -Y. (2000). Information search for round up all: When mouse meets robin.
Taichung, Taiwan: National Museum of Natural Science.
Hsieh-Yee, I. (2001). Research on Web search behavior. Library & Information Science Research, 23(2), 167–185.
Huang, C. -Y. (2004). The establishment of a thinking style questionnaire for
elementary students and its creativity related research. (Unpublished master’s thesis).
National Hsinchu University of Education, Hsinchu, Taiwan.
Israel, S. E. (2007) Using metacognitive assessments to create individualized reading instruction. Newark, Delaware (DC): International Reading Association.
Jacobson, T., & Fusani, D. (1992). Computer, system, and subject knowledge in novice searching on a full-text, multifile database. Library and Information Science Research, 14(1), 97–106.*
Johnson-Laird, P. N. (1983). Mental Models: Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.
Jörgensen, C., & Jörgensen, P. (2005). Image querying by image professionals.
Journal of the American Society for Information Science and Technology, 56(12), 1346–1359.
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329-354.
Kim, K.S. (2001). Information-seeking on the Web: Effects of user and task variables.
Library & Information Science Research, 23(3), 233–255.
Kim, K. S., & Allen, B. (2002). Cognitive and task influences on Web searching behavior. Journal of the American Society for Information Science and Technology, 53(2), 109–119.
Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user’s perspective. Journal of the American Society for Information Science, 42(5), 361–371.
Laxman, K. (2010). A conceptual framework mapping the application of information search strategies to well and ill-structured problem solving. Computers & Education, 55(2), 513–526.
Lazonder, A. W., Biemans, H. J. A., & Wopereis, I. G. J. H. (2000). Differences between novice and experienced users in searching information on the World Wide
Web. Journal of the American Society for Information Science, 51(6), 576–581.
Lay, J. G. (1999). A study of map cognition on elementary school and high school students. Journal of Cartography, 10, 49–58.
Leader, W. S. (2008). Metacognition among students identified as gifted or nongifted using the discover assessment (Unpublished doctoral dissertation). Graduate College of the University of Arizona, Tucson, AZ.
Liben, L., Patterson, A., & Newcombe, N. (1981). Spatial representation and behavior across the life span. NY: Academic Press.
Lin, C. -C., & Tsai, C. -C. (2005). Navigation flow map method of representing students’ searching strategies on the Web. Paper presented at 2005 World Conference on Educational Multimedia, Hypermedia & Telecommunications, Montreal, Canada.
Lin, C. -C., & Tsai, C. -C. (2007). A “navigation flow map” method of representing students’ searching behaviors and strategies on the Web, with relations to searching outcomes. CyberPsychology & Behavior, 10(5), 689–695.
Lin, Y. -S. (2002). The study on the metacognition performance of junior high school students in the learning of Biology through integrated technology (Unpublished master’s thesis). National Taiwan Normal University, Taipei, Taiwan.
Linn, M. C., & Petersen, A. C. (1985). Emergence and characterization of gender differences in spatial abilities: A meta-analysis. Child Development, 56(6), 1479–1498.
Liu, C. -F. (2000). An analysis of users’ browsing behavior on World Wide Web (Unpublished master’s thesis). Fu Jen Catholic University, Sinjhuang, Taiwan.
Liu, Y. -L. (2003). An action research of thefifth-grade students in searching and integrating internet information (Unpublished master’s thesis). National Chiayi University, Chiayi, Taiwan.
Lohman, D. F. (1979). Spatial ability: Individual differences in speed and level.
Stanford, CA: Stanford University.
Lu, C. -Y., Gien, M. -F., & Chen, R. -H. (1988). Differential aptitude test. Taipei:
Chinese Behavioral Science Corporation.
Lu, C. -Y., Ou, T. -H., & Lu, C. -M. (1994). Multifactor aptitude test. Taipei: Chinese Behavioral Science Corporation.
Lu, I. -W. (1998). A study of portal site service quality: using search web as an
example (Unpublished master’s thesis). National Taiwan University of Science and Technology, Taipei, Taiwan.
Marchionini, G. (1995). Information seeking in electronic environments. Cambridge, MA: Cambridge University Press.
Matthews, M. H. (1984). Environmental cognition of youth children: Images of journey to school and home area. Transactions of the Institute of British Geographers, 9(1), 89–105.
Matusiak, K. K. (2006). Information seeking behavior in digital image collections: A cognitive approach. The Journal of Academic Librarianship, 32(5), 479–488.
Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions?
Educational Psychologist, 32(1), 1–19.
Mayer, R. E. (2001). Multimedia learning. NY: Cambridge University Press.
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (pp. 31–48). New York:
Cambridge University Press.
Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100(2), 380–386.
Mayer, R. E., & Massa, L. J. (2003).Three facets of visual and verbal learners:
Cognitive ability, cognitive style, and learning preference. Journal of Educational Psychology, 95(4), 833–841.
Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning.
Educational Psychology Review, 14(1), 87–99.
Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words?
Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389–401.
Mayer, R. E., Steinhoff, K., Bower, G., & Mars, R. (1995). A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology Research and Development, 43(1), 31–43.
McGee, M. G. (1979). Human spatial abilities: Psychometric studies and environmental, genetic, hormonal, and neurological influences. Psychological Bulletin, 86(5), 889–918.
Metcalfe, J., & Shimamura, A. P. (1994). Metacognition: Knowing about knowing.
Cambridge, MA: MIT Press.
Meyer, J. W., Butterick, J., Olkin, M., & Zack, G. (1999). GIS in the k-12 curriculum:
A cautionary note. The Professional Geographer, 51(4), 571–578.
Navarro-Prieto, R., Scaife, M., & Rogers, Y. (June 3, 1999). Cognitive strategies in Web searching. Retrieved October 3rd, 2006, from
http://zing.ncsl.nist.gov/hfweb/proceedings/navarro-prieto/index.html
Novak, J. (1990). Concept mapping: A useful tool for science education. Journal of Research in Science Teaching, 27(10), 937–949.
O'Reilly T., & McNamara, D. S. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional "high-stakes" measures of high school students' science achievement. American Educational Research Journal, 44(1), 161–196.
Organisation for Economic Co-operation and Development (2000). Measuring student knowledge and skills: The PISA 2000 assessment of reading, mathematical and scientific literacy. Retrieved January 8, 2011, from
http://www.oecd.org/document/49/0,3343,en_32252351_32236159_33688881_1_1_1 _1,00.html
Ouyang, C. -L. (1981). The application of cognitive map in geography. Education of Geography, 8, 63–70.
Ouyang, C. -L. (1982). The development of children's spatial conception. Taipei, Taiwan: National Taiwan Normal University.
Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart &
Winston.
Paivio, A. (1986). Mental representations: A dual coding approach. Oxford, England:
Oxford University Press.
Palmquist, R. A., & Kim, K. S. (2000). Cognitive style and on-line database search experience as predictors of Web search performance. Journal of the American Society for Information Science, 51(6), 558–566.
Park, Y., & Black, J. B. (2007). Identifying the impact of domain knowledge and cognitive style on web-based information search behavior. Journal of Educational Computing Research, 36(1), 15–37.
Pellegrino, J. W., & Hunt, E. B. (1991). Cognitive models for understanding and
assessing spatial abilities. In H. A. H. Rowe (Ed.), Intelligence: Reconceptualization
assessing spatial abilities. In H. A. H. Rowe (Ed.), Intelligence: Reconceptualization