在知識建構與分享環境中發展概念覺察

全文

(1)國立交通大學資訊工程系博士論文. 在知識建構與分享環境中發展概念覺察 Developing Conceptual Awareness in a Knowledge Construction and Sharing Environment. 研究生：高宜敏指導教授：孫春在. 教授. 林珊如. 教授. 中華民國九十七年七月.

(2) 在知識建構與分享環境中發展概念覺察 Developing Conceptual Awareness in a Knowledge Construction and Sharing Environment. 研究生：高宜敏. Student：Gloria Yi-Ming Kao. 指導教授：孫春在. Advisor：Chuen-Tsai Sun. 林珊如. Co-Advisor：Sunny S. J. Lin. 國立交通大學資訊工程系博士論文 A Dissertation Submitted to Department of Computer Science College of Computer Science National Chiao Tung University in partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Computer Science July 2008 Hsinchu, Taiwan, Republic of China. 中華民國九十七年七月.

(3) 在知識建構與分享環境中發展概念覺察. 學生：高宜敏. 指導教授：孫春在博士林珊如博士. 國立交通大學資訊工程學系﹙研究所﹚博士班. 摘. 要. 本論文擬利用資訊科技發展出輔助使用者進行概念覺察的知識建構與分享環境。從一開始的前導性實驗「資訊分享與搜尋」，我認為透過網路傳遞與分享的這些大量資料需要進一步被有效利用，以避免學習者消極的接收資訊而導致學習效果不佳，因此我接著提出「知識主動建構學習」和「創意思考潛能提升」的後續研究。在這一系列的數位學習系統環境中，運用搜尋引擎、概念圖工具與相關資訊技術，我希望以認知與情意的學習理論為基礎，設計出人機介面支援使用者對學習歷程中不同面向的自我覺察，尤其是希望能幫助使用者發展概念覺察，以深化學習效果或提升使用者的創意思考潛能，我將之稱為「創意知識工程」。以下採分點方式條列出各個研究主題。 1. 個人化資訊分享與搜尋: 目前網路上放置著大量群眾分享的資料，因此本論文的前導性實驗關注如何以「搜尋技術」精確的找到對個人具有參考價值的內容。有別於以資料探勘的方式，分析歸納出使用者的搜尋行為模式，我以社會科學的角度出發，嘗試找出影響網路搜尋行為的重要使用者個人因素。 2. 知識主動建構學習：創意知識工程的第二步提出網路知識分享的五層次策略與系統：單向分享、分享並推送通知、分享並給予回饋、分享並雙向互動、以及分享並主動建構學習。我認為網路學習環境除了提供分享的機制之外，並需能提供使用者主動建構或整合知識的活動，以幫助深化學習。 3. 創意思考潛能提升：創意知識工程的第三步希望能突破舊知識的巢臼，支援使用者對概念侷限或概念之間遠端連結的覺察，並探討學生在系統中因自我覺察所能產生的知識架構之改變，以發展使用者的創意潛能。本研究的實驗對象包括國小學生與大學生。研究結果顯示：1.幫助使用者覺察資訊 i.

(4) 分享與搜尋歷程，以便根據使用者的認知與情意心理特徵，設計出更精準的搜尋人機介面，更有效幫助知識的取得。2.幫助使用者覺察，當使用網路觀看他人作品，或是與他人分享知識或經驗時，需要進行知識主動解構、建構與再累積，才能藉由站在巨人或眾人的肩膀上，有效將知識內化。3.輔助學生藉由觀摩、比較同儕的概念圖，覺察並打破自我概念的侷限，以避免單方面思考所可能造成的盲點，進而產生知識架構之改變與提升創意潛能。. ii.

(5) Developing Conceptual Awareness in a Knowledge Construction and Sharing Environment Student：Gloria Yi-Ming Kao. Advisors：Dr. Chuen-Tsai Sun Dr. Sunny S. J. Lin. Department﹙Institute﹚of Computer Science National Chiao Tung University. ABSTRACT The goal of this dissertation is to develop a series of Internet-based knowledge construction and sharing environments that facilitate user awareness capabilities, especially in terms of conceptual awareness. The dissertation includes a pilot study that focuses on information sharing and search behavior. It is my contention that information shared via the Internet requires further utilization to benefit learners. I therefore designed and executed two studies on active knowledge construction and creative thinking potential enhancement. To construct these human-computer interaction environments I used a combination of information technology and educational theory to assist users in regulating their efforts to benefit from information retrieval or learning processes. In this dissertation I also propose a creative knowledge engineering model to use as a foundation for research. A guiding goal throughout this dissertation is enhancing users’ self-awareness for the purpose of reducing or eliminating the restrictive effects of habitual thinking on learning outcome and/or creative potential. Results from experiments involving freshman undergraduates or elementary school students indicate that the activities are practical for (a) identifying the search intention prediction factor to facilitate information sharing and searches, (b) encouraging active. iii.

(6) knowledge integration via a “beyond sharing” design through which students are motivated to incorporate valuable shared information into cognitive structures and to elaborate on their knowledge for deeper understanding, and (c) improving conceptual awareness so as to break conceptual boundaries and encourage creative potential via the introspective and comparative features of integrated concept maps. It is my hope that future researchers will be able to extend creative knowledge engineering applications for various purposes and to elaborate on underlying theories and design principles to fully understand the benefits of creative knowledge engineering.. iv.

(7) 誌. 謝. 終於抵達博士研究的最後一個階段了。回顧這幾年的研究歷程，固中滋味還真是難以形容。期間當然有許多辛苦的階段需要突破，而且許多困難是原先想像不到的，多謝老師和家人、朋友們能適時提供幫助或為我加油打氣，讓我能秉持認真作研究的精神，一路走下去，在論文完成的剎那，心中真的感到非常的充實與興奮！首先感謝孫春在老師這幾年來的指導，讓我對作研究這件事又有了更深一層的認識。老師針對同一件事情總是能提出許多不同的觀點來深入分析，並佐以許多有趣的實例，您獨特的見解和過人的表達能力(是我最好的典範/真是令人佩服)。感謝林珊如老師提供我許多研究設計與統計分析上的建議，讓我作研究的功夫更加紮實，並時常的鼓勵、關心我，讓我倍感溫暖。再來要謝謝口試委員曾憲雄老師、袁賢銘老師、張國恩老師、楊淑卿老師、許聞廉老師、黃國禎老師對我研究的肯定，並給予我許多寶貴的建議。也謝謝實驗室裡一起作研究的夥伴和學弟妹們，由於你們的相互砥礪，讓我的研究生活不孤單。還有要謝謝國中時期的好姊妹們和以前公司同事們對我的期許，讓我不敢懈怠，尤其是筱玲開朗、熱心的個性，讓我感染到妳的熱情，覺得不管作什麼事都要開心。感謝我的家人能體諒我因為太忙，所以沒辦法常回家。爸爸雖然表面上很嚴肅，但是背後卻一直支持我；媽媽煮的菜最好吃了，真希望可以常回家陪媽媽逛市場、吃媽媽煮的菜；大哥和大嫂一直很關心我的研究狀況，在我出國研修一年的時候，還幫我準備了好多補給品；姊姊會幫我梳妝打扮和講冷笑話給我聽，謝謝你們！最後特別要感謝的是我的男朋友泊寰，不管在任何狀況下，永遠的支持和陪伴我，還會逗我玩、幫我舒壓，是我繼續往前進的最大力量，讓我的人生更有意義、更快樂，我相信跟你在一起一定會很幸福！. v.

(8) Table of Contents Abstract (in Chinese)…………………………………………………………………………...i Abstract (in English)…………………………………………………………………………..iii Acknowledgements ....................................................................................................................v Table of Contents ......................................................................................................................vi List of Tables ...........................................................................................................................viii List of Figures............................................................................................................................ix Chapter 1. Introduction...............................................................................................................1 1.1. Motivation ...................................................................................................................1 1.2. Creative knowledge engineering model ......................................................................2 1.3. Goal .............................................................................................................................4 Chapter 2. Pilot study: Integrating human factors in information sharing and searches............6 2.1. Predicting user intention for narrowing search results ................................................6 2.2. Structured presentation of search results .....................................................................7 2.3. Related works on searches...........................................................................................8 2.3.1. Individual differences in Web searches............................................................8 2.3.2. Thinking style .................................................................................................10 2.4. Study design of information search ...........................................................................11 2.4.1. Participants .....................................................................................................11 2.4.2. Search task......................................................................................................12 2.4.3. Procedure ........................................................................................................13 2.4.4. Data collection and pre-analysis.....................................................................13 2.5. Analysis and results of search ...................................................................................16 2.5.1. Relationship between search target setting and thinking style level ..............16 2.5.2. Differences among the four conditions ..........................................................17 2.6. Discussion of information search ..............................................................................19 2.7. Conclusion of information search .............................................................................20 Chapter 3. Beyond sharing information: Engaging students in cooperative and competitive active learning ..........................................................................................................................22 3.1. Sharing.......................................................................................................................24 3.2. Beyond sharing: Personal integration for active learning .........................................26 3.3. Peer assessment .........................................................................................................29 3.4. The BeyondShare environment .................................................................................29 3.4.1. Primary interfaces...........................................................................................30 3.4.2. Teacher observation........................................................................................32 3.4.3. Evaluating results ...........................................................................................32 3.5. BeyondShare Evaluation ...........................................................................................34 vi.

(9) 3.5.1. Participants .....................................................................................................34 3.5.2. Procedures ......................................................................................................34 3.5.3. Scoring............................................................................................................36 3.5.4. Questionnaire..................................................................................................37 3.6. Results and discussion of beyond sharing .................................................................37 3.7. Conclusion of active knowledge construction...........................................................43 Chapter 4. Breaking concept boundaries to enhance creative potential ...................................47 4.1. Computer-assisted concept mapping system.............................................................49 4.2. Meta-cognition ..........................................................................................................50 4.3. Self-awareness ...........................................................................................................51 4.4. From self-awareness to creative potential .................................................................52 4.5. Study design of breaking concept boundaries ...........................................................53 4.5.1. Concept boundaries ........................................................................................53 4.5.2. Research questions and framework for conceptual self-awareness................54 4.6. The Integrated Concept Map System (ICMSys) .......................................................56 4.6.1. ICMSys interface............................................................................................58 4.7. Case study of improving conceptual self-awareness.................................................59 4.7.1. Participants and materials...............................................................................60 4.7.2. Procedure ........................................................................................................60 4.7.3. Conceptual Self-awareness rating method .....................................................61 4.8. Results and discussion of improving conceptual self-awareness ..............................63 4.8.1. Does the ICMSys promote conceptual self-awareness?.................................63 4.8.2. Does the ICMSys help learners make positive conceptual changes in their revised maps? ...........................................................................................................64 4.8.3. Does ICMap viewing frequency affect conceptual self-awareness level? .....66 4.8.4. Is there a correlation between conceptual self-awareness level in the revised map and conceptual improvements?.........................................................................67 4.8.5. ICMSys questionnaire responses....................................................................67 4.9. Conclusions of breaking concept boundaries ............................................................68 Chapter 5. Conclusion and future works ..................................................................................71 References ................................................................................................................................73. vii.

(10) List of Tables Table 1. Global style percentages of search target-setting patterns. ........................17 Table 2. Local style percentages for search target-setting patterns..........................17 Table 3. Mean rank of each search behavior indicator according to the four thinking style level conditions. .......................................................................................18 Table 4. Statistically significant contrasting pairs of conditions for the three significant search behavior indicators. .............................................................19 Table 5. Beyond sharing activity structure...............................................................28 Table 6. Student perceptions of BeyondShare ease-of-use ......................................38 Table 7. Student perceptions of personal map constructions (first level) ................39 Table 8. Student perceptions of peer assessment and competition (second level) ...41 Table 9. Student perceptions of sharing construction (third level)...........................43 Table 10. Scores on personal and sharing construction............................................43 Table 11. Questionnaire to measure student perceptions of the ICMSys.................61 Table 12. Concept map scoring. ...............................................................................62 Table 13. Statistics for the student, expert, and student/expert conceptual structure scores. ...............................................................................................................64 Table 14. Improvement in conceptual self-awareness in terms of the four criteria. 64 Table 15. Concept map quality as assessed by experts in terms of the four criteria.66 Table 16. Data for Integrated Concept Map (ICMap) viewing frequency. Group 1 = high, Group 2 = low..........................................................................................67. viii.

(11) List of Figures Figure 1. Creative knowledge engineering (CKE) model ..........................................2 Figure 2.Perspectives (including human factors, tools, and task types) that affect Web search strategies. ........................................................................................9 Figure 3. Search target quantification (three indicators). .........................................15 Figure 4. Sharing for shared understanding (item 1-4) and active learning (item 5). ..........................................................................................................................25 Figure 5. Personal construction interface example...................................................33 Figure 6. Sharing construction interface example ....................................................33 Figure 7. Research flow diagram and three effects ..................................................36 Figure 8. Research focus and Integrated Concept Map System architecture. ..........54 Figure 9. Main research framework and questions...................................................55 Figure 10. Two proposition integration patterns. .....................................................58 Figure 11. Integrated Concept Map System user interface and an integrated concept map with student A1’s map highlighted (translated into English for demonstration purposes)...................................................................................59 Figure 12. Features of the 3 studies..........................................................................71. ix.

(12) Chapter 1. Introduction 1.1. Motivation A growing number of networked-based information sharing applications and learning environments have been developed for delivering information and instructional materials to Internet users or students. This age is also marked by a sharp increase in the popularity of search engines, which allow users with little or no training to access a seemingly unlimited amount of information. This presents a new challenge for users and instructors: the information itself may have less value than in the past. I therefore believe that information shared via the Internet requires further utilization to benefit users in terms of learning and creative thinking potential. In particular, Taiwanese company executives are placing greater emphasis on manufacturing and Taiwanese educators on learning, in both cases without giving much attention to developing creative thinking potential. This can lead to negative consequences in an age marked by an overabundance of information. After describing a pilot study addressing human factors that influence information search behavior patterns, I will offer suggestions for search interface design to facilitate information sharing and search efficiency. Next, I will address the issue of making the best use of distributed information to facilitate learning, to assist learners in meaningful knowledge construction, and to enhance creative thinking potential. Educators and many organization managers are acutely aware of the significance of creativity for learning and economic activity. However, creativity involves a complex mix of factors; it is not easy for students to generate creative end products in a short period of time. Therefore, this dissertation mainly serves as an initial step toward achieving greater potential for creative thinking by means of improving conceptual awareness. To assist in this effort, I 1.

(13) have used a combination of information technology (IT) tools and education theory to create a methodological model I refer to as creative knowledge engineering (CKE) (Fig. 1). The purpose of CKE is to develop multiple Internet-based learning environments in which users can benefit from self-awareness via information sharing or learning processes, especially in terms of conceptual awareness. By establishing self-awareness, users can avoid the restrictive effects of habitual thinking, and consequently deepen their learning based on information shared over the Internet and develop creative thinking potential through the breaking of concept boundaries.. 1.2. The creative knowledge engineering model. •of cognitive/social processes •through self-disclosure and peer feedback. Self-awareness. Self regulation. 1. Information Search Incorporating new information into cognitive structures. 2. Knowledge Construction Finding novel relationships by breaking boundaries. 3. Creative Thinking. Figure 1. Creative knowledge engineering (CKE) model. The CKE model consists of three phases: 1. Information sharing and search. This step involves applying search technologies to 2.

(14) locate valuable information to achieve efficient information retrieval. During this process, users must be aware of what they are looking for and the relationship between required information and acquired information in order to avoid getting off-task or having to deal with irrelevant search results. My belief is that thinking style—a distinctly human factor—can be incorporated into search engine interface design to better predict search intentions and to help users comprehend search results. 2. Active knowledge construction instead of passive information sharing. CKE considers information sharing as an intermediate step in a process consisting of active engagement in meaningful learning and knowledge integration. As a result of my literature review and from personal observations concerning popular Web applications, I have created four sharing activity categories: basic sharing, sharing with notification, sharing with feedback, and sharing with interactions. To overcome the tendency to passively absorb delivered information, I have designed a “beyond sharing” approach that emphasizes the integration of cross-unit knowledge in the pursuit of personal goals to generate productive exchanges among students. Students need to be aware of what they acquire in order to grasp the complexity of a problem and to find special meaning from self-experience to accommodate or assimilate new information into their personal cognitive structures. 3. Creative thinking potential. This step emphasizes the idea of using computer technology as an auxiliary tool to externalize multiple viewpoints, facilitate individual awareness of concept boundaries, and enhance creative potential. I believe taking advantage of concept mapping to help students become aware of possible gaps in their existing conceptual structures is an essential step in improving student learning effects and creative potential. Various concepts or leads generated by peers may be used to stimulate creative associations that individuals may not otherwise come up with. 3.

(15) because of their inflexibility in utilizing prior knowledge. In this manner, the restrictive impact of habitual thinking on creative potential can be reduced or eliminated.. 1.3. Research goal This dissertation aims to develop a series of Internet-based knowledge construction and sharing environment that facilitate users’ awareness ability, especially in terms of conceptual awareness. I will begin with a pilot study that focuses on information sharing and search behavior and proceed to two studies on active knowledge construction and enhancing creative thinking potential to explore the power of utilizing distributed information over the Internet. These research activities are designed and conducted to activate or improve self-awareness and self-regulation of user behaviors when (a) searching for and incorporating valuable information into cognitive structures through a process of active knowledge construction, (b) discovering novel relationships by overcoming conceptual boundaries, and (c) identifying and considering creative ideas. Users can repeat the information search and knowledge construction steps in order to grasp the complexity of an assignment by getting glimpses of what others have done to address the same assignment, by finding reference data, and by identifying problems through knowledge re-construction. After users collect sufficient information and learn corresponding knowledge that allows them to fully understand the context of a problem, they can further look for either novel relationships or remote associations between ideas in the acquired knowledge. However, simply possessing knowledge is insufficient for creativity to occur—imagination is also required. I believe self-awareness plays an essential role in bridging the gap between imagination and knowledge. Induced by self disclosure or peer feedback, self-awareness can assist in the generation of creative associations, since people with greater self-awareness can 4.

(16) more easily observe changes in self or environment and to use such observations to make creative changes and adaptations. Aspects of self-awareness could focus on cognitive and social processes. In this dissertation I emphasize conceptual awareness when building Internet-based knowledge sharing and construction environments. The guiding goal is to deepen users’ learning experience and even to remove barriers to creative thinking by giving learners opportunities to observe differences between their own and their peers’ knowledge structures. The participants in the experiments described in this report are from elementary schools and colleges, but future researchers can recruit participants from any age group they desire to replicate these studies, to confirm the results, and to provide more thorough analyses. The knowledge sharing and knowledge construction environments, as well as the beyond sharing and concept boundary-breaking activities presented in this dissertation, can easily be introduced to students of all ages. However, it is important to use learning materials that fit the learners’ comprehensive abilities and needs.. 5.

(17) Chapter 2. Pilot study: Integrating human factors in information sharing and searches As one of the most prevalent applications in today’s network computing environment, Web search engines are widely used for information seeking and knowledge elaboration. However, search-related technology has not yet reached a level of maturity, therefore academic and private researchers continue to look for “the perfect search technology” (Battelle, 2005). Many researchers are experimenting with ways of predicting user search intentions, with some testing new ideas on presenting information visually so as to help users locate information more efficiently. My assertion is that the concept of thinking style—a distinguishing human factor—should be incorporated into any search engine interface design for better search intention prediction and to help users comprehend search results.. 2.1. Predicting user intention for narrowing search results Most search engines use keyword-based techniques as part of their primary interface design. This presents a problem: should users search for what they already know or what they do not know? The answer most likely lies somewhere in between—that is, most searches are for what users “partly” know, since they need prior knowledge of precise keywords in order to find the information they desire. According to Bilal (1998), users without this knowledge frequently choose imprecise keywords and therefore must adjust and re-adjust keywords and filter out large numbers of hits in order to locate information of interest. Even individuals with considerable search engine experience and/or good domain knowledge must deal with this issue. 6.

(18) Many search engine users—especially children and people with little Information Technology (IT) experience—have problems selecting precise keywords. Bilal and Kirby (2002) note that children usually fail to find desired information due to an inclination to use complete sentences, misspelled words, or over-generalized terms. They observe that children have problems formulating adequate or alternative keywords for completing search tasks and usually do not evaluate the quality of search results. In an attempt to help inexperienced users by predicting their intentions to create better search experiences, designers of advanced search engines such as Ask.com and A9.com recommend the use of relative search results for locating targeted or more precise information. For instance, users who type in the query “How do elephants sleep?” to Ask.com will be presented with such questions as “Why is an elephant called an elephant?” and “How do elephants eat?” This relieves users of the task of keying in relative keywords to explore core search topics.. 2.2. Structured presentation of search results Regardless of the internal algorithm employed—e.g., Bharat and Mihaila’s (2001) Hilltop, Brin and Page’s (1998) PageRank, Haveliwala’s (2002) topic-sensitive PageRank, or Kleinberg’s (1998) HITS—search results are sorted using relevance-ranking mechanisms that for the most part do not provide significant or structured presentations to help users quickly comprehend the retrieved information. Thus, users are usually required to sift through long lists of excerpts to create an overall picture of the search topic or to glean the best information. Children find it especially difficult to judge and analyze the correctness and value of search results and rarely evaluate or supplement the ones they receive (Hsieh-Yee, 2001). Categorizing search results is one obvious solution for dealing with information overload. Clustering is one method that allows users to view categorized results without having to deal with the costs and complexities of building taxonomies (see, for example, the Vivisimo search engine). Zamir and Etzioni (1999) made an empirical comparison of standard 7.

(19) ranked-list and clustered presentation systems when designing a search engine interface named Grouper, and reported substantial differences in use patterns between the two. Some researchers who have experimented with highly metaphorical visualizations (e.g. Cugini, Laskowski, & Sebrechts, 2000) present users with structural overviews of result sets and promote visualization as the best approach to dealing with broad search tasks. Visualization structures of this type appear to make it easier for users to locate worthwhile information and to comprehend search results. Based on the hypothesis that thinking style can assist with user interest or intent predictions, my suggestion for search engine designers is to incorporate this human factor into their interfaces to enhance human-computer interaction.. 2.3. Related works on searches 2.3.1. Individual differences in Web searches Web searches involve complex cognitive processes that are strongly affected by individual user characteristics. The literature contains many studies focused on differences in cognitive perspective, especially in the area of prior knowledge (see, for example, Last, O’ Donnell & Kelly, 2001; Rouet, 2003; Shapiro, 2000). Kim and Allen (2002) note that cognitive style and task type directly influence search behaviors, and Yuan (1997) adds that search experiences influence search command decisions. Holscher and Strube (2000) and Lazander, Biemans and Wopereis (2000) are among researchers who have explored differences in information search behaviors associated with different levels of information search expertise, which implies different types or strengths of cognitive factors. According to Bilal and Kirby (2002), a list of such factors should include user comprehension of the search task, individual experience with Web surfing, skill level for manipulating search engines, and the amount of attention an individual gives to a search task. All of the researchers listed in this paragraph have considered how differences in user cognitive or skill perspectives impact 8.

(20) search behavior. Groups of users can still develop search strategies based on shared prior knowledge. Ford, Miller and Moss (2005) report that attitudes toward the Internet and demographic factors can also affect Web search strategies. In an earlier study, Ford and Miller (1996) observed females who were unable to find their way, frequently became lost or lacked a sense of control, and tended to only look at items suggested to them. Ford and Miller also studied how self-efficacy (in this context, indicating an individual’s judgment of his or her personal ability to find information) impacts perceptions of and approaches to information seeking. Besides human factors, researchers such as Bilal (2000, 2001), Kim and Allen (2002), and Last, O’Donnell and Kelly (2001) state that search task type affects student reactions to hypertext.. Figure 2.Perspectives (including human factors, tools, and task types) that affect Web search strategies. The studies cited to this point allow for a summary of human factors that influence search strategies (including cognitive, affective, skill, and demographic) (Fig. 2) and to. 9.

(21) analyze how thinking style levels (an affective human factor) help determine young students’ search strategies—a topic that has not received proper attention in search behavior studies. This dissertation also constitutes an attempt to summarize human, search engine, and search task factors that can serve as indicators of how students interact with and respond to search engine interfaces. Combined, all of these indicators influence search strategies. One current approach to improving the user search experience consists of providing a personalized interface; most search engines use some form of a personal (Google) or social (Yahoo) search history mechanism to achieve this. Data mining-related techniques are used to analyze search histories to recognize search patterns (interests) that reflect human factors. Human factors that can be identified as exerting significant impacts on search behaviors can be used to predict search intentions. As an important human factor that strongly affects daily personal behavior, thinking style has significant potential for impacting information seeking behavior on the Web. Thus, instead of using data mining techniques to explore raw data for recognizing user search patterns, integrating thinking style into search engine interface design may exert a much greater impact on search intention identification.. 2.3.2. Thinking style Thinking style refers to personal preferences in one’s abilities to deal with problems, not the abilities themselves. Accordingly, people with the same abilities may express different behaviors due to the strengths of their preferences (Sternberg, 1988, 1994). Human mental functions can be discussed in terms of five “mental self-government” dimensions: function, form, level, scope, and leaning. The function dimension involves preferences for formulating ideas, carrying out rules initiated by others, or comparing and evaluating ideas. The form dimension concerns various goal-setting and self-management behavioral styles. The level dimension distinguishes between preferences for dealing with problems at relatively abstract 10.

(22) or detailed levels. The scope dimension includes a preference for working alone or with others. The learning dimension addresses a preference for working on tasks that involve novelty and ambiguity or tasks that require adherence to existing rules and procedures (Zhang & Sternberg, 2005). Sternberg and Grigorenko (1995) suggest that individuals look for learning activities that match their preferred thinking style. With the advent of Internet technology, some researchers are focusing on how thinking styles impact Internet-centered learning contexts. However, to the best of my knowledge the literature does not contain any studies on the impacts of thinking style on Internet-based information seeking behavior (frequently referred to as “search behavior”). One of my goals in this dissertation is to determine if a specific thinking style emerges over time when conducting Internet searches in the same manner that it emerges as part of other daily life skills and abilities. Thinking style can affect judgments concerning immediate issues at hand. In the face of different activities that happen concurrently, individuals may initiate different goals or develop different behavioral patterns. Using goal setting as an example, some people tend toward single-mindedness, others carefully set priorities, and still others are motivated by multiple (often competing) goals perceived as having equal importance. During the search process, some individuals are inclined to grasp the “big picture” of a search task while others focus on a few specific concepts to establish a deeper understanding. The former are satisfied with abstract issues and the latter require detail.. 2.4. Study design of information search 2.4.1. Participants Study participants were 355 fifth grade students attending an elementary school in. 11.

(23) central Taiwan. Each student’s thinking style level was determined using a questionnaire to be described in a later section. Of the 350 students who completed the questionnaire, 311 were instructed to use Google to search for information on pollution and to fill out a worksheet. All of the participants had two years’ worth of training in computer usage, meaning that they had basic skills with Windows, Microsoft Word, a Web browser, and Web information search techniques.. 2.4.2. Search task Bilal (2000, 2001) categorizes search tasks as fact-finding or research-based. Fact-finding tasks involve searches for specific answers to simple questions and research-based tasks involve searches for less clear-cut answers to more complex questions. He also notes that different search task types influence children’s cognitive and physical search behaviors. My aim was not to address the impact of various search task types, but to analyze the impact of various strengths of thinking style level on search target settings and search behaviors. Achieving this required the use of a research-based search task to encourage students to perform more extensive searches for the purpose of attaining comprehensive understandings of their personal preferences. The topic chosen for the participating students was “pollution”—something that Taiwanese students are well aware of in their daily lives. They had to establish initial search targets in order to attain desired results. After browsing ordered lists of search results, the students made decisions on refining their targets to move closer to their preferred results. They were asked to write down their “search targets” (i.e., Google search keywords) on their worksheets and to regularly revise their sheets according to their current search target interests. Participants were given 80 minutes to complete the task.. 12.

(24) 2.4.3. Procedure Students were given training on basic search skills using the Google search engine. Specifically, they were asked to type in the keyword “energy resources” as practice to ensure that they knew how to use a computer mouse and keypad to browse for information. Next, the 355 students in the original sample were asked to complete the “level dimension” of the thinking styles questionnaire described in the following section. Of the 350 students who completed the questionnaire, 311 performed searches on the topic of pollution and completed their worksheets. Searches were recorded using the Camtasia Recorder 3.0 screen capture program for further analysis.. 2.4.4. Data collection and pre-analysis 1. Investigation of thinking style level The questionnaire used in this research was adapted from the Sternberg–Wagner Thinking Styles Inventory (Sternberg & Wagner, 1999). A modified version (Huang, 2004) suitable for Taiwanese elementary school students was created to measure the strength of the participants’ style preferences when dealing with relatively large and abstract issues (global) compared to detailed and concrete issues (local). The test consists of 10 items with answers measured along a scale of 1 to 5. According to the test results (N = 311), 72 students constituting the highest 27% of the global group were classified as high global, 66 students constituting the lowest 27% were classified as low global, and the remaining 173 students were classified as medium global. Using the same percentages, the respective numbers of students in the high local, medium local, and low local groups were 65, 184, and 62. Representative data were used due to the complexity of analyzing the search strategies and processes of 311 students. I created four conditions: a) 26 students who were concurrently 13.

(25) in the highest 27% of the global group and lowest 27% of the local group, designated as the high global style (HG) group; b) 32 students who were concurrently in the highest 27% of the local group and lowest 27% of the global group, designated as the high local style (HL) group; c) 6 students who were concurrently in the highest 27% of the global and local groups, designated as the bi-high style (Bi-H) group; and d) 6 students who were concurrently in the lowest 27% of the global and local groups, designated as the bi-low style (Bi-L) group. The remaining 241 students were excluded from the search behavior analysis.. 2. Investigation of student prior knowledge To determine if the students’ prior knowledge of natural science affected the search target setting and search behavior variables, I collected, averaged, and used their grades for introductory natural and social science courses to represent their prior knowledge of the pollution topic. The 87 students in the highest 27% grade group were classified as having high prior knowledge, 81 students in the lowest 27% grade group were classified as having low prior knowledge, and the remaining 143 students were classified as having medium prior knowledge.. 3. Investigation of search target settings with worksheets Students were asked to write down their Google search engine target terms on their personal worksheets and to revise the terms as their search intentions changed. The data were quantified and recorded as number of search targets (T), coverage of search targets (C), and maximum extension of search targets (E). As shown in Figure 3, the six search targets could be divided into the concept categories of “air pollution” and “noise pollution,” resulting in a coverage value of 2. Four of the six search targets focused on air pollution and the other two on noise pollution, so the maximum extension value was 4. To apply the search targets to subsequent analyses, I divided them into three types: focused (C<=2 AND E>2), dispersed (C>2 AND E<=2), and mixed. 14.

(26) Figure 3. Search target quantification (three indicators).. 4. Investigation of search behavior Files containing data on keyboard and mouse operations were reformatted into navigation flow maps (Lin & Tsai, 2005)—graphic displays of relationships among search keywords, visited Web pages, and task questions. The maps and search target settings recorded on the students’ worksheets were used to analyze their information search behaviors according to six factors adapted from Lin and Tsai: a) number of keywords (variation in searched information); b) visited pages (variation in task information sources); c) maximum depth of exploration; d) average depth of Web page adoption (average exploration depth for task completion); e) revisited pages (degree of search navigation recursion); and f) Web pages for refining answers (frequency of refining or improving answer quality).. 15.

(27) 2.5. Analysis and results of search 2.5.1. Relationship between search target setting and thinking style level One goal was to determine if the participants’ prior knowledge affected their search target setting patterns (focused, dispersed, or mixed type). Results from a chi-square test indicate no significant relationship between the two variables (χ2 (2) = 6.568, p = .161 > .05), therefore prior knowledge was excluded from subsequent analyses. Next, I combined the high, medium, and low global styles into a single independent variable and performed a chi-square test to identify relationships with the search target dependent variable (Table 1). The results indicate a significant relationship (χ2 (2) = 25.351, p = .000 < .001). Among the low global style students, only 20.8% dispersed their search targets, 59.7% focused their attention on concept elaboration, and 19.4% showed no preference for either search target setting type. Among the medium global style students, 34.7% dispersed their search targets, 41.6% focused on similar search targets, and 23.7% showed no preference. Among the high global style students, 59.1% dispersed their search targets, 25.8% maintained a steady scope of interest, and 15.2% showed no preference. Results from a separate chi-square test revealed a significant relationship between local style (all levels) and search target setting (χ2 (2) = 14.174, p = .007 < .01) (Table 2). Among low local style students, 52.3% dispersed their search targets, 26.2% maintained a steady scope of interest, and 21.5% showed no preference for either search target setting. Among medium local style students, 35.9% dispersed their search targets, 44.6% focused on similar search targets, and 19.6% showed no preference. For high local style, only 22.6% dispersed their search targets, 53.2% focused on search result elaboration, and 24.2% showed no preference. 16.

(28) Table 1. Global style percentages of search target-setting patterns. Pattern Type. Dispersed Focused Mixed. Style Low Global. Medium Global. High Global. (N=72). (N=173). (N=66). 20.8% 59.7% 19.4%. 34.7% 41.6% 23.7%. 59.1% 25.8% 15.2%. Table 2. Local style percentages for search target-setting patterns. Pattern Type. Dispersed Focused Mixed. Style Low Local. Medium Local. High Local. (N=65). (N=184). (N=62). 52.3% 26.2% 21.5%. 35.9% 44.6% 19.6%. 22.6% 53.2% 24.2%. 2.5.2. Differences among the four conditions The small sample size (indicating that nothing was known about the parameters of the variable of interest in the population) required the use of nonparametric methods for the following analyses. Specifically, Spearman’s r was used to express relationships between two variables. Results from a Spearman’s non-parametric test failed to indicate any clear correlations between prior knowledge of the assigned search task and the six indicators (number of keywords: r = .053; visited pages: r = .060; maximum depth of exploration: r = .181; average depth of Web page adoption: r = -.098; revisited pages: r = -.040; Web pages visited for refining answers: r = -.053). Prior knowledge was therefore excluded from subsequent analyses. Next, the four thinking style level conditions were compared in terms of the mean rank of each search behavior indicator (Table 3). Kruskal-Wallis statistical tests were performed due to the small sample size (HG: N = 26, HL: N = 32, Bi-H: N = 6, Bi-L: N = 6). The results 17.

(29) indicate no significant differences among the conditions in terms of the number of keywords (χ2 (3) = 2.191), number of visited pages (χ2 (3) = 4.173), or number of average depth of Web page adoption (χ2 (3) = 4.375), but significant differences for maximum depth of exploration (χ2 (3) = 13.378, p = .004 < .001), number of revisited pages (χ2 (3) = 8.604, p = .035 < .05), and number of Web pages visited for refining answers (χ2 (3) = 9.254, p = .026 < .05). In addition to identifying states of independence among the significant dependent measures, the Spearman test results indicate a correlation between maximum depth of exploration and Web pages visited for refining answers (rs =.301, p = .011 < .05); however, no correlation was identified between maximum depth of exploration and revisited pages (rs =.226), or between revisited pages and Web pages visited for refining answers (rs =.235).. Table 3. Mean rank of each search behavior indicator according to the four thinking style level conditions. Condition. Number of keywords Visited pages Maximum depth of exploration Average depth of Web page Adoption Revisited Web pages Web pages visited for refining Answers. HG. HL. Bi-H. Bi-L. N=26. N=32. N=6. N=6. 34.40 31.88 27.06. 33.77 38.56 43.70. 46.42 44.67 38.92. 38.58 25.67 24.92. ns ns p = .004. 30.77. 39.91. 38.50. 29.50. ns. 30.19. 38.22. 49.50. 30.00. p = .035. 30.37. 39.53. 44.25. 27.50. p = .026. Significance. When Kruskal-Wallis test results were significant at the 0.05 level, Mann-Whitney U tests were performed to measure contrasts between pairs of conditions. Significant pairs are listed in Table 4. A post hoc contrast of two conditions revealed a significantly higher maximum depth of exploration scores in the HL condition compared to the HG condition (U = -3.348, p < .001), suggesting that HL students tended to conduct more detailed searches in 18.

(30) order to fully understand specific topics. For example, a depth of exploration score of 7 was earned by an HL student who found information on how air pollution was produced and how to prevent it, but an HG student only earned a score of 2 for surveying the broad topic of “water, noise, air, sea, and trash pollution.” A separate post hoc contrast of two conditions revealed a significantly higher number of revisited pages among Bi-H students compared to HG students (U = -2.611, p < .001), indicating that Bi-H students were more likely to re-visit Web pages for purposes of knowledge elaboration than for skimming. One student in the Bi-H group revisited the same page 7 times, but an HG student only revisited the same page once and quickly moved on to other pages. A third post hoc contrast revealed a significantly higher number of HL (U = -2.324, p < .05) and Bi-H (U = -2.412, p < .05) students who visited a larger number of Web pages to refine their answers compared to HG students. I observed that one HL student made three revisions to an answer, while an HG student made only one.. Table 4. Statistically significant contrasting pairs of conditions for the three significant search behavior indicators. Condition Pair. Mean Rank. Significance. Maximum depth of exploration. HG (N=26) HL (N=32). 21.88 35.69. p = .001. Revisited Web pages. HG (N=26) Bi-H (N=6). 14.92 23.33. p = .009. HG (N=26). 25.35. p = .020. HL (N=32) HG (N=26) Bi-H (N=6). 32.88 15.29 21.75. p = .016. Web pages visited for refining answers. 2.6. Discussion of information search The study result confirm that students with different thinking style levels perform 19.

(31) variously in terms of three search behavior indicators: maximum depth of exploration, number of revisited pages, and number of Web pages visited for refining answers. Future researchers may be interested in testing other thinking style dimensions to determine their impacts on important search behavior indicators. In order to create better search experiences by predicting user search intention, it is suggested that search engine designers consider incorporating such human factors into preference settings. For instance, after users have chosen their first keywords, instead of forcing them to filter large amounts of search results, search engines can be designed to recommend related information and/or search results that match the users’ personal thinking style levels. For HL or Bi-H users, more focused and detailed search results can be provided to support in-depth understanding or answer refinement. For HG users, related search results in other categories can be provided to satisfy their curiosity for larger or more abstract issues. For Bi-H users who tend to re-visit Web pages, recent pages in personal search histories should be made accessible as part of a search result presentation (e.g., a nearby cluster or category), thus eliminating the need to redo searches for useful Web pages.. 2.7. Conclusion of information search In addition to providing a review of the current literature on how human factors (cognitive, affective, skill, and demographic) influence search strategies, in this section I examined the topic of thinking style level (an affective factor), which in the past has not received proper attention. No attempt was made to analyze how these human factors influence search strategies, but a summary was offered of human factor, search engine, and search task types that can serve as indicators of how students interact with and respond to search engine interfaces. The results indicate that thinking style level is indeed reflected in information seeking 20.

(32) behavior. HG students are inclined to grasp the overall picture of a search task and HL students tend to investigate and build deeper understandings of specific concepts. Accordingly, HG students are satisfied working on a relatively abstract level and HL students prefer working with details. I therefore suggest that thinking style level influences search target setting and search behavior, and can be used in addition to or apart from data mining techniques to identify user search patterns for predicting search intentions. The data points to a need for search engine designers to create interfaces that a) help users narrow their searches to reduce information complexity according to their individual information needs and thinking style differences, and b) present large bodies of search results in ways that are easier for users to comprehend. Tailoring search engine interfaces to conform to personal information needs will be an important topic for future research.. 21.

(33) Chapter 3. Beyond sharing information: Engaging students in cooperative and competitive active learning The concept of sharing has taken on new importance in a world that has the Internet—a tool that allows for resource access from any place at any time. Examples of Internet-based sharing include personal websites, blogs, discussion forums, and instant messaging; a growing number of applications support sharing using different media (e.g., del.icio.us, Flickr, YouTube). These tools disseminate individual or group beliefs in a manner that binds geographically dispersed individuals with common interests. When applied to group-based pedagogy, the anyplace-anytime characteristic enables a shift from real-time learning to asynchronous distributed learning (Kreijns et al., 2002). The same characteristic enables researchers to create sharing activities that entail concurrent, multi-user interactions (Greenberg & Marwood, 1994; Yang et al., 2004). One example is the use of information technology tools to share musical ideas via exchanges of audio files instead of through verbal discussions of concepts (McCarthy et al., 2005). However, many pedagogical or research projects address the how or what of sharing to benefit collaborative learning without questioning the why or examining the effects of sharing on learning contexts. To reap the benefits of collaboration entailing mutual engagement as opposed to simple cooperation entailing labor divisions (Roschelle & Teasley, 1995), teachers and researchers frequently design tasks that involve information sharing followed by discussion (see, for example, Häkkinen et al., 2003). The interactive structure of computer-supported collaborative learning (CSCL) environments creates additional constraints or freedoms for learners. One of several impediments to a desired social interaction is the tendency to assume that it will automatically occur because the environment. 22.

(34) makes it possible (Baker et al., 1999; Kreijns et al., 2002). Research suggests that few students are willing to participate in CSCL discussion forums without some additional motivation, and that factors such as social loafing (e.g., the “free-rider” and “sucker” effects) can lead to responsibility diffusion (Barron et al., 1992). Consequently, spaces set aside for collaboration or cooperation are often misused for chatting or storage at the expense of the desired goal of collaborative learning through sharing. Such discrepancies may be due to a lack of sufficient structure—for instance, the failure of teachers to completely organize learning tasks. I addressed this issue by viewing sharing as an intermediate step in a process consisting of active engagement in meaningful learning and knowledge integration. Specifically, learning roles are made more active and meaningful as students (a) construct personal concept maps for an assigned learning unit, (b) share personal concept maps across units while critically evaluating their peers’ contributions from other units, and (c) actively integrate concept maps across all units using a meta-plan to create a “patchwork” of knowledge. Process details will be described in a later section. In other words, BeyondShare approach described in this dissertation emphasizes the integration of cross-unit knowledge in pursuit of personal goals to generate productive exchanges among students. Instead of simply expecting students to automatically share resources and negotiate with each other in a CSCL environment, I tried to inject a sense of competition to encourage active learning. As part of this sharing process, I experimented with a cooperative competitive learning (CCL) strategy (Lin et al., 2002) that accommodates both cooperation and competition in a manner that yields greater intrinsic motivation (Johnson et al., 1981; Tauer & Harackiewicz, 2004). My formal evaluation of BeyondShare was designed to answer the following research questions: 1. How many students are able to finish “beyond-sharing activities” (to be described. 23.

(35) in a later section) using BeyondShare? 2. Did students perceive BeyondShare as easy to use? 3. Did the three activities designed for BeyondShare evaluation achieve the goal of promoting active learning? 4. What percentage of students became actively engaged in both personal and sharing construction? 5. Did a larger percentage of students engage in active learning during personal construction or sharing construction activities?. 3.1. Sharing A considerable amount of research effort in this area has focused on building a shared sense of understanding or meaning—that is, finding common ground within groups in collaborative learning settings (e.g., Baker et al., 1999; Mulder et al., 2004). Four categories can be created according to this perspective (items 1-4, Fig. 4; black silhouettes represent students who play active roles): 1.. Basic sharing. Citing or using an idea from a peer is the most basic sharing format. However, most learning situations lack proper motivation for sharing, therefore some self-regulated individuals model or cite works while others do not, even when requested or instructed. Furthermore, those who benefit from sharing usually have no channel for notifying idea originators, who therefore remain unaware of how others use their ideas.. 2.. Sharing with notification. In this variation of basic sharing, cited authors are notified that their ideas are being used. Various technologies (e.g., Really Simple Syndication, or RSS) allow authors to push their latest ideas to subscribers, thus facilitating the timely spread of knowledge.. 24.

(36) 3.. Sharing with feedback. By providing feedback, users help the original authors revise and improve their work. The Computer-Supported Intentional Learning Environment (CSILE) constructed by Scardamalia and Bereiter (1991) is one example of a method designed to promote user feedback.. 4.. Sharing with interactions. Authors can interact via discussion threads—for example, Greenberg & Marwood’s (1994) GROUPKIT (see also Yang et al., 2004). However, participation requires individual motivation.. *Note: Black silhouettes represent students who play active roles.. Figure 4. Sharing for shared understanding (item 1-4) and active learning (item 5). Researchers such as Häkkinen et al. (2003) and Mulder and Swaak (2002) have used qualitative, quantitative, or a combination of the two approaches to assess collaboration 25.

(37) during the sharing process. Completed acts of sharing are followed by quality discussions. Special attention must be paid to the effects of group dynamics on shaping shared meaning (Stahl, 2005), as well as acknowledging that shared contributions cannot be accepted as indicators of shared understanding among all team members (Beers et al., 2005). In other words, it is important to separate the term shared knowledge (Edmonds & Pusch, 2002) from shared understanding or shared meaning. While researchers expect to bring shared understanding into full play in a collaborative learning context, they must note whether the learning activities are structured in a manner that facilitates mutual understanding rather than simple exchanges of information. Today’s Web 2.0 (O'Reilly, 2005) technologies facilitate different applications (e.g., blogs, Wikipedia, del.icio.us, Flickr, YouTube) that support the sharing of various kinds of multimedia content. These applications are popular because users enjoy expressing their own viewpoints by distributing their articles, bookmark collections, photos, or video clips, and readers/viewers enjoy or use the information gathered from the shared works. These applications all have the same key element—providing users with spaces to share their work and /or to find others with similar interests. In other words, to some degree they all fit into one or more sharing typology categories. For example, most bloggers are interested in sharing hyperlinks with others interested in the same domain knowledge, yet bloggers in the same domain may compete to attract more visitors to their web sites and therefore work to maintain a favorable page ranking on a major search engine. This phenomenon suggests that competition is a motivating factor for bloggers to update and improve their articles.. 3.2. Beyond sharing: Personal integration for active learning As Suthers (2005) suggests, the online replication of face-to-face learning is not. 26.

(38) acceptable as a CSCL goal; the same is true for using CSCL to duplicate social interactions over the Internet. Instead, educators should aim at using the unique features of the Internet as a large resource pool, especially its distribution characteristic (Scardamalia & Bereiter, 1991). When designing BeyondShare, I purposefully implemented the sharing construction principle (Resnick, 1996) to encourage students to share and reuse ideas from each other’s constructions. Examples of approaches that require students to reuse or model parts of their peers’ projects to enhance their own personal integration include LEGO MindStorms (Resnick, 2002) and Knowledge Soup (Canas et al., 2001). In addition to shared constructions, I injected a sense of competition into BeyondShare to promote active engagement. As depicted in item 5 of Figure 4, students become active learners for the purpose of integrating personal knowledge. They are encouraged to evaluate their peers’ efforts regarding other learning units, select “personal best-fits,” and incorporate works they define as useful into their final personal products. Understanding of the learning material is strengthened through a process of incorporating ideas from their peers’ personal constructions as well as reflecting on feedback concerning their own constructions. Students compete to have their constructions selected by others as the most useful. As with bloggers, competition is used to motivate students to create, update, and upgrade quality products to share with others, as well as to evaluate their peers’ work in a serious manner. Through this competition, they gain a more comprehensive understating of the learning material. Each student plays several roles and has specific responsibilities throughout an activity. The interchangeability of those roles encourages students to become active learners rather than passive information receivers (Table 5). Details will be described in the Procedure subsection of the BeyondShare evaluation section. The term “beyond sharing” refers to combining the features of structuring and competition to achieve such goals. Many new teachers initially assume that all learning. 27.

(39) (including listening to lectures) is inherently active. But the preponderance of research over the past few decades suggests that students must do more than just listen—they must actively discover and understand facts through reading and discussion, then transform and construct knowledge by writing or engaging in problem solving (Johnson et al., 1998; Moreno & Mayer, 2000). Active involvement means that students must engage in higher-order thinking tasks that entail analysis, synthesis, and evaluation (Turner et al., 1998). BeyondShare promotes active learning by encouraging (a) deep understanding of learning material via concept map construction (what Novak & Gowin [1984] refer to as “meaningful learning”); (b) active reflection on the quality of individual constructions through sharing and peer evaluation; and (c) the active synthesis of dispersed knowledge by integrating self- and peer-produced constructions (Fig. 7).. Table 5. Beyond sharing activity structure Expected Learning Outcome 1. Construct a personal concept map. 2. Compete to be chosen with other students.. 3. Evaluate and compare peers’ concept maps. 4. Construct an integrated concept map.. Task Unit. Within a given unit.. Student Role Interchange Active sharer vs. passive to-be-shared.. Cooperation Goal Personal accountability.. Positive task interdependence via sharing cross-unit concept maps; sense of competition enhance motivation. Cross-unit. Peer assessor Help peers revise vs. receiver of their work; gain peer feedback. information about other units. Based on a Active Based on a given unit given unit to integrator. for interlinking link across concepts across all all units. units. Within unit, Within-unit cross-unit. competitor vs. cross-unit helper.. *Note: See “Primary Interfaces” section.. 28. Learning Format. Meaningful learning: reading, understanding, organization. Social facilitation and modeling.. BeyondShare support* Personal construction interface. Personal construction interface; sharing construction interface.. Active learning: critical evaluation.. Sharing construction interface.. Active learning: integrate personal and peers’ ideas according to a meta-plan.. Sharing construction interface..

(40) 3.3. Peer assessment Peer assessment is a widely used strategy in secondary and post-secondary classrooms for teaching principles in such diverse fields as writing, teaching, business, science, engineering, and medicine (Falchikov, 1995; Freeman, 1995; Rada, 1998; Strachan & Wilcox, 1996). The process requires such cognitive activities as reviewing, summarizing, clarifying, giving feedback, diagnosing errors, and identifying missing knowledge or deviations from an ideal (Van Lehn et al., 1995). Receiving abundant and immediate feedback from peers is strongly correlated with effective learning outcomes (Bangert-Drowns et al., 1991; Crooks, 1988; Kulik & Kulik, 1988). In conventional classroom settings, teacher feedback may be of higher quality but less frequent and immediate than peer assessments (Topping, 1998). In peer assessment scenarios, students have more opportunities to view a larger number of projects, allowing them to gain inspiration from concrete examples instead of relying on models centered on a teacher’s cognitive skills or knowledge structure. Peer assessment projects require more on-task time than conventional teacher assessment settings; arguably this is the most important factor in facilitating learning. Falchikov & Magin (1997), Lin et al. (2002), and Liu et al. (2002) are among researchers who state that reliable and valid peer assessment requires three conditions: (a) students must fully understand and be committed to the purpose of their assessment activities; b) students need to be involved in the process of determining criteria, rating scales, and assessment procedures; and c) students need to receive feedback on peer assessment scores in relation to their own performance as well as to the overall score pattern.. 3.4. The BeyondShare environment I incorporated concept mapping into the BeyondShare environment as an activity 29.

(41) based on the assertions of Novak and Gowin (1984), Roth and Roychoudhury (1992), and others that concept maps are effective tools for knowledge construction. Instead of requiring students to participate in group discussions to create collaborative maps (a process that can lead to unequal contributions), I applied the CCL strategy (Lin, Sun , & Kao, 2002) as a more effective approach to evaluating, synthesizing, and incorporating ideas from maps created by their peers. In implementing this strategy, the learning material must be divided into several units (in this study, three units). As part of the BeyondShare process, final concept map products reflect individual and shared construction efforts that fulfill the requirements of independence and interaction (Katz, 2002). In classrooms that have access to state-of-the-art learning technologies, teachers can use concept map approaches that focus on synchronous (real-time) cooperative behavior (Komis et al., 2002). Although these systems have clear advantages, I purposefully designed BeyondShare with the characteristic of asynchronous distributed learning based on the belief that it is available in a larger percentage of classrooms.. 3.4.1. Primary interfaces I used a combination of Microsoft Visual Basic 6.0 and SQL Server7 to design two BeyondShare interfaces: 1.. A personal construction interface that provides a form-based environment. This interface is disabled when students proceed to the sharing construction phase, thereby preventing students from modifying their own concept maps based on the work of others in the same learning unit (Fig. 5). After reading personal assignments for a given learning unit, students begin the personal construction activity in the concept mapping section by pressing the start button (which triggers a time log) and using the construction forms to build and connect 30.

(42) self-defined concept nodes with links. A concept map in progress is shown in the current personal concept map section. Concept nodes and linking words are not fixed, giving students greater flexibility for knowledge construction. They use the current personal concept map area to observe and change node positions to revise concept hierarchies. Nodes and linking words can be removed from the storage section once they become irrelevant to the concept map. Students move back and forth between procedures to construct their maps as they see fit.. 2.. A form-based sharing construction interface consists of interlinks among different concept maps. Interlinks differ from links, which connect ideas within individual concept maps. In Figure 6, the bold arrows with dashed lines indicate interlinked connections between two concept maps. Students can use this interface to view their own completed maps in the personal concept map section. In the modeling section, a system of anonymous selector IDs prevents students from purposefully choosing concept maps made by their friends as their favorites. After choosing selector IDs from the other units, students can study maps in their peer concept map sections, then press the start button to begin the sharing construction process. Students can establish interlinks between their own and their chosen maps in the interlinking section and make comments in the feedback section according to a set of reference criteria. As in the personal construction interface, students can delete interlinks displayed in the storage section. The interlinking process consists of selecting single concept nodes from two maps and adding a linking word. Students can establish as many interlinks as they want between concept map nodes.. During the sharing construction phase, students evaluate all peer concept maps in 31.

(43) other units, select “personal best-fit” concept maps, and establish interlinks between their own and selected maps. Interlinks can be established between near concept nodes or nodes in remote categories. Links in the latter category are known as “cross-links,” implying associations between concepts that many people would not recognize (Novak & Gowin, 1984). In BeyondShare, such links are considered signs of creativity. Choices for establishing interlinks represent cooperative partner selection—the result of a peer assessment evaluation process that encourages critical thinking. Sharing and incorporating information across units with cooperative partners are both encouraged; within units, competition is encouraged.. 3.4.2. Teacher observation BeyondShare contains a teacher interface for monitoring student progress, meaning that students who fall behind the learning schedule can be given special attention. The monitor interface presents a student’s personal concept map, information on the student’s chosen favorites, the number of interlinks between two maps, how much time a student spends on constructing interlinks, and how many other students choose the same map as their favorite. The interface also allows teachers to view information on how many choose the target student’s concept map as their favorite, their personal concept maps, and respective interlinks. All preference data can be logged for peer rating analysis.. 3.4.3. Evaluating results After the sharing construction phase is completed, concept maps are arranged in decreasing order of score (number of votes) for each learning unit. The map receiving the most votes within one unit earns the designation of “best-fit.” Reflective thinking is triggered via comparisons of personal maps with best-fit maps. Furthermore, teachers can construct 32.

(44) their own “expert” concept maps for comparison with best-fit maps for two purposes: determining which knowledge structures are acknowledged by the greatest number of students, and helping students make adjustments to incomplete or incorrect concept maps.. Figure 5. Personal construction interface example. Figure 6. Sharing construction interface example. 33.