運用電腦動畫增進問卷設計、技能學習與教師培育：三個在科學教育情境下的研究

全文

(1)Department of Earth Sciences National Taiwan Normal University. Master‟s Thesis. Leveraging on Animations to Improve Questionnaire Design, Skill Learning, and Teacher Preparation: Three Studies in Science Educational Settings. Yu-Ta Chien Supervisor: Dr. Chun-Yen Chang. January 2011.

(2)

(3)

(4) Acknowledgements. I would like to thank the people without whom nothing would have been possible. First, I much appreciate Prof. Chun-Yen Chang‟s enlightenments and ongoing support to my research works. I do thank my research partners, Chau-Fu Yang, Chia-Li Debra Chen, Chung-Yen Lin, Hsiu-Lan Kuo, Ming-Chao Lin, Shu-Wen Sophia Wang, Terrence Wang, Ting-Kuang Yeh, Yueh-Hsia Chang, and Yung-Tsai Lin, for their great assistance in conducting my research and paper writing. I also thank my dear friends, Chen-Han Chou, Liang-Wei Chang, and Yan Li, for brainstorming research ideas with me all the time. I would like to thank Prof. Ming-Puu Chen of the Graduate Institute of Information & Computer Education, National Taiwan Normal University and Prof. Tzu-Hua Wang of the Education Department, National Hsinchu University of Education for giving me invaluable comments on research works. Finally and most importantly, I deeply thank my father, my mother, and my brother for their infinite support, assurance, and toleration. Thank my lovely family for believing in me even if I did not know where I stood..

(5) Abstract. This thesis explored the educational uses of computerized animations in three science educational settings, including science educational questionnaire design, science process skill learning, and science teacher preparation. In Chapter II, based on dual coding theory, the feasibility of using an animation-based questionnaire to survey college students‟ perceptions of a future science learning environment was explored. The findings revealed that using animations to visualize the key concepts of survey questions had great potential to bound students‟ visual images stimulated from question descriptions, and therefore it could reduce the probability that students misinterpret survey questions. In Chapter III, from the perspective of cognitive load theory, the comparative instructional efficiency among one graphic-based and two animation-based tutorials for assisting high school students in learning a topographic measuring skill was investigated. The results indicated that the degree of user-control in animations would influence students‟ cognitive load and achievements in multimedia learning environments. The additional supporting strategies for improving educational animation design were discussed. In Chapter IV, a framework of instructional design anchoring on cognitive apprenticeship model was proposed to facilitate science pre-service teachers in producing animation-based coursewares. This framework was implemented to reform a science teacher education course and evaluated using both quantitative and qualitative approaches. The results indicated that this framework significantly promoted the pre-service teachers‟ technology competencies and enhanced their confidence in implementing animation-based science instruction. Moreover, it can hone pre-service teachers‟ reasoning on the interplays between technology, pedagogy, and content. Potential additions for incorporating this framework into science teacher education courses were recommended. The preliminary findings reported in this thesis may contribute to a deeper and broader understanding of how and why the uses of computerized animations would benefit the practice in science education.. I.

(6) Table of Contents. Chapter I. Overview ...................................................................................................... 1 Chapter II. Exploring the Impact of Animation-Based Questionnaire on Conducting a Web-Based Educational Survey and its Association with Vividness of Respondents‟ Visual Images ....................................................................... 6 II.1. Introduction ........................................................................................................ 6 II.2. Methods ............................................................................................................. 10 II.2.1. Participants.................................................................................................. 10 II.2.2. Measurements ............................................................................................. 10 II.2.2.1. TBQ ..................................................................................................... 10 II.2.2.2. ABQ ..................................................................................................... 11 II.2.2.3. Vividness of visual imagery scale ........................................................ 12 II.2.2.4. Attitude toward animation questionnaire inventory............................. 13 II.2.3. Procedure and data analysis ........................................................................ 14 II.3. Results ................................................................................................................ 16 II.3.1. Difference in students‟ responses to TBQ and ABQ................................... 16 II.3.2. Vividness of visual imagery in determining the response change between TBQ and ABQ ............................................................................................ 17 II.3.3. Students‟ perceived effectiveness of ABQ .................................................. 18 II.4. Discussion and Implication ................................................................................ 19 Chapter III. Comparison of Instructional Efficiency of Different Multimedia Forms for Improving Students Topographic Measuring Skill Learning ................ 22 III.1. Introduction ...................................................................................................... 22 III.1.1. Cognitive architecture and cognitive load ................................................. 23 II.

(7) III.1.2. Impediment to animation-based learning .................................................. 24 III.1.3. Potential aid in animation-based learning ................................................. 25 III.2. Purpose of the Study ......................................................................................... 27 III.3. Methods ............................................................................................................ 28 III.3.1. Learning subject ........................................................................................ 28 III.3.2. Instructional conditions ............................................................................. 28 III.3.3. Participants and research design ................................................................ 30 III.3.4. Measuring instruments .............................................................................. 31 III.3.4.1. Subjective mental effort scale............................................................. 31 III.3.4.2. Practical performance test .................................................................. 32 III.3.4.3. Instructional time-span ....................................................................... 32 III.3.4.4. Instructional efficiency ....................................................................... 33 III.3.5. Data analysis .............................................................................................. 34 III.4. Results .............................................................................................................. 35 III.4.1. Difference in subjective mental effort ratings ........................................... 36 III.4.2. Difference in practical performance scores ............................................... 36 III.4.3. Difference in instructional efficiency ........................................................ 36 III.4.4. No difference in instructional time-spans.................................................. 37 III.5. Discussion and Implication .............................................................................. 39 Chapter IV. Engaging Pre-Service Science Teachers to Act as Active Designers of Online Animation-Based Coursewares: A MAGDAIRE Framework ........ 42 IV.1. Introduction ....................................................................................................... 42 IV.1.1. Framework for innovating science teacher education courses .................. 45 IV.2. Purpose of the Study ......................................................................................... 50 IV.3. Methods ............................................................................................................ 51 IV.3.1. The use of multimedia and information technologies ............................... 51 III.

(8) IV.3.2. Context of the study ................................................................................... 51 IV.3.3. Data collection and analysis ...................................................................... 54 IV.3.3.1. Quantitative approach ......................................................................... 54 IV.3.3.2. Qualitative approach ........................................................................... 55 IV.4. Findings and Discussion ................................................................................... 56 IV.4.1. Advancing technology competence for science teaching .......................... 56 IV.4.2. Reconsidering interplays between technology, pedagogy, and content ..... 57 IV.4.2.1. Select appropriate components to be transformed with technology ... 58 IV.4.2.2. Use technology beyond the fun factor ................................................ 59 IV.4.2.3. Present information as a web of interconnections .............................. 59 IV.4.2.4. Provide activities for students to interact with computers .................. 62 IV.4.2.5. Negotiate technology-integrated pedagogy with actual classroom settings ................................................................................................. 63 IV.5. Conclusion and Recommendations ................................................................... 65 Chapter V. A Final Word ................................................................................................ 68 Bibliography .................................................................................................................. 71. IV.

(9) List of Tables. Table 2.1. Item Descriptions of TBQ ............................................................................ 11 Table 2.2. Comparison of TBQ Score of TBQ-ABQ Group and ABQ Score of ABQ-TBQ Group.......................................................................................... 16 Table 2.3. Comparison of TBQ and ABQ Scores within TBQ-ABQ and ABQ-TBQ Groups ........................................................................................................... 17 Table 2.4. Descriptive Statistics of Students‟ VVIS and RC Scores of TBQ-ABQ Group ............................................................................................................ 18 Table 2.5. Linear Regression Model Testing the Relationship between VVIS and RC scores of TBQ-ABQ Group .......................................................................... 18 Table 3.1. Comparison of Mathematics and Science Achievement Scores between Groups ........................................................................................................... 31 Table 3.2. Marking Scheme for the Practical Performance Test ................................... 32 Table 3.3. Comparison of Mental Effort Ratings, Practical Performance Scores, Instructional Time-spans, Instructional Efficiency for Groups ..................... 33 Table 4.1. Course Content ............................................................................................. 53 Table 4.2. Comparisons between Pre-Service Teachers‟ Pre-Test and Post-Test Scores ....................................................................................................................... 57 Table 4.3. Taxonomy and Descriptive Statistics for Pre-Service Teachers‟ Perceptions of What They Gained from the Course ......................................................... 57. V.

(10) List of Figures. Figure 1.1. Framework of this thesis.. ........................................................................... 5 Figure 2.1. Screenshots of item no. 2 of ABQ............................................................... 13 Figure 2.2. The research design. .................................................................................... 14 Figure 3.1. Basic tutorial for specifics of Abney Level. ................................................ 28 Figure 3.2. Three forms of multimedia packages. (a) SG; (b) SLPA; (c) FLPA. .......... 30 Figure 3.3. Relative instructional efficiency representation for three instructional conditions.................................................................................................... 38 Figure 4.1. Framework of MAGDAIRE. ...................................................................... 47 Figure 4.2. (a) Pre-service teacher presenting and introducing his ABC to peers in the classroom; (b) Online platform for pre-service teachers. ........................... 54 Figure 4.3. Screenshots of pre-service teachers‟ ABCs. ................................................ 56. VI.

(11) Chapter I Overview. The introduction of multimedia, namely the integration of multiple media such as text, image, audio, video, and spatial model into a computer system (von Wodtke, 1993), has a tremendous impact on the contemporary science education. According to a newly national report about teachers‟ use of educational technology (National Center for Education Statistics, 2010), 69% of the American public elementary and secondary school teachers had often used computers to develop and present multimedia presentations of science subject matter in 2009. As argued by Mayer (2003), the multimedia message is composed of multimode representations, commonly including both verbal (e.g., text or narration) and visual (e.g., illustration or photo) representations, so that it can stimulate more than one human sense at a time. The diverse stimuli can not only leverage the full capacity of human cognitive resources to process information, but also complement one another when an individual is able to mentally integrate verbal and visual representations into a coherent model (Mayer, 2003; Mayer & Moreno, 2002). With the rapid development of graphic and information technologies, computerized animation has become one of the chief ingredients of emerging multimedia presentations (Ainsworth & VanLabeke, 2004; Bétrancourt, 2005; Ploetzner & Lowe, 2004). According to Baek and Layne (1988), animation is broadly defined as a “series of frames containing an object or objects so that each frame appears as an alteration of the previous frame” (p. 132). Lowe (2004) further distinguishes three types of sequential frames depicted in animations as follows: (1) translation that involves the movement of the object(s) from one location to another, such as relative movements of 1.

(12) the sun, Earth, and moon; (2) transformation that involves alterations in size, shape, color, texture, or such properties of the object(s), such as the rifting and break-up of a continent; (3) transition that involves shifts in appearance and disappearance of the object(s), such as the decay of a radioactive isotope. These characteristics of animations encompass the nature of real-world phenomena. In contrast to the conventional visual stimuli in multimedia presentations (i.e., static graphics), animations are regarded as the more powerful representations for viewers to perceive dynamic changes of phenomena much as they would in the physical world (Ainsworth & VanLabeke, 2004; Ploetzner & Lowe, 2004; Wu, Chang, Chen, Yeh, & Liu, 2010). Moreover, animations provide a means to enable the concreteness of abstract phenomena which are too small, large, fast, or slow to be observed or experienced directly with the unaided human senses (Cook, 2006; Wu, et al., 2010). The current educational uses of animations rely on two main underlying potential for facilitating individuals in processing multimedia information. In term of representing a dynamic system or a phenomenon, an animation can directly offer individuals external visualizations of the continuous changes on every micro step and timing, which intrinsically have to be mentally imagined and inferred from a series of static graphics. It makes the dynamics explicit thus may enable individuals to be devoted to comprehending the meaning of the display material, rather than being diverted to generating and running internal representations from the conventional static graphics (Ainsworth & VanLabeke, 2004; Bétrancourt, 2005; Ploetzner & Lowe, 2004; Schnotz & Lowe, 2003; Wu, et al., 2010). From the affective perspective, the dynamic changes in and cosmetic appeal of an animation can attract individuals‟ attention. An animation, therefore, may motivate and sustain individuals to attend to the display material (Kim, Yoon, Whang, Tversky, & Morrison, 2007). However, the effect of animations on the outcome of the human cognition process 2.

(13) in multimedia environments is by no means a straightforward matter. According to a literature review by Tversky, Morrison, and Bétrancourt (2002), the studies related to multimedia comparisons showed that animations per se, in contrast to the static graphics, were quite ineffectual in facilitating individuals to understand information conveyed from multimedia presentations. Based on rigorous experimental settings, Mayer, Hegarty, Mayer, and Campbell (2005) found that the use of animations even hindered individuals from comprehending the subject matter presented in multimedia, and suggested that the static graphics synchronized with corresponding text would still be the best form to deliver multimedia messages. Nevertheless, a recent meta-analysis of 64 pair-wise multimedia comparisons conducted by Höffler & Leutner (2007) identified that instructional animations had an overall small-to-medium-sized advantage over static graphics in terms of fostering better learning achievements. These figures imply whether the uses of animations result in positive effects on information processing of individuals in multimedia environments is far from conclusive. In spite of these conflicting findings, the convergence has been recognized that the design of animations often leans on educational practitioners‟ intuition; unfortunately, it is even merely driven by entertaining purposes (Chandler, 2004, 2009; Höffler & Leutner, 2007; Mayer, 2005; Tversky, et al., 2002). As a matter of fact, the explosion in the use of animations is much in advance of the research-based accounts of how humans can best process multimedia messages (Chandler, 2004, 2009; Lowe, 2004; Mayer, 2005). The growing area of multimedia learning as well as e-learning needs more empirical research to verify the advantage or disadvantage of the application of animations across a variety of contexts (Mayer, 2008; Mayer, et al., 2005; Ploetzner & Lowe, 2004; Tversky, et al., 2002). Moreover, the research into the design and implementation of animations should carefully consult the theoretical mechanisms of human cognition and generate pragmatic guidelines of 3.

(14) multimedia presentations to inform educational practitioners (Chandler, 2004, 2009; Mayer, 2005, 2008; Moreno, 2006). Aligning with such research agenda, this thesis is compiled from three research reports to investigate into the educational uses of computerized animations from theoretical aspects to practical aspects. As shown in Figure 1.1, two experiments described in Chapter II and III were conducted to explore the feasibility of using animations to develop assessment and instruction in science educational settings, respectively. The experimental results were discussed based on the theoretical architecture of human cognition and generated principles for improving animation design. In Chapter II, based on Paivio‟s dual coding theory (1986), the feasibility of using an animation-based questionnaire to survey college students‟ perceptions of a future science learning environment was explored. The practical implication of the use of animation-based questionnaires was discussed. In Chapter III, from the perspective of cognitive load theory (Sweller & Chandler, 1994), the comparative. instructional. efficiency. among. one. graphic-based. and. two. animation-based tutorials for assisting high school students in learning a topographic measuring skill was investigated. Additional supporting strategies to improve animation design were discussed. Chapter IV proposed an instructional design model anchoring on Collin‟s cognitive apprenticeship (1988) to facilitate pre-service science teachers in learning technology integration through developing animation-based instruction and assessment. It aimed to foster pre-service science teachers to adopt the theoretical principles for instructional animation design (including the findings reported in Chapter II and III) into their filed practices. This model was implemented to reform a science teacher education course in Taiwan, and its effectiveness was evaluated using both quantitative and qualitative approaches. Potential additions for incorporating this framework into science teacher education courses were recommended.. 4.

(15) Figure 1.1. Framework of this thesis.. All in all, it is hoped that this thesis would contribute to a deeper and broader understanding of how and why the applications of computerized animations would benefit the practice in science education.. 5.

(16) Chapter II Exploring the Impact of Animation-Based Questionnaire on Conducting a Web-Based Educational Survey and its Association with Vividness of Respondents’ Visual Images1. II.1. Introduction. With the globalization of the Internet, it has become commonplace to complete survey questionnaires through a web browser. In contrast to traditional paper-and-pencil deliveries, administrating a web-based questionnaire is cheaper to reach participants who are not in the same geographical locations, it has a shorter turn-around time, and it is easier to transcribe data for coding and analysis (Hardre, Crowson, Xie, & Ly, 2007). Moreover, it has been suggested that the web-based questionnaire is psychometrically equivalent to the parallel paper-based version (Riva, Teruzzi, & Anolli, 2003; Yu & Yu, 2007). However, just as paper-based surveys, the data collection of web-based questionnaires suffers from the heterogeneity within respondents‟ interpretations of question meaning. For instance, consider a straightforward question from the Tobacco Use Supplement of the Current Population Survey, “Have you smoked at least 100 cigarettes in your entire life?” (U.S. Census Bureau for the Bureau of Labor Statistics, 2008). When answering this question, some respondents would include only tobacco cigarettes, but others would also include cloves, marijuana, and cigars (Suessbrick, Schober, & Conrad, 2000). It shows that different respondents interpret the question‟s. 1. The pilot study of this chapter has been published as: Chien, Y. T., & Chang, C. Y. (2010). Exploring the feasibility of an online contextualized animation-based questionnaire for educational survey. British Journal of Educational Technology, 41(5), E104-E109. 6.

(17) meaning quite differently. Such phenomenon exists widely across web-based surveys and is recognized as a serious source of measurement error (Conrad, Couper, Tourangeau, & Peytchev, 2006; Conrad, Schober, & Coiner, 2007; Graesser, Cai, Louwerse, & Daniel, 2006). From the perspective of cognitive psychology, the cognitive process that occurs when a respondent is asked a survey question is generally modeled into four sequential stages, including (1) question interpretation, (2) memory retrieval, (3) judgment, and (4) response selection (Tourangeau, Rips, & Rasinski, 2000). As a starting point for question answering, the respondent must encoded the presented question into an mental representation that serves as a signal for driving memory retrieval and decision making (Tourangeau et al., 2000; Willis, Royston, & Bercini, 1991). If the respondent‟s mental representation of the question does not match those of the questioners, the misinterpretation of the question meaning will probably occur. As a consequence, the respondents‟ answers to that question will virtually become invalid (Conrad et al., 2007; Graesser et al., 2006; Willis et al., 1991). How to reduce the probability that respondents misinterpret survey questions is one of the important issues in questionnaire design (Conrad et al., 2007; Graesser et al., 2006; Presser et al., 2004; Willis et al., 1991). Despite a variety of methods proposed in the literature, few techniques leverage on the Internet and focus on web-based questionnaires (Presser et al., 2004). A typical strategy for improving web-based questionnaire design is to embed hyperlinked definitions for the key concepts of questions to clarify question meaning for respondents (Conrad et al., 2006; Conrad et al., 2007). It is a convenient and low-cost way to align respondents‟ interpretations of question wording with the questioners. However, it is not guaranteed that all respondents will retrieve the definitions when they need (Conrad et al., 2007; Lind, Schober, & Conrad, 2001). Furthermore, these definitions are often so long that lower 7.

(18) respondents‟ willingness to read them thoroughly and diminish respondents‟ likelihood of completing the survey (Lind et al., 2001). Obviously, the aforementioned technique aims to provide respondents with verbal aid while they are trying to comprehend survey questions. However, according to Paivio‟s dual coding theory (1986), the human cognition generally processes linguistic stimuli, i.e., written and spoken language, through dual routes, including not merely the verbal but also the imagery systems. It is proposed that when an individual reads (or listens) to a word, phrase, or sentence, these linguistic stimuli will be recoded (or encoded) into verbal representations and probably then activate the corresponding visual representations, i.e., visual images, in the individual‟s brain (Paivio, 1986; Sadoski & Paivio, 2007). Empirical studies have pointed out that there exists a wide distribution in vividness of visual images that different individuals generate from the same text (Cui, Jeter, Yang, Montague, & Eagleman, 2007). It implies that the mental representation that a respondent forms from the text description of a survey question consists of both verbal representation and visual imagery. The individual difference in vividness of visual images that are stimulated from a survey question may influence the signals in respondents‟ cognition processes for driving memory retrieval and decision making for question answering. Therefore, the variance in respondents‟ visual images is considered as a potential source leading to question misinterpretations in this study. The technique of web-based animation provides some insight into questionnaire design. Research in the field of computerized assessment has suggested that animations can function as vividly visual aids to enable the concreteness of abstract or ambiguous concepts described in a question, and it therefore improve respondents‟ understanding of the question‟s intent (Dancy & Beichner, 2006; Wu, Chang, Chen, Yeh, & Liu, 2010; Wu, Yeh, & Chang, 2010). Moreover, the cosmetic appeal of animation-based questions can enhance respondents‟ motivation to complete the assessment (Wu, Chang et al., 8.

(19) 2010; Wu, Yeh et al., 2010). Diverging from the verbal approach, this preliminary study employed animations as the vividly visual representations to assist college students in comprehending the survey questions. A set of animations were created to visualize the key concepts of survey questions. It was anticipated that the students could directly perceive the external visual images of question meaning from animations, rather than generate internal visual images by themselves, and therefore the individual difference in vividness of visual imagery would be controlled. The relationship between the vividness of students‟ visual images stimulated from text descriptions of the survey questions and the students‟ response changes between the Text-Based Questionnaire (TBQ) and the Animation-Based Questionnaire (ABQ) was tested. The research questions guiding the investigation were: RQ1: Is there any difference in students‟ responses to TBQ and ABQ? RQ2: What is the relationship, if any, between the vividness of students‟ visual images and their response changes? RQ3: Do students think ABQ is helpful to make survey questions more comprehensible?. 9.

(20) II.2. Methods. II.2.1. Participants A total of 112 college students from National Taiwan Normal University in north Taiwan participated in this study during the month of June, 2010. The students‟ mean age was 19. The gender mix was more or less equally distributed (57 males and 55 females). The students were randomly assigned to taking two questionnaire formats in TBQ-ABQ or ABQ-TBQ orders. Therefore, the sample size of TBQ-ABQ group was 56, and the sample size ABQ-TBQ group was 56.. II.2.2. Measurements II.2.2.1. TBQ TBQ was developed as an instrument to investigate college students‟ preferences of the Future Innovative Science Learning Environment (FISLE). FISLE was proposed by the center for excellence in e-learning sciences in Taiwan (Chang & Lee, 2010) to integrate image processing, speech recognition, mobile communication, and other such modern technologies into a physical classroom setting. Before the deployment of FISLE, students‟ preferences of FISLE were surveyed along with this study. The question descriptions of TBQ were designed according to the project content of FISLE (c.f. Chang & Lee, 2010). TBQ consisted of 8 items, and each item described how one of technologies proposed in FISLE would be used in a classroom setting. TBQ was scored on the 5-ponit Likert scale by choosing 1 (strongly disagree) through to 5 (strongly agree) and reviewed by a panel of specialists including four science education researchers. A pilot test of TBQ was conducted in a national university in Taiwan, June 2009 (Chien & Chang, 2010), and the revised items TBQ are shown in Table 2.1.. 10.

(21) Table 2.1 Item Descriptions of TBQ I hope the technologies described below will be employed in FISLE: No. 1 Teachers are able to know each student‟s name and classroom learning situations through the facial characters identified/captured on the screen at the lecture podium. No. 2 When the teacher mentions some key words/terms, the system will automatically recognize his/her voice to access and display teaching materials. No. 3 Different teaching materials or student group works are able to be displayed simultaneously with double/multiple screen projection. No. 4 The classroom is surrounded with 3D projection technologies to create simulative and immersive virtual reality for teaching and learning. No. 5 Through mobile devices, students are able to send instant messages to teacher at the lecture podium to raise or answer questions. No. 6 Students are able to write down answers to teacher‟s questions or group work results on the electronic tablet device to share with the whole class instantly on the electronic whiteboard. No. 7 With automatic assessment tools with quizzes and answers, students are able to have self-evaluation on their science achievement in the classroom. No. 8 Students are able to browse, download, and retrieve data from recorded class sessions, including notes and illustrations that the teacher presents in class. Note. TBQ, text-based questionnaire; FISLE, future innovative science learning environment. II.2.2.2. ABQ Considering the technologies described in TBQ may be unfamiliar to students, eight animations were developed to visualize the key concepts of the survey questions that were early presented in Table 2.1. These eight scenarios constituted the framework of ABQ. ABQ not only depicted technological instruments, but also vividly visualized the situations in which innovative technologies would be deployed in FILSE. For instance, Figure 2.1 presents a series of screenshots of an animation that visualized the question description of item no. 2. Figure 2.1(a) shows that, in class, a student mentioned that she went to visit Pong-Hu Island over the summer; in Figure 2.1(b), the teacher asked the student if she could see the columnar basalt in Pong-Hu? Then, the system in the classroom automatically recognized the teacher‟s voice and captured the 11.

(22) key word „basalt‟; in Figure 2.1(c), the system automatically accessed the teaching materials related to basalt in the database, and displayed that information to the teacher; in Figure 2.1(d), the teacher used the teaching material provided by the system to start the lecture about basalts. Moreover, the design of animations were grounded on the cognitive load theory and took into consideration the duration and capacity limitations of a human‟s working memory (Sweller & Chandler, 1994). The basic point of cognitive load theory emphasizes that the „free space‟ of an individual‟s working memory can facilitate his/her information processing (Sweller, van Merriënboer, & Paas, 1998). Therefore, to prevent the students‟ visual working memory from being overloaded while viewing ABQ, the annotations in ABQ were presented in an auditory way rather in visual ways (Mayer & Moreno, 1998). All item descriptions of ABQ were transformed into the animations, instead of text. The items of ABQ were presented in the identical response format (5-Likert scale) and sequence (no. 1 to 8) of TBQ. In addition, the same panel of specialists who reviewed TBQ confirmed the consistency between ABQ and TBQ.. II.2.2.3. Vividness of visual imagery scale For assessing the vividness of students‟ visual images stimulated from the text descriptions of survey questions, this study developed a self-reported Vividness of Visual Imagery Scale (VVIS) adapted from Marks‟ (1995) instrument. The VVIS asked students to imagine the scenes described in TBQ and then to rate the vividness of their visual images on the 7-ponit Likert scale by choosing 1 (no image at all) through 7 (perfectly clear). The following question is a sample item from VVIS: Think of the scenes of the future science learning environment described below. Consider the picture that comes before your mind‟s eye: 1. Teachers are able to know each student‟s name and classroom learning 12.

(23) situations through the facial characters identified/captured on the screen at the lecture podium.. Figure 2.1. Screenshots of item no. 2 of ABQ. II.2.2.4. Attitude toward animation questionnaire inventory This study also examined students‟ attitudes toward ABQ as a new questionnaire format. An 8-question inventory, namely Attitude toward Animation Questionnaire Inventory (AAQI), was developed to evaluate the usefulness students perceived for each animation-based item. The complete items of AAQI corresponded to the descriptions of TBQ. The perceived usefulness of each animation-based item was rated on a dichotomous format, known as yes or no. The following question is a sample item from AAQI: The animation made the survey question more comprehensible: 13.

(24) 1. Teachers are able to know each student‟s name and classroom learning situations through the facial characters identified/captured on the screen at the lecture podium.. II.2.3. Procedure and data analysis The research design is illustrated in Figure 2.2. All students were required to rate the vividness of their visual images on VVIS. Thereafter, half of the students were randomly assigned to take the two questionnaire formats in the TBQ-ABQ order, and the others took questionnaires in the ABQ-TBQ order. The time interval between the administration of TBQ and ABQ for both two groups was 10 minutes. Finally, the students evaluated the usefulness of each animation-based item through AAQI. Each student completed the whole procedures by using a PC, a LCD, and a pair of earphones. The data was collected online through an Apache HTTP server with PHP 5.0.. Figure 2.2. The research design. VVIS, vividness of visual imagery scale; TBQ, text-based questionnaire; ABQ, animation-based questionnaire; AAQI, attitude toward animation questionnaire inventory. In order to answer RQ1, this study used two types of t-test statistical analysis to 14.

(25) detect if significant difference exists between students‟ responses to TBQ and ABQ between and within TBQ-ABQ and ABQ-TBQ groups. A two-tailed independent t-test was firstly conducted on the mean TBQ scores of TBQ-ABQ group and the mean ABQ scores of ABQ-TBQ group to compare the students‟ responses to TBQ and ABQ between groups. Then, two-tailed paired t-tests were conducted on TBQ and ABQ scores for TBQ-ABQ and ABQ-TBQ groups, respectively, to detect whether there exists any statistical significantly response change within each group. For answering RQ2, this study derived each student‟s Response Change (RC) score by computing the absolute interval of mean TBQ and mean ABQ scores within groups. Each RC score was thus calculated by the following equation: RC ＝｜mean TBQ score － mean ABQ score︱ To further investigate the relationship between the vividness of students‟ visual images and the students‟ response changes in the different questionnaire formats (TBQ and ABQ) within groups, this study used the mean VVIS score as the predictor in the linear regression model to predict the RC score. Finally, the students‟ mean scores of AAQI for both TBQ-ABQ and ABQ-TBQ groups were calculated to answer RQ3. The effect sizes for t-test and linear regression methods were described as Cohen‟s d and f 2, respectively. According to Cohen‟s rough characterization (1988), d = 0.2 is deemed as a small effect size, d = 0.5 a medium effect size, and d = 0.8 as the large effect size; the f 2 effect sizes of 0.02, 0.15, and 0.35 are termed as small, medium, and large, respectively. All statistical tests were conducted at the alpha = .05 significance level by using Statistical Package for Social Sciences (SPSS) version 15.0.. 15.

(26) II.3. Results. II.3.1. Difference in students’ responses to TBQ and ABQ The reliability coefficients of TBQ and ABQ scores of both TBQ-ABQ and ABQ-TBQ groups were estimated as Cronbach's alpha (α). The reliabilities of the data collected from these two questionnaire formats were relatively high (all α > .8). A two-tailed independent t-test was conducted on the mean TBQ scores of TBQ-ABQ group and the mean ABQ scores of ABQ-TBQ group to detect if significant difference exists between the students‟ responses to the two questionnaire formats (TBQ and ABQ) between groups. A shown in Table 2.2, the result revealed that there was no statistical significantly difference in the students‟ responses to TBQ and ABQ (t = 0.642, p = .522). This study went on 2-tailed paired t-tests on TBQ and ABQ scores of both TBQ-ABQ and ABQ-TBQ groups to detect whether there exists any statistical significantly response change within these two groups. As shown in Table 2.3, there was a statistical significantly difference between students‟ responses to TBQ and ABQ within the TBQ-ABQ group (t = -2.410, p = .019, d = 0.33, small effect size); as for the ABQ-TBQ group, no statistical significantly difference was found in students‟ responses to TBQ and ABQ (t = -1.683, p = .098). Table 2.2 Comparison of TBQ Score of TBQ-ABQ Group and ABQ Score of ABQ-TBQ Group TBQ-ABQ group. ABQ-TBQ group. TBQ score. ABQ score. 2-tailed independent t-test. n. M. SD. n. M. SD. t(110). p. d. 56. 3.87. 0.71. 56. 3.78. 0.86. 0.642. .522. 0.11. Note. TBQ, text-based questionnaire; ABQ, animation-based questionnaire.. 16.

(27) Table 2.3 Comparison of TBQ and ABQ Scores within TBQ-ABQ and ABQ-TBQ Groups TBQ-ABQ group TBQ score. ABQ score. 2-tailed paired t-test. n. M. SD. n. M. SD. t(55). p. d. 56. 3.87. 0.71. 56. 4.04. 0.66. -2.410. .019*. 0.33. ABQ-TBQ group ABQ score. TBQ score. 2-tailed paired t-test. n. M. SD. n. M. SD. t(55). p. d. 56. 3.78. 0.86. 56. 3.87. 0.82. -1.683. .098. 0.22. Note. TBQ, text-based questionnaire; ABQ, animation-based questionnaire. * p < .05. II.3.2. Vividness of visual imagery in determining the response change between TBQ and ABQ Considering there was a significant difference between TBQ and ABQ scores within the TBQ-ABQ group, a simple linear regression analysis was performed with RC as the dependent variable and with VVIS as the independent variable in order to further examine whether the vividness of students‟ visual images would induce the response change in the different questionnaire formats (TBQ and ABQ) for the TBQ-ABQ group. The descriptive statistics of TBQ-ABQ group‟s mean VVIS and mean RS scores are summarized in Table 2.4. As shown in Table 2.5, the regression analysis revealed that VVIS was a significant predictor in explaining RC (t = 2.950, p = .005, R2 = 0.133, f. 2. = 0.15, medium effect size). It suggested that the clearer the. visual imagery stimulated from a survey question description, the more likely the student changed his/her response more prominently to that question on ABQ after they took TBQ. It should be noted that the vividness of students‟ visual images could only predict the absolute interval between TBQ and ABQ responses. The positive or negative direction of the shift of TBQ and ABQ responses could not be revealed in the regression model. 17.

(28) Table 2.4 Descriptive Statistics of Students’ VVIS and RC Scores of TBQ-ABQ Group n VVIS RC. M 5.06 0.67. 56 56. SD 0.87 0.45. Note. VVIS, vividness of visual imagery scale; RC, response change. Table 2.5 Linear Regression Model Testing the Relationship between VVIS and RC scores of TBQ-ABQ Group Dependent Variable. Predicting Variable. RC. VVIS Constant. B 0.193 -0.307. SE 0.065 0.336. β 0.373. t 2.950 -0.914. p .005** .365. Note. RC, response change; VVIS, vividness of visual imagery scale. R2 = .139, adjust R2 = .123. ** p < .01.. II.3.3. Students’ perceived effectiveness of ABQ Most of the students reported that ABQ helped them understand the meanings of survey questions in a clearer manner than TBQ (on average, 99.3% of TBQ-ABQ group and 95.8% of ABQ-TBQ group, respectively). Overall, students held very positive attitudes toward the ABQ being employed as a new questionnaire format.. 18.

(29) II.4. Discussion and Implication. In this preliminary study, a set of animations were integrated into a questionnaire as a new technology-enhanced instrument for conducting an educational survey. The animations were designed to vividly visualize the key concepts of survey questions. Students could directly perceive the external visual images of question meaning from animations, rather than generate internal visual images by themselves. It anticipated that the individual differences in vividness of visual imagery would be controlled and therefore the probability that students misinterpret survey questions would be reduced. A set of comparisons between the students‟ responses on the innovative questionnaire format (ABQ) and on the traditional one (TBQ) was carried out between and within the TBQ-ABQ and ABQ-TBQ groups. The results of statistic analysis on the mean TBQ score of TBQ-ABQ group and the mean ABQ score of ABQ-TBQ group indicated that there was no significant between the students‟ responses to TBQ and ABQ (t = 0.642, p = .522) when the students took TBQ and ABQ independently. However, when the students took TBQ first and then ABQ, there was a statistical significantly difference between students‟ responses to TBQ and ABQ within the TBQ-ABQ group (t = -2.410, p = .019, d = 0.33, small effect size). It implied that the students‟ initial visual images of survey questions stimulated from the text descriptions in TBQ were quite different to the external images they later perceived from ABQ. The incongruence of the visual images, between the students and questionnaire designers, indeed would influence students‟ responses to the questionnaire topic. These findings substantiated the argument stated by previous researches (Sadoski, Goetz, & Fritz, 1993; Sadoski, Goetz, & Rodriguez, 2000) that the individual‟s visual images evoked by reading materials have powerful effects on his/her comprehension for text. In particular for the current study, the students‟ visual images influenced their responses to survey questions. 19.

(30) On the other side, when the students took ABQ first and then TBQ, there was no statistical significantly difference was found in students‟ responses to TBQ and ABQ (t = -1.683, p = .098). In other words, students virtually did not change their responses to the survey questions on TBQ after they took ABQ. It implied that ABQ bounded the students‟ interpretations of the survey questions on TBQ. The plausible reason is that ABQ presented the identical external images to visualize the key concepts of the survey questions for all students. This technique equalized the vividness of students‟ visual images that were stimulated from question descriptions for students to ponder over their responses to survey questions. Moreover, ABQ was design according to the questionnaire designers‟ visual images toward survey questions, and therefore functioned as the aid in dispelling the incongruence between students‟ and questionnaire designers‟ visual images toward the same question descriptions. It suggests that ABQ can reduce the probability that students misinterpret survey questions thus improve data validity collected from web-based surveys. Most of the students actually agreed that the design of ABQ helped them comprehend the survey questions in a clearer manner than TBQ (99.3% of TBQ-ABQ group and 95.8% of ABQ-TBQ group, respectively). The relationship between the vividness of students‟ visual images and students‟ response changes between TBQ and ABQ was further examined. The result of the regression analysis revealed that the vividness of students‟ visual images was a significant predictor in explaining the students‟ response changes between ABQ and TBQ (t = 2.950, p = .005, R2 = 0.133, f 2 = 0.15, medium effect size) when these two questionnaire formats were administrated in the TBQ-ABQ order. It suggested that the clearer the students‟ visual images stimulated from the description of a survey question in TBQ, the more likely the students changed their responses more prominently to that question on ABQ. This finding further confirmed that the students interpreted a survey 20.

(31) question not only based the verbal representations they formed from the question descriptions but also visual images. It should be noted that the „clearer‟ visual images which the students derived from the text descriptions were often different from the external images that were later depicted in ABQ. The questionnaire design should more cautiously take this individual difference into account. In conclusion, the variance in the vividness of students‟ visual images that were stimulated from survey question descriptions was detected in this study, and it was found that this variance would influence the students‟ responses to survey questions. The results suggest that the use of ABQ has great potential to control this intervening variable of survey question answering by equalizing each student‟s visual images. The findings of the present study may shape insights for educators and researches to employ the animation-based items as an alternative method in educational questionnaire design. It should be noted that the results and interpretations are limited by the effect size found in the experiments. The difference between students‟ responses to TBQ and ABQ attained only a small effect size. It suggests that the results should be generalized more cautiously in a practical sense and further replication studies are needed. Future research is also needed to explore what type of questionnaire topics presented in ABQ is more suitable in terms of conducting an educational survey.. 21.

(32) Chapter III Comparison of Instructional Efficiency of Different Multimedia Forms for Improving Students Topographic Measuring Skill Learning2,3. III.1. Introduction. The use of computerized animations is widely regarded as a promising instructional strategy for science education. It is commonly assumed that the frame-by-frame animation is a more powerful vehicle, than static graphics, for depicting dynamic phenomena of real-world context (Ainsworth & VanLabeke, 2004; Chandler, 2004; Mayer, & Moreno, 2002; Tversky, Morrison, & Bétrancourt, 2002). However, even though the multimedia comparison between animated and static visualizations has been intensively investigated for the past decade, little evidence supports that the animation is superior to static graphics in terms of facilitating better learning outcomes (e.g., Chandler, 2009; Mayer, Hegarty, Mayer, & Campbell, 2005; Ploetzner & Lowe, 2004; Tversky, Morrison, & Bétrancourt, 2002). The controversy over the educational multimedia comparison has recognized that the effects of multimedia on students‟ learning outcomes should also consider the instructional design method used to develop or implement the media, rather than merely the delivery media per se (Clark, 2001; Mayer, 2003). The focus of research related to multimedia learning. 2. Portion of this chapter has been published as: Chien, Y. T., & Chang, C. Y. (2010). Minimizing the extraneous cognitive load in learning: Integrating interactive functions into instructional animation. Journal of CyberTherapy and Rehabilitation, 3(2), 194-196. 3 The abstract of this chapter has been accepted by the Annual Meeting of the National Association for Research in Science Teaching (NARST) 2011 conference and will be presented at Orlando, FL, USA, 2011. 22.

(33) therefore should put more emphasis on how to enhance the instructional design of animation in terms of promoting students‟ understanding (Mayer & Moreno, 2002, 2003). The cognitive load theory (Sweller, & Chandler, 1994; Sweller, van Merriënboer, & Paas, 1998) has been widely adopted as the theoretical underpining to inspect the nature of animations and improve animation design.. III.1.1. Cognitive architecture and cognitive load In general, theories of the human cognitive architecture premise that the structure of memory can be distinguished into either long-term memory or short-term memory (e.g., Chandler, 2004; Sweller, & Chandler, 1994; Sweller, van Merriënboer, & Paas, 1998). Long-term memory in essence can hold an unlimited amount of information and has a vast capacity, whereas short-term memory is severely limited in both capacity and duration. According to Sweller and Chandler‟s notion (1994), an individual is only conscious of the information currently being held and processed in short-term memory. Therefore, the term short-term memory is often replaced by working memory for accentuating its characteristic in information processing. Working memory can generally store about seven (plus or minus two) units of information at a time, but it operates on just two to four units simultaneously (Sweller, & Chandler, 1994; Sweller, van Merriënboer, & Paas, 1998). The load that occupies an individual‟s working memory while he/she is processing information is defined as cognitive load. The basic point of cognitive load theory emphasizes the free space of an individual‟s limited working memory so that he/she can easily process the learning material associated with schema (Chandler, 2004; Sweller, van Merriënboer, & Paas, 1998; van Merriënboer & Sweller, 2005). The load imposed on working memory which results from the complexity of the learning subject matter is defined as intrinsic cognitive load. Intrinsic cognitive load reflects the nature of learning material that must 23.

(34) be processed in learning, and it cannot be altered by instructional interventions (Sweller, & Chandler, 1994). The unnecessary load which results from inappropriate formats used to present the learning material is defined as extraneous cognitive load. Consequently, the quality of instructional design of multimedia determines the amount of extraneous cognitive load which imposes on an individual in learning (Chandler, 2004; Sweller, van Merriënboer, & Paas, 1998; van Merriënboer & Sweller, 2005).. III.1.2. Impediment to animation-based learning For meaningful multimedia learning to occur, an individual has to derive and select relevant words and images from the instructional materials, then organize them mentally into coherent verbal and visual representations, and finally build referential connections between the visual and verbal representations (Mayer & Anderson, 1992; Mayer & Moreno, 2002; Moreno, 2006). In terms of mentally organizing representations for corresponding verbal and visual information in multimedia, an individual is forced to temporarily hold previously presented material in working memory, and then connect it with incoming material by his/her inference (Mayer et al., 2005; Mayer & Moreno, 2003). This productive process is what Mayer and Moreno (2003) call representational holding. As an individual perceives instructional multimedia, representational holding often consumes a large portion of an individual‟s cognitive resources. The visualization depicted in an animation dynamically changes over time. These animations may pour a large amount of information to an individual in a very short period of time. Furthermore, the general design of animation hardly provides an opportunity for an individual to inspect and re-inspect the fleeting information of visualizations (Tversky, Morrison, & Bétrancourt, 2002). Therefore, the presentation of animation often imposes a highly extraneous cognitive load on an individual in the 24.

(35) process of representational holding (Evans & Gibbons, 2007; Mayer & Chandler, 2001; Mayer & Moreno, 2003). Consequently, organizing mental representations of animation tends to overwhelm an individual‟s working memory, thus decreasing the instructional efficiency of animation-based learning (Evans & Gibbons, 2007; Mayer & Chandler, 2001; Mayer & Moreno, 2003).. III.1.3. Potential aid in animation-based learning As argued by Tversky et al. (2002), the level of learner-control in animations would be the key to overcoming the drawbacks of animation-based learning. It is suggested that incorporating the simple learner-pacing function (i.e., stop and play buttons) into an animation allows individuals to break off and resume the fleeting information of visualizations; this function would ease individuals‟ cognitive load on representational holding (Mayer & Chandler, 2001). A few studies have confirmed that animation embedded with simple learner-pacing functions foster better learning outcomes than the animation without any learner-pacing mode (e.g., Evans & Gibbons, 2007; Mayer & Chandler, 2001; Mayer & Moreno, 2003). However, the simple learner-pacing function, in essence, can only ease individuals‟ solicitude about the transition in animations (i.e., the shifts in appearance and disappearance of display objects; cf. Lowe, 2004). It lacks for the flexibility to allow individuals to control the speed of translation and transformation (i.e., the position and form changes of the display objects; cf. Lowe, 2004) on micro steps. The translation and transformation in an animation are still paced by the computer system so that may hinder the process of representational holding. Therefore, it is anticipated that if the learner-pacing function is promoted to allow students to directly manipulate parts and wholes of the presented objects at will, then students can be fully in control of the process of representational holding at their own 25.

(36) paces in animation-based learning. By this means, namely the full learner-pacing function, students can perceive external dynamic representations in an animation with less amount of extraneous cognitive load than the condition of learning with the simple leaner-pacing function. However, it is noted that hardly any investigations have been carried out to compare any type of learner-pacing animations with equivalent static graphics. Whether the animation embedded with learner-pacing functions fosters better learning outcomes than static graphics remains unknown.. 26.

(37) III.2. Purpose of the Study. In this study, a set of computer-based multimedia were employed to assist students in learning a basic science-process skill in the geographic domain, i.e., topographic measuring. Three types of visualizations were designed, including the Static Graphics (SG), Simple Learner-Pacing Animation (SLPA), and Full Learner-Pacing Animation (FLPA). In addition to being able to stop or resume the animation, the students could physically manipulate the virtual measuring mechanism in FLPA, rather than passively observe dynamic or static images. It was anticipated that this strategy would allow students to fully control representational holding at their own paces; this process would thus require less mental effort and study time from students to construct visual representations. Therefore, it was assumed that FLPA would facilitate better learning outcomes than both SLPA and SG. The intention of this study was to investigate instructional efficiency and instructional-processing time of FLPA, SLPA, and SG in the architecture of cognitive load theory. Instructional efficiency in multimedia comparison is defined as the combined measure of learners‟ cognitive load and performance levels (Sweller, van Merriënboer, & Paas, 1998; Tuovinen, & Paas, 2004). The research hypotheses prior to the investigation were: H1: FLPA imposes less cognitive load on students than both SLPA and SG. H2: FLPA facilitates better learning outcomes than both SLPA and SG. H3: FLPA has higher instructional efficiency than both SLPA and SG. H4: FLPA requires less study time than both SLPA and SG.. 27.

(38) III.3. Methods. III.3.1. Learning subject The subject matter focused on how to make use of the Abney Level to carry out trigonometric leveling. The Abney Level is a hand level mechanism, as shown in Figure 3.1, used for determining elevations and angles of slope in topographic measurements. Once students attain the vertical angle and either the horizontal or the slope distance between two points by using an Abney Level, they can apply the fundamentals of trigonometry to calculate the difference in elevation between the points. The subject of instruction was new to all students.. Figure 3.1. Basic tutorial for specifics of Abney Level.. III.3.2. Instructional conditions Three versions of computer-based multimedia packages were developed by using Adobe Flash CS3 as well as Action Script 3.0. These packages had the same basic tutorial, as shown in Figure 3.1, to explain the functions of the major parts of Abney Level. They were only different in the formats of visualizations for demonstrating the application of trigonometric leveling. The three instructional conditions included (1) 28.

(39) Static Graphics (SG): static graphics were presented next to the corresponding texts; (2) Simple Learner-Pacing Animation (SLPA): continuously dynamic illustrations were synchronized with explanations. Besides, the simple learner-pacing function (i.e., pause, continue, and backward buttons) were provided for students; (3) Full Learner-Pacing Animation (FLPA): interactive dynamic illustrations were synchronized with explanations. In addition to being able to pause or resume the animation, the students could physically manipulate a virtual Abney Level by pressing direction-keys on the keyboard. Thus, the students could be in control of the spatial relations such as the angles, heights, and distances, depicted in the animation. As shown in Figure 3.2, all multimedia forms provided equivalent content. In all of the three instructional conditions, the students could view and review the visualizations and texts again and again. Information on how to use the learner-pacing functions was presented to students before the initiation of each type of visualizations.. 29.

(40) (a). (b). (c). Figure 3.2. Three forms of multimedia packages. (a) SG; (b) SLPA; (c) FLPA.. III.3.3. Participants and research design Twenty-seven tenth-grade female students from a public high school participated in this study. A randomized post-test comparison-group experimental design was adopted (Campbell & Stanley, 1966). The students were randomly assigned to three 30.

(41) experimental groups and then took the instructions on their own PCs. After the students finished their tutoring lessons, they were asked to estimate the mental efforts encountered in learning and perform trigonometric leveling with a real Abney Level. To ensure the equivalent of groups, the three groups were firstly compared with a one-way analysis of variance (ANOVA) on their school mathematics and science achievement scores before the experiment. As shown in Table 3.1, the statistic analysis indicated that there was no significant difference among groups in prior mathematics or science achievement (F(2, 24) = 0.893, p = .422, for mathematics; F(2, 24) = 1.579, p =.227, for science).. Table 3.1 Comparison of Mathematics and Science Achievement Scores between Groups Variable. Group. M. SD. F(2, 24). p. Mathematics score. SG SLPA FLPA SG SLPA FLPA. 70.89 66.78 70.44 75.56 71.89 75.56. 9.05 7.43 4.10 3.84 5.75 3.68. 0.893. .422. 1.579. .227. Science score. Note. SG, static graphics; SLPA, simple learner-pacing animation; FLPA, full learner-pacing animation.. III.3.4. Measuring instruments III.3.4.1. Subjective mental effort scale The mental effort scale, originally developed by Paas (1992), was used as a subjective cognitive load measurement in this study. Mental effort refers to the amount of cognitive capacity or resources that students allocate to accommodate the task demands (Paas, 1992). This self-report measure is widely accepted as a valid and 31.

(42) noninvasive method to estimate cognitive load (e.g., Paas, 1992; Paas & van Merriënboer, 1993; Sweller, van Merriënboer, & Paas, 1998; Tuovinen & Paas, 2004; van Merriënboer, & Sweller, 2005). In this study, after students finished instructions, they were immediately asked to report the mental effort which they invested in learning. The mental effort was rated on the 9-ponit Likert scale by choosing 1 (very, very low) through 9 (extremely high).. III.3.4.2. Practical performance test In order to determine the learning outcomes, all students were required to measure the specific object with a real Abney Level using trigonometric leveling. Each student‟s practical performance was scored according to the marking scheme as shown in Table 3.2.. Table 3.2 Marking Scheme for the Practical Performance Test Criterion. Score. To read the observed height on the leveling rod by aligning the level bubble with the index line To determine the observer‟s eye-height. 1 1. To determine the projecting angle by adjusting the arm on the protractor To write down the calculation process of measuring data To attain the correct answer by applying the fundamentals of trigonometry. 1 1 1. Note. If the student met one of the above criteria, he/she would get 1 point.. III.3.4.3. Instructional time-span Data on the time-spans (in seconds) which each student spent on learning was collected online. During the experiment, the students could replay and rewind the visualizations as well as texts again and again. Only when a student pressed the logout 32.

(43) button of the computer-based multimedia would the server record his/her total time-span in learning.. III.3.4.4. Instructional efficiency To date, an absolute value that distinguishes between the acceptable and unacceptable levels of cognitive load has not been established. The relative instructional efficiency which combines the mental effort ratings and performance levels of students in different instructional designs may offer more informative and practical implications for the multimedia comparison (Paas & van Merriënboer, 1993; Sweller, van Merriënboer, & Paas, 1998; Tuovinen & Paas, 2004). The approach, originally developed by Paas and van Merriënboer (1993), converts the raw scores of students‟ mental effort ratings and performance levels into z-scores (the score is subtracted from the grand mean and then is divided by the grand standard deviation), respectively. Thus, a set of z-scores for Mental effort ratings (M) and Performance levels (P) is obtained. Then relative instructional Efficiency scores (E) can be computed for each student by using the following formula:. Obviously, if P is equal to M, instructional efficiency is zero. This formula is derived from computing the perpendicular distance from a point, which is labeled as (M, P) in a Cartesian coordinate system, to the zero efficiency line (i.e., E = 0). The E-value obtained from this formula reduces the threat that the students‟ subjective mental effort ratings merely report their self-confidence or comfort levels in learning rather than cognitive load (Paas & van Merriënboer, 1993; Sweller, van Merriënboer, & Paas, 1998; Tuovinen & Paas, 2004). Accordingly, if P is lower than the M, instructional efficiency 33.

(44) is negative. On the other hand, if P is higher than M, instructional efficiency is positive.. III.3.5. Data analysis A one-way ANOVA was conducted on the students‟ cognitive load ratings, practical performance scores, instructional time-spans, and instructional efficiency to detect any significant difference between the three experimental groups. Since the sample size was rather small, the coefficient of effect size may offer more informative implications of the data in this case. Here, Cohen‟s f is appropriate to describe the effect size for F-test. According to Cohen‟s rough characterization (1988), ƒ effect sizes of 0.1, 0.25, and 0.4 are termed small, medium, and large, respectively. All statistical tests were conducted at the alpha = .05 significance level by using Statistical Package for Social Sciences (SPSS) version 15.0.. 34.

(45) III.4. Results. Table 3.3 presents the results of the one-way AVONA on mental effort ratings, practical performance scores, instructional time-spans, and instructional efficiency for groups. Moreover, once a significant F-value was obtained in the AVONA, Tukey's Honestly Significant Difference test (Tukey's HSD) was employed as the post-hoc test to exactly verify which means were significantly different from which other ones. The detailed comparison between groups for each variable is described in the following sections.. Table 3.3 Comparison of Mental Effort Ratings, Practical Performance Scores, Instructional Time-spans, Instructional Efficiency for Groups SD. F(2, 24). p. f. Tukey‟s HSD. SG(1) 7.56 SLPA(2) 6.56 FLPA(3) 5.44. 1.13 1.24 1.59. 5.646*. .010. 0.69. (3) < (1). Practical SG(1) 1.67 performance score SLPA(2) 1.11 FLPA(3) 2.89. 0.87 1.17 1.05. 6.931**. .004. 0.76. (3) > (1) (3) > (2). Variable. Group. Mental effort rating. M. Instructional time-span. SG(1) 301.49 117.66 0.459 SLPA(2) 282.84 120.14 FLPA(3) 253.98 73.38. .637. 0.20. Instructional efficiency. SG(1) -0.60 SLPA(2) -0.46 FLPA(3) 1.14. .003. 0.79. 0.89 0.96 0.38. 7.547**. Note. SG, static graphics; SLPA, simple learner-pacing animation; FLPA, full learner-pacing animation. * p < .05; ** p < .01.. 35. (3) > (1) (3) > (2).

(46) III.4.1. Difference in subjective mental effort ratings As shown in Table 3.3, there was a statistically significant difference in mental effort ratings among three groups (F(2, 24) = 5.646, p = .010, f = 0.69, large effect size). On average, the mental effort ratings of FLPA group were lower than those of both SG and SLPA groups. Tukey‟s HSD showed that the FLPA group subjectively invested significantly less mental effort than did SG group (p = .007). However, the differences between FLPA/SLPA and SLPA/SG failed to reach the significant level (p = .202 and .269, respectively).. III.4.2. Difference in practical performance scores The mean scores of students‟ practical performances of FLPA group were, on average, higher than those of both SLPA and SG groups. As shown in Table 3.3, the differences of practical performance scores among groups reached the statistic significant level (F(2, 24) = 6.931, p = .004, f = 0.76, large effect size). In addition, Tukey‟s HSD showed that FLPA group significantly outperformed both SLPA and SG groups on the practical performance (p = .004 and .05, respectively). Besides, there was no significant difference between SLPA and SG groups in terms of practical performance scores (p = .501).. III.4.3. Difference in instructional efficiency As shown in Table 3.3, the differences in instructional efficiency that was computed by the z-score combination of students‟ mental effort ratings and practical performance scores among the three groups obtained the statistic significant level (F(2, 24) = 7.547, p = .003, f = 0.79, large effect size). Moreover, Tukey‟s HSD further indicated that the design of FLPA brought students significantly higher instructional efficiency than those of both SLPA and SG (p = .01 and .005, respectively). The 36.

(47) difference of instructional efficiency between SLPA and SG failed to reach the significant level (p = .952). To synthetically understand the relative instructional efficiency between the three formats of multimedia designs, a graphical method (Paas & van Merriënboer, 1993) was used to visualize the combined effects of the two measures, including the subjective mental effort ratings and practical performance scores. The mean z-scores of students‟ Mental efforts (M) and practical Performance (P) of each instructional condition were transformed into the format of (X, Y), i.e., labeled as (M, P), and then plotted on the four quadrant diagram as shown in Figure 3.3. The perpendicular distance from the zero efficiency line where E = 0 to each of the points plotted on the mental effort–practical performance cross of axes was the instructional Efficiency value (E) for that group calculated previously. As illustrated in Figure 3.3, FLPA (E = 1.05) is located at the high-efficiency (top-left) quadrant, whereas both SLPA (E = -0.46) and SG (E = -0.60) are located at the low-efficiency (bottom-right) quadrant. It clearly reveals that, among the three instructional conditions, the design of FLPA brought about relatively lower cognitive load with higher performance, and SLPA and SG brought about more cognitive load with lower performance.. III.4.4. No difference in instructional time-spans Although FLPA group on average spent less time on learning in contrast to both SG and SLPA groups, no significant effect was found in instructional time-spans (F(2, 24) = 0.459, p = .637, f =0.20, small effect size).. 37.

(48) Figure 3.3. Relative instructional efficiency representation for three instructional conditions. FLPA, full learner-pacing animation; SLPA, simple learner-pacing animation; SG, static graphics; E, instructional efficiency.. 38.

(49) III.5. Discussion and Implication. Even though the mental effort ratings of FLPA group were, on average, lower than those of both SLPA and SG groups, the results of statistic analysis on mental effort ratings among groups indicated that only the difference between FLPA and SG group reached the significant level (p = .007). It, however, only supports the partial statement of H1, that is, FLPA imposes less cognitive load on students than SG. Nevertheless, it clearly reveals that the addition of the simple leaner-pacing function is not enough; the students‟ cognitive load of SLPA was more or less equivalent to those of SG. Only when an animation is integrated with the full learner-pacing function (i.e., to allow students to control the speed, orientation, and change of presented objects in animation) will it significantly lower students cognitive load in multimedia learning as compared to static graphics. In addition, the results of statistic analysis on practical performance levels among groups revealed that the students of FLPA significantly outperformed than those of SLPA as well as SG (p = .004 and .05, respectively). It supports H2 that FLPA facilitates better learning outcomes than both SLPA and SG. Moreover, the results of statistic analysis on instructional efficiency among groups support H3 that FLPA has higher instructional efficiency than both SLPA and SG (p = .01 and .005, respectively). By following the basic point of cognitive load theory, to ease cognitive load in multimedia learning should facilitate better learning outcomes. As shown in Figure 3.3, the design of FLPA brought about relatively higher performance which accompanied with lower cognitive load, by comparing with the design of SLPA and SG. These findings also diminish the possibility that the mental effort ratings which students reported in this study merely reflected their self-confidence or comfort levels in learning rather than cognitive load. 39.

(50) However, the results of statistic analysis on instructional time-spans among groups reject H4 that FLPA requires less study time than both SLPA and SG. Although the FLPA group, on average, spent less time on learning than did both SLPA and SG groups, the differences among groups only obtained a small effect size (f =0.20). It might result from that the design of SLPA and SG sparked off the underwhelming effect (Lowe, 2004). The inappropriate design of SLPA and SG may induce students in an illusory feeling of understanding. The students, therefore, ceased to learn from multimedia in a state of being insufficiently engaged in information processing. Consequently, students spent not much time on multimedia learning and then performed poor achievement on the post-test. However, little research investigates the impact of different multimedia forms on students‟ instructional-processing time (Höffler & Leutner, 2007). Further studies are needed to confirm the aforementioned inferences. Nevertheless, this study at least backs up that, in contrast with SLPA and SG, the design of FLPA would not take students more time for learning. The practical implications these of results, in instructional animation design for science teaching, strongly suggest that the interactive functions of FLPA could serve as the aid in easing students‟ cognitive load on representational holding. The interactivity of FLPA allowed students to manipulate the external representation of the Abney Level at will, so that students were in complete control of the speed, orientation, and changes of presented objects in the animation. The students could, therefore, have spare visual working memory to cope with the on-screen explanations such as spatial relations between objects and running calculations to measuring data. This technique is a potential way for overcoming the drawback of the animation (i.e., the transient nature in comparison with statics graphics) as well as enhancing its advantage (i.e., the aid of depicting dynamic phenomenon) in computer-based science learning. As suggested by Tversky et al (2002), an instructional animation must be fitted to each learner to 40.