讀出眼睛的真心話: 以眼動追蹤法探討利用電子書學習英語之閲讀行爲

全文

(1)國立臺灣師範大學英語學系碩. 士. 論. 文. Master’s Thesis Department of English National Taiwan Normal University. 讀出眼睛的真心話: 以眼動追蹤法探討利用電子書學習英語之閲讀行爲. Meaningfulness is in the Eye of the Reader: Eye-tracking Insights of L2 Learners Reading Ebooks. 指導教授：劉宇挺 Advisor: Dr. Yeu-Ting Liu 研究生：劉素維. 中華民國一百零四年六月 June 2016.

(2) 摘要近年來利用電子書提升第二語言學習日漸廣泛，但現存研究還未能找出最佳數位內容的呈現模式以及不同閱讀輔助對學習行為與成效之影響。因此，本研究以眼動追蹤法（eyetracking）針對 52 位英語為第二語之高階學習者在兩種呈現模式（同時呈現、序列呈現）與六種閱讀輔助（三種局部理解：字彙字義註解、重點字彙剖析（包含型、義、語用資訊）、註腳；三種整體理解：插圖、資訊圖表、照片）不同閱讀情境下的眼動行為進行分析。本研究發現，當以第二語學習者以字彙習得為閱讀目的時，他們會花更多凝視時間聚焦在重點字彙剖析與字彙字意註解上；但是第二語學習者以理解為閱讀目的時，則花更多的凝視時間在插圖分析上。此外，序列呈現模式（數位內容逐步呈現）則引導第二語學習者花更多的凝視時間，聚焦在重點字彙剖析上；這也代表在此模式下，受試者比較注意局部輔助資訊，閱讀目地也偏向以字彙習得為主。整體而言，本研究證明電子書學習第二外語之有效性，同時也發現在兩種電子語料呈現模式和不同閱讀目的下，六種多媒體輔助對閱讀行為的影響有相對顯著的差異。最後，根據本研究成果，本論文提出針對教師、電子書使用者以及電子書出版商對於閱讀教學之建議、並針對未來研究提供建議。. 關鍵字：眼動追蹤技術、多媒體學習、電子書、第二語言習得、字彙習得、閱讀理解. i.

(3) Abstract Employing the eye-tracking technology, this study probed the online reading behaviors of advanced L2 English learners reading for vocabulary acquisition and comprehension. Fifty-two university students with advanced L2 (English) proficiency read an e-book containing six types of multimedia supports for either vocabulary acquisition or comprehension. The six multimedia supports consisted of three micro-level supports that provided information about specific words (glosses, vocabulary focus, and footnotes), and three macro-level supports that provided global or background information (illustrations, infographics, and photos). The participants read the e-book under two input presentation modes frequently seen in instructional e-books: 1) simultaneous mode: where digital input and supports were presented at the same time; and 2) sequential mode: where the digital content and supports were incrementally presented. Analyses of the participants’ total fixation duration showed that vocabulary focus and glosses were significantly fixated on when reading for vocabulary acquisition, and illustrations were more intensely fixated on when reading for comprehension. Additionally, vocabulary focus received significantly higher total fixation duration when the digital content was incrementally presented. This suggests that reading under the sequential presentation mode has the potency to guide L2 learners’ conscious attention toward micro-level supports that encompass lexical form, meaning, and use. This study confirmed the effectiveness of multimedia learning, while illustrating significant distinctions between the relative contributions of micro-level and macro-level multimedia supports under different literacy purposes and presentation modes. Based on the findings of this study, L2 learners’ online e-book reading behaviors and pedagogical implications are discussed.. ii.

(4) Key words: eye-tracking, multimedia learning, e-books, second language learning, vocabulary acquisition, comprehension. iii.

(5) Acknowledgement I would like to sincerely express my gratitude to my thesis advisor, Professor Yeu-Ting Liu, for his unwavering support, guidance, understanding, and wisdom. Professor Liu has gone far beyond the duties of an advisor and has given me the confidence in believing I can overcome obstacles I never thought was possible. Whether it was persevering through difficulties or simply exchanging words of wisdom in life and research, Professor Liu has deeply planted the spirit and mindset of a researcher in me. This thesis will forever remind me that I am standing on the shoulders of a giant, and will never forget the debt and gratitude I owe to Professor Liu. I would also like to thank Professor Chun-Yin Chen from the English Department at National Taiwan Normal University, and Professor Heng-Tsung Huang from the Department of Foreign Languages and Literatures at National Taiwan University. Their insights and passionate support throughout the process has broadened my perspectives on the meaningfulness and potential capabilities of this research. I also owe my gratitude to the participants in this study, for it would not have been possible to complete this research without them. Most importantly, I must strongly express my gratitude to my parents. Their emotional support and understanding are the foundational building blocks of who I am today and the reason why the completion of this study was possible.. iv.

(6) 謝辭感謝父母親無邊的關懷，疼愛，體諒，與支持。父母親的培養與教育讓我懂得知恩，感恩，與報恩，讓我在研究過程中堅持不懈。沒有父母親的恩慈，就沒有今天的我，也就沒有今天的成就。我也要感謝劉宇挺指導教授的支持與引導。教授對學生的真誠跟研究的熱誠讓我學會度過所有困難。無論是事務或心靈上的煎熬，教授的精神是這研究過程中不可缺的一份力量與助力。我也要感謝師大英語系陳純音教授跟臺大外國語文學系黃恒綜教授的輔導跟栽培，讓這研究成功。. v.

(7) Table of Contents CHAPTER ONE. INTRODUCTION………………………………………………………...5. 1.1 Background…………………………………………………………………………..5 1.2 Overview of Existing Studies………………………………………………………..7 1.3 Gap in Existing Literature…………………………………………………………..8 1.4 Rationale of the Study……………………………………………………………….8 1.5 Research Questions…………………………………………………………………10 1.6 Organization of the Study………………………………………………………….11 CHAPTER TWO. LITERATURE REVIEW………………………………………………12. 2.1 Input Processing Predilection of L2 Learners…………………………………….12 2.2 Multimedia Supports for Written Vocabulary Acquisition in L2 Digital Research…………………………………………………………..…………………14 2.3 Multimedia Supports for Reading Comprehension in L2 Digital Research……18 2.4 Presentation Mode of Multimedia Supports for Promoting L2 Acquisition and Comprehension Purposes…………………………………………………………..22 CHAPTER THREE. METHODOLOGY...............................................................................23. 3.1 Participants………………………………………………………………………….23 3.2 Materials…………………………………………………………………………….24 3.3 Design………………………………………………………………………………..25 3.4 Procedure……………………………………………………………………………26 3.5 Apparatus……………………………………………………………………………27 3.6 Data Analysis………………………………………………………………………..28 CHAPTER FOUR. RESULTS………………………………………………………………..29. 1.

(8) CHAPTER FIVE. DISCUSSION AND CONCLUSION……………………………………35. 5.1 Discussion……………………………………………………………………………35 5.2 Pedagogical Implications…………………………………………………………...44 5.3 Conclusion…………………………………………………………………………...45 5.4 Limitations and Future Directions…………………………………………………45 References……………………………………………………………………………………….48 Appendix 1 E-book page one…………………………………………………..……………….54 Appendix 2 E-book page two………………………………………………………………..….55 Appendix 3 E-book page three………………………………….…………………………..….56 Appendix 4 E-book page four…………………………………………………………………..57. 2.

(9) List of Figures FIGURE 1. EXAMPLE PAGE OF AN E-BOOK WITH MULTIMEDIA SUPPORTS…….....5. FIGURE 2. ILLUSTRATION OF THE STUDY’S DESIGN………………………………....22. FIGURE 3. SNAPSHOT OF THE SIMULTANEOUS MODE……………………………….23. FIGURE 4. SNAPSHOT OF THE SEQUENTIAL PRESENTATION MODE……………….23. FIGURE 5. COLLAPSED HEAT MAP OF THE PARTICIPANTS’ TOTAL FIXATION DURATION ACROSS ALL PAGES WHEN READING FOR VOCABULARY ACQUISITION…………………………………………………………………..28. FIGURE 6. COLLAPSED HEAT MAP OF THE PARTICIPANTS’ TOTAL FIXATION DURATION. ACROSS. ALL. PAGES. WHEN. READING. FOR. COMPREHENSION……………………………………………………………..28 FIGURE 7. COLLAPSED HEAT MAP OF THE PARTICIPANTS’ TOTAL FIXATION DURATION ACROSS ALL PAGES WHEN READING UNDER THE SEQUENTIAL MODE…………………………………………………………..30. FIGURE 8. COLLAPSED HEAT MAP OF THE PARTICIPANTS’ TOTAL FIXATION DURATION ACROSS ALL PAGES WHEN READING UNDER THE SIMULTANEOUS MODE……………………………………………………....30. 3.

(10) List of Tables TABLE 1. SUMMARY OF INTERACTION RESULTS BETWEEN MULTIMEDIA SUPPORTS, LITERACY PURPOSE, AND PRESENTATION MODE………….31. 4.

(11) CHAPTER ONE INTRODUCTION 1.1 Background When an e-book is read, do second-language (L2) learners roam over the digital content and multimedia supports? Or do they selectively attend to different contents vis-à-vis different reading purposes? What elements are being attended and what elements are neglected? And, what are the pedagogical implications based on the insights of L2 learners’ e-book reading profiles? To answer the above inquiries, online measures are needed to capture their minds at work and examine how they process information between the lines. With language learning materials evolving into the digital era, so are L2 research studies shifting toward the realm of online methodologies. Among common mobile digital resources for reading are e-books. Digital mediums have become one of the most prevailing literacy training tools for English as a foreign language (EFL) learners to foster their reading competence beyond the classroom settings. Various multimedia supports have been embedded in e-books to foster reading competence and literacy skills for L2 learners. Such e-books can be further subdivided into two major categories: 1) reflowable e-books, which are characteristic of animated multimedia supports (e.g., videos), and 2) page-fidelity e-books, which only include static multimedia supports (see Rockinson-Szapkiw, Courduff, Carter, & Bennett, 2013). The multimedia supports in page-fidelity e-books can be presented in the form of glosses, vocabulary focus, footnotes, illustrations, infographics, and photos (Mayer, 1997). Among these multimedia supports in page-fidelity ebooks, some (e.g., glosses, vocabulary focus, and footnotes) intend to assist micro-level (local) comprehension of specific words. Glosses, vocabulary focus, and footnotes are supports frequently used in L2 e-books to assist micro-level comprehension. The three micro-level supports,. 5.

(12) however, differ in terms of their nature. Specifically, glosses – being defined as “a brief definition or synonym of words provided in text in L1 or L2” (Nation, 2002, pp.174-175) – primarily offer meaning information of either novel or known words in the context of the article; whereas vocabulary focus includes form (phonetics, parts of speech), meaning (definitions, synonyms), and use (contextualized sentence examples) of novel words (Al-Seghayer, 2001; Yanguas, 2009). Footnotes provide additional information of specific terms, such as the historical or cultural background of a keyword. Other multimedia supports (e.g., illustrations, infographics, and photos) aim to aid macrolevel (global) understanding, or the gist of the entire text. For example, illustrations and photos offer visualizations of the background or setting of the text. Similarly, aiming to enhance readers’ macro-level understanding, infographics present concise visual (e.g., diagrams) and textual (e.g., keywords) information to schematize the synopsis of the text (Gilbert; 2010; Mol, 2011; Smiciklas, 2012). Figure 1 below visually presents the aforementioned six micro- and macro-level supports, all of which are frequently seen in L2 page-fidelity e-books. Figure 1. 6.

(13) Example page of an e-book with multimedia supports. (Circled annotations: micro-level supports; squared annotations: macro-level supports.). 1.2 Overview of Existing Studies Understanding of what and how multimedia supports are processed by L2 learners in their real-time digital reading experience would offer empirically-established guidelines for what digital content should be included, highlighted, and how the content should be presented so as to optimize reading outcomes and minimize distractions (Author, 2015). To this end, researchers have appealed to numerous measures to gain further insights into learners’ underlying processing profiles while reading an e-book (e.g., Chen & Yen, 2013; Ciampa, 2012; Lin, 2010; Author, 2014; Roskos, Brueck, & Widman, 2009; Smeets & Bus, 2012; Tseng, Yeh, & Yang, 2015; Yoon, 2013). It is important to note that in an effort to probe language learners’ processing files, methodologies of existing research on e-books have mostly relied on data collected offline, either before or after the reading process. These methodologies included surveys, questionnaires, post-study interviews, simulated recalls, and reading comprehension post-tests (Chen & Yen, 2013; Ercetin, 2013; Hoffman & Paciga, 2014; Huang, 2013; Korat, Levin, Atishkin, & Turgeman, 2014; Lwo & Lin, 2012; Miller & Warschauer, 2014; Smeets & Bus, 2014; Türk & Ercetin, 2014). These offline methods are informative in offering insights into researchers’ observations and learners’ retrospective accounts on their reading process, and can be further collaborated with real-time processing profiles. Previous research not only mostly relied on offline measures, but also predominantly examined micro-level types of supports that serve to aid L2 learners’ lexical processing (Akbulut, 2007; Huang & Liang, 2015; Lin & Tseng; Shalmani & Sabet, 2010). The types of supports. 7.

(14) commonly studied included glosses and pictures, or a combination of both, for the purpose of vocabulary acquisition or form-focused processing. However, few studies have manipulated photos for the purpose of comprehension, or meaning-focused processing, and have scarcely examined the relative effects between form-focused and meaning-focused supports on L2 learners’ reading behaviors; an issue that will also be addressed in this study.. 1.3 Gap in Existing Literature First, with existing research mostly implementing offline measures and primarily focusing on multimedia supports geared toward lexical-based comprehension of digital texts, the lack of examination into L2 learners’ real-time conscious reading profiles has yet to be addressed. Second, with the ecological reality of instructional L2 e-books include both micro- and macro-level supports, research has yet to systematically address whether the disparate nature of supports significantly influence L2 learners’ digital reading experiences.. 1.4 Rationale of the Study Considering the gaps, the current study employed the eye-tracking technology to shed light on what and how the six aforementioned multimedia supports are attended to by L2 learners from their total fixation duration data during their real-time e-book reading experience. Total fixation duration reflects conscious visual attention and mental effort. The longer the fixation duration, the more the cognitive load and conscious attention invested by the L2 learner (Liu, Fan, & Paas, 2014; Tsai, Hou, Lai, Liu, & Yang, 2012; van der Laan, Hooge, De Ridder, Viergever, & Smeets, 2015). Examination of this data can thus shine light on what multimedia supports are consciously processed more by L2 learners (See the methodology section for more information).. 8.

(15) The current study analyzes L2 learners’ total fixation duration for two purposes. First, the study seeks to explore whether reading for acquisition or comprehension purposes would change what and how multimedia supports are processed by L2 learners for different literacy purposes. This objective is important and needs to be considered in the investigation of e-books designed for literacy purposes because, as indicated in Huang, Chern, and Lin (2009) and Author (2014), readers would change their input processing profiles in consideration of different literacy purposes (acquisition vs. comprehension). Especially with the ecological availability of both micro- and macro-level supports in e-books, it is unclear how different types of supports are affected due to literacy purposes. Moreover, although previous studies have examined the effect of literacy purposes, they rely mainly on offline behavioral data. This is precisely why the current study will employ the online eye-tracking approach to see whether different literacy purposes would alter how various multimedia support cues are attended. Second, the current study investigates whether the presentation method of digital content would alter how advanced L2 learners process the micro- and macro- level multimedia supports while reading an instructional L2 e-book. An idiosyncratic feature of e-books is their flexibility in presenting digital content. In real-life, e-book developers can pre-determine how digital content and multimedia supports are presented to readers. Two digital input presentation possibilities are frequently seen in existing e-books: (1) simultaneous presentation mode: under which the digital text and multimedia supports are presented together (e.g., Press, 2011); and (2) sequential presentation mode: under which the text is incrementally presented (e.g., Brown, 2008). With the exception of a few studies (e.g., Author, 2015), the ecological reality of e-books’ simultaneous and sequential. presentation. methods. is. rarely. considered. in. general. computer-assisted. teaching/learning studies. The present study intends to explore whether readers’ processing. 9.

(16) profiles of multimedia supports would vary under different (simultaneous and sequential) input presentation modes. Lastly, the study will examine whether L2 learners’ reading behaviors are consistent with VanPatten’s (2004) theory (discussed below) on language learners’ input processing profiles. Although the theory addresses processing profiles in non-digital settings, the current study deliberately uses this framework to examine whether the theory still hold true in digital settings. In sum, this study intends to address methodological (online measures), pedagogical (ecological nature of e-book designs for instructional purposes), and theoretical (VanPatten’s input processing theory) issues in the context of L2 digital learning.. 1.5 Research Questions In light of the above two major objectives, the current study explores the following two interlocking questions in the context of L2 page-fidelity instructional e-books: 1. Does literacy purpose (reading for vocabulary acquisition or comprehension) influence L2 learners’ processing preference for the micro-level and macro-level multimedia supports? If so, what are the relative weights of these supports? 2. Does input presentation mode (simultaneous vs. sequential) influence L2 learners’ processing preference for the micro-level and macro-level multimedia supports? If so, what are the relative weights of these supports? The two research questions aim to shed light on what and how multimedia supports are processed by L2 learners under different literacy purposes and different reading modes. It is important to note that the current study explores the two research questions using advanced L2 learners. Real-time processing profiles of advanced L2 learners “are presumably limited,. 10.

(17) and…involve few deviances from the native norm” (Hyltenstam, 1988, p.70). Examination of what and how multimedia supports are processed by advanced L2 learners could thus serve as the index of optimal reading strategies (Author, 2009).. 1.6 Organization of the Study The first chapter discussed the foundation underlying the purpose of this study, and provided an overview of gaps among existing literature in which the current study intends to address. The second chapter first discussed VanPatten’s (2004) theoretical framework in which this research was contextualized, and then reviewed previous studies examining L2 learners’ input processing profiles when reading for vocabulary acquisition and comprehension in digital settings. The third chapter explained the types of participants examined, the reading materials implemented for the study, the testing procedure, the type of eye-tracker used, and how the data were analyzed. The fourth chapter demonstrated the total fixation duration results, and the fifth chapter discussed the major findings. Pedagogical implications and future directions were also suggested in the last chapter.. 11.

(18) CHAPTER TWO LITERATURE REVIEW The purpose of integrating multimedia supports in e-books is to facilitate L2 learners’ reading outcomes for different literacy purposes. However, the multimedia supports are not always deemed important by L2 learners, with some of them being processed with focal attention and others being left unattended. Consequently, various (microvs. macro-level) multimedia supports exert differential impacts on the learners’ reading experiences. To provide a theoretical account of how learners selectively process various reading supports, the ensuing sections will review VanPatten’s (2004) Input Processing Principles (Han & Liu, 2013; Han & Peverly, 2007), and then discuss computer-assisted language learning (CALL) studies examining L2 learners’ input processing profiles in the context of acquisition and comprehension reading purposes. Note that existing CALL research has established the efficacy of various types of (textual) glosses (e.g., L1 and L2 glosses) as macro-level supports in enhancing vocabulary acquisition and comprehension (see Lin & Yu, 2016; Author, 2015). What still remains unresolved concerns the comparative efficacy of glosses in relation to other (micro- and macro-level) support cues. Thus, the review of relevant CALL research will focus mainly on studies that are set out to examine the relative effects of glosses and other support cues in the context of promoting L2 vocabulary acquisition and comprehension. The literature review will conclude with a discussion of how input presentation modes may affect L2 learners’ reading behaviors.. 2.1 Input Processing Predilection of L2 Learners Based on the findings of a substantial amount of L2 cognition studies, VanPatten (2004) proposed the oft-cited Input Processing Principles theory. According to VanPatten, two major. 12.

(19) cognitive processes are performed by L2 learners when exposed to L2 input: (1) analysis of form for the purpose of acquisition, and (2) extraction of meaning for understanding and exchanging of ideas. Optimal L2 learning outcomes require the processing of both form and meaning. However, simultaneous processing of and attention to both form and meaning are extremely difficult due to limited attentional capacity. Based on the above view, VanPatten makes four specific propositions concerning L2 input processing: 1) learners do not process meaning-bearing and form-bearing information at the same time; 2) learners process input mainly for meaning; 3) learners can only attend to form after (overall) meaning (of the input) is comprehended; and 4) simultaneous processing of formal and meaning-bearing information can lead to inadequate processing of either or both types of information and is thus non-optimal. The above four propositions, which have gained a great number of empirical support over the past two decades (e.g., Hulstijn, 1989; VanPatten, 1990; VanPatten, 2008; VanPatten & Wong, 2004), entail that the processing of form and meaning is separate, modular, and sequential (meaning-focused processing takes place prior to form-focused processing); and that learners would give higher priority to meaning-bearing information to form-bearing information. VanPatten’s account noted above has important implications on what multimedia supports (microvs. macro-level) may be perceived as more important by L2 learners while processing digital content. Specifically, while micro-level supports such as glosses and vocabulary focus aim to direct their attention to either form or form-meaning (and therefore belong to form-bearing information), macro-level supports such as illustrations and infographics aim to orient learners’ attention to the meaning aspect of digital content (and thus pertain to meaning-bearing information). With the aforementioned L2 input processing predilection, it is conceivable that L2 learners would give more priority to macro-level supports that enhance meaning-focused. 13.

(20) processing than micro-level supports that promote form-focused processing or form-meaning mapping. Notwithstanding, it should be noted that VanPatten’s account has emanated primarily from studies of L2 learners processing non-digital input (e.g., printed books). Whether these stipulations still hold true for L2 digital input processing remain to be tested. Furthermore, it remains unknown whether the proclaimed L2 input processing preferences – giving higher priority to meaningbearing information such as macro-level supports – would change as a result of meeting different task demands (e.g., explicit instruction to ask learners to read for acquisition or for meaning) and reading e-books under different presentation modes (simultaneous vs. sequential). The study reported here attempts to delve into this issue.. 2.2 Multimedia Supports for Written Vocabulary Acquisition in L2 Digital Research Numerous CALL studies have operated various multimedia supports to investigate L2 learners’ reading behaviors and (written) vocabulary acquisition in digital research (Akbulut, 2007; Al-Seghayer, 2001; Ariew & Ercetin, 2004; de Jong & Bus, 2003; Yanguas, 2009). Despite the availability of various micro- and macro-level multimedia supports in L2 digital learning materials, existing studies have mainly focused on the following two supports: glosses and pictures. As a result of this skewed focus, the general understanding of the relative efficacy and importance of the six micro- and macro-level supports under investigation is still very limited. The ensuing section will review major CALL studies that are set out to investigate the relative efficacy of glosses and pictures (referring either to illustrations or photos) in promoting (written) vocabulary acquisition.. 14.

(21) A notable early study conducted in the area of multimedia learning for written vocabulary acquisition was the one endeavored by Chun and Plass (1996). Three separate experiments were undertaken based on a within-subjects design to examine the relative weight between multimedia supports for incidental written vocabulary acquisition when the primary task purpose is to read a short story for comprehension. Supports used in the short story were glosses, glosses with picture, and glosses with video. All participants were L1 English university students enrolled in secondyear L2 German courses. Acquisition scores from vocabulary tests showed significantly higher scores for glosses with pictures and with videos than glosses alone, with pictures having the highest scores. This finding indicated that both pictures (i.e., photos) and videos were effective in promoting vocabulary gains, with pictures being the most beneficial support. Parallel to the results from the immediate vocabulary tests, again, learners showed a preference for glosses with pictures (i.e., photos) as the major retrieval cue compared to other types of support. In light of this finding, the authors speculated that pictures have a constant quality allowing learners to form a vivid mental image in their memories, whereas videos consist of dynamic animations giving learners less time to establish a constant mental model. The authors concluded that pictures were the most influential support. Chun and Plass’s (1996) finding – the prominent role of picture – is later challenged by AlSeghayer’s (2001) experiment. The study was based on a within-subjects design exposing all 30 intermediate ESL participants to three conditions for the purpose of written vocabulary acquisition in an English narrative: glosses alone, glosses with pictures (i.e., illustration), and glosses with videos. Two types of tests were administered, recognition and production, to measure vocabulary acquisition. Overall results from both tests yielded the conclusion that pictures were not the optimal multimedia support in facilitating written vocabulary acquisition; a direct contradiction to. 15.

(22) Chun and Plass’s (1996) study. The authors suggested videos build better mental images due to its multitudinous combination of modalities (dynamic animations, sound, and textual support) and hence, better retention of unknown written words. In line with Al-Seghayer’s (2001) results were findings collected by a more current study, Lin and Tseng (2012), in which pictures (i.e., illustrations) were not found to be the optimal multimedia support for vocabulary acquisition. Lin and Tseng examined 88 beginner-level EFL students who were randomly assigned to one of three designs: glosses, glosses with pictures, and glosses with videos. Immediate and delayed post-tests in both recognition and production of newly learned vocabulary words revealed significant differences between the three groups, with video supports leading to the highest scores and glosses alone resulting in the lowest scores. Drawing on Weiss, Knowlton, and Morrison’s (2002) interpretation on videos, Lin and Tseng (2012; p.351) reasons that still pictures such as illustrations “can neither illustrate complicated meanings embedded in the target words nor serve as a retrieval channel [for the forms of the target vocabulary]. On the other hand, the dynamic animations, such as videos, offer richer contexts to facilitate learners’ construction of schema, and are thus better able to generate form-meaning connections of unknown vocabulary when reading L2 digital content. However, Akbulut (2007) offers findings distinctive from those previously mentioned. This study found no significant differences between videos and pictures (i.e., photos) as multimedia supports for vocabulary acquisition. In this study, 69 advanced EFL students were randomly assigned to one of the following three multimedia support conditions: glosses alone, glosses with pictures, and glosses with videos. Results were analyzed based on three categories: form recognition, meaning recognition, and meaning production. Immediate and delayed post-tests from all three categories collectively revealed that both glosses with pictures and with videos were. 16.

(23) significantly more effective than glosses alone, but that no significant difference was found between pictures and videos. In other words, the two visual multimedia supports, videos and pictures, both facilitate incidental vocabulary learning and retention; but neither type of visual support is superior than the other. Yanguas’s (2009) study of low-intermediate L2 learners of Spanish yielded findings that seem to be suggestive of the superior role of pictures (a mixed use of illustration and photos) to gloss. In this study, 94 participants were assigned to read a digital text under four conditions: 1) gloss-only; 2) picture-only; 3) gloss + picture; and 4) control. Yanguas found that the participants’ comprehension scores were significantly higher under the gloss-plus-picture group, indicating that comprehension was best enhanced by combination of pictorial and textual information. Although no significant differences were found among the other groups, many participants indicated in the think-aloud protocol that pictures are “weird,” “distracting,” and, “not helpful”. Such negative comments, however, were not found in the gloss group. This discrepant finding leads Yanguas to conclude that (textual) gloss seems to be more helpful than pictures. Recently, using illustration as a macro-level support, Author (2015) examined highintermediate EFL learners’ L2 (English) vocabulary retention performance under the following reading conditions: (1) illustration + L2 gloss; (2) illustration; (3) illustration + L1 pictographic gloss; and (4) illustration + L1 gloss. Author found that the participants had the best recall of novel L2 words under the “illustration” and “illustration + L2 gloss” conditions; however, the difference between the two are not statistically significant. The ‘additive’ contribution of (L2) glosses thus seems very limited. This in turn entails that illustration appears to be a more effective comprehension support compared with (L1 and L2) glosses; but its relative effect in comparison to other macro-level supports such as infographics and photos are yet to be established.. 17.

(24) The above review shows that researchers have loosely used the generic term “picture” to refer to either graphic “illustration” (e.g., Al-Seghayer, 2001) or “photo”, i.e., image made using a camera (e.g., Chun & Plass, 1996; Yanguas, 2009). In this regard, photos and illustrations are both subsumed under the same (i.e., “picture”) multimedia support category. Because of this conceptual or methodological conflation, the net and relative effects of photos and illustrations remain unclear. It is likely that illustrations and photos may invoke very different underlying input processing profiles. What further exacerbates the picture regarding the relative effects of illustrations and photos is that the efficacy of glosses and ‘pictures’ (referring to either illustrations or photos) are typically examined in combination with other supports (e.g., gloss + illustration or gloss + video). Consequently, findings of existing studies do not clearly reveal the relative contribution of glosses, illustrations and photos. Last but not least, with the exception of Author (2015), ‘pictures’ (either photos or illustration) are generally operationalized by researchers as micro-level comprehension supports (e.g., pictorial gloss) to assist learners’ understanding of novel L2 vocabulary, rather than as macro-level supports to aid learners’ global understanding, as seen in many learning materials designed for adolescent or adult L2 learners. Due to this ecological validity issue in research, the efficacy of photos and illustrations as macro-level supports – a common practice in actual L2 learning materials – is yet to be established. The following section will review research on using multimedia supports for promoting reading comprehension.. 2.3 Multimedia Supports for Reading Comprehension in L2 Digital Research Studies on multimedia supports for the purpose of reading comprehension have similarly attained inconsistent results also probably due to the methodological issues noted above (e.g.,. 18.

(25) conflating photos and illustrations as being multimedia supports of the same nature). Shalmani and Sabet (2010) explored the effect of multimedia supports on reading comprehension of academic texts using three experimental groups: glosses, pictures (i.e., illustrations), and glosses with pictures (i.e., glosses + illustrations). Using a pre-test and post-test experimental design, the study recruited 120 intermediate-level Iranian L2 learners and explored the participants’ reading outcomes after reading five academic passages under the aforementioned three multimedia support conditions. Findings indicated passages facilitated with both glosses and pictures outperformed either supports alone, with glosses alone as the least effective treatment. That in turn directly suggests that pictures are more effective than glosses in promoting comprehension gains. The authors argued that pictures are more easily remembered than glosses, and are readily transferred to learners’ long-term memory. This is in parallel with Underwood’s (1989) statement: “A commonplace principle of human learning is visual memory. We remember words better if they are strongly associated with images” (p. 19). In parallel with Shalmani and Sabet is a study by Yanguas (2009) who explored the effect of glosses, pictures (i.e., illustrations), and glosses with pictures (glosses + illustrations) on comprehension. 94 second-year university students taking L2 Spanish courses were assigned to one of the experimental conditions and were administered a comprehension test immediately afterwards. Findings indicated the group with both glosses and pictures outperformed all others, which is consistent with findings in other studies (Miyasako, 2002; Yoshii & Flaitz, 2002) that found combinational multimedia supports (i.e., glosses + pictures) more beneficial than either glosses or pictures alone. However, Sakar and Ercetin (2005) found a negative relationship between the use of multimedia supports and comprehension. Forty-four intermediate-level L2 learners were given the. 19.

(26) freedom to access any multimedia supports available in a hypermedia reading text, including glosses, graphics, pictures (i.e., photos), audio recordings, and videos. Analyses between the time spent on each multimedia support and reading comprehension scores all yielded negative correlations, while pictures (i.e., photos) had the highest frequency of usage and were preferred significantly more than textual supports (i.e., glosses). Although participants perceived pictures as beneficial, quantitative results implicated increased usage of combinational multimedia supports and reliance (time spent) on the pictures do not necessarily guarantee better comprehension outcomes. Similarly, Ariew and Ercetin (2004), which focused on intermediate and advanced learners, attained findings consistent with Sakar and Ercetin’s (2005) view. In Ariew and Ercetin’s (2004) study, two types of multimedia supports were used to facilitate micro-level (glosses that provided word definitions) and macro-level (contextual supports that provided global information on the topic, e.g., pictures, and videos) comprehension. Participants’ behavior toward the use of multimedia supports were recorded, and comprehension tests were administered immediately afterwards. Results illustrated all the micro- and macro-level supports (i.e., glosses, pictures, audios, and videos) did not significantly enhance comprehension. Negative correlations between comprehension scores and the time spent on the multimedia supports were found, with pictures (i.e., illustrations) and textual glosses showing no significant effect on the L2 learners’ comprehension. There were no significant differences between usage frequency of pictures and glosses either. A plausible explanation can be inferred from the interviews where participants indicated they took advantage of most of the multimedia supports that were available and easily accessible, even if they were already familiar with the information. Hence, it is likely that when the participants were exposed to a rich amount of supports, they spent too much time processing. 20.

(27) the supporting information at the cost of not investing sufficient focal attention on synthesizing the main text (Aust, Kelley, & Roby, 1993). In this regard, multimedia supports were not processed to the advantage of the participant’s comprehension outcomes and were not effective in enhancing comprehension if the time spent on them was more than necessary. This in turn suggests that teachers probably need to selectively (re)orient L2 learners’ attention to particular multimedia support cues for different reading purposes. Taken together, similar to research on acquisition, studies on comprehension yielded mixed findings on the most desirable support cues due to methodological issues. Specifically, the relative effects of photos and illustrations on promoting L2 comprehension are unclear due to methodological conflation of these two supports. Empirical evidence yielding clearer insights into the net and/or relative effects of photos and illustrations are warranted. It is worth noting that most of the existing studies have not directly investigated whether reading e-books for different literacy purposes would alter the relative effects of multimedia supports. Yanguas (2009) and Author (2015) are one of the very few studies that delved into this issue. Yanguas and Author found that combinations of glosses and pictures (i.e., illustration) consistently remain the optimal reading supports for comprehension and acquisition purposes. That is, literacy purpose does not seem to modulate the relative efficacy of multimedia supports. However, Yanguas’ finding is yet to be verified by empirical real-time reading evidence. Building on top of this, future research should take into account previously studied support cues to examine how presentation modes of the supports affect reading processes. This inquiry would shed light on the optimal digital content presentation for enhanced reading outcomes – an issue we will now turn to in the following section.. 21.

(28) 2.4 Presentation Mode of Multimedia Supports for Promoting L2 Acquisition and Comprehension Purposes What and how L2 input is presented can jointly influence learners’ choice when exposed to a multitude of possibilities, and consequently, their underlying cognition and ultimate learning outcomes. This point was illustrated in Gweon and Schulz’s (2011) cognitive study. This research showed that despite receiving the same learning materials, young learners would reach different inferences and decisions when the materials were presented in different sequences. Namely, how the input is displayed matters. Similarly, how e-books present the digital content may influence learners’ reading choices and behaviors. The following two e-book studies are among the few that address the presentation manner/mode issue of multimedia supports. AbuSeileek (2011) found textual annotations placed physically closer to the text were more effective for acquisition and comprehension than those placed farther from the text. The author reasons that shorter distances require less attentional split while integrating information. In a similar vein, Author’s (2015) study compared simultaneous and sequential presentation of multimedia supports, where findings indicated the simultaneous mode as more effective for both acquisition and comprehension. Although AbuSeileek (2011) and Author (2015) focused on different forms of presentation modes, both studies mainly focused on the efficacy of micro-level supports. The current study intends to take a step further and speculate L2 learners’ reading preferences toward micro-level supports together with macro-level supports under the influence of different literacy purposes and presentation modes.. 22.

(29) CHAPTER THREE METHODOLOGY 3.1 Participants A total of 59 EFL undergraduate students enrolled at a national university in northern Taiwan were originally selected to participate in the study. Ages ranged from 18 to 22 years, and all the participants’ native language were Mandarin Chinese. The participants came from different academic disciplines: education, psychology, human development, Chinese, history, and music. To ensure that the recruited participants were indeed advanced L2 (English) learners, they were expected to have a total TOEFL iBT score above 100 points (out of 120) and have a reading score between 25 and 30 (out of 30) on the reading section of the TOEFL iBT, which falls within the C1-level reading proficiency band prescribed by the Common European Framework of Reference (CEFR) Levels. The CEFR is an empirically and internationally established framework used to compare exam scores across languages with detailed analyses for each competency level, and L2 English learners with C1-level reading proficiency score can be deemed as advanced L2 learners. This study examined advanced learners since it is assumed they are skilled readers (Hyltenstam, 1988), and should therefore provide optimal reading profiles for the purpose of this study. The reading profiles of advanced L2 learners could also serve as a model for instructors, L2 learners, and e-book designers in understanding how the interface of e-books can be tactfully used to guide lower proficiency learners to improve their reading strategies and process input similar to advanced learners. Participants who had prior experience studying abroad in an English-speaking country were not qualified for the experiment. To ensure consistency and accuracy of eye-tracking data, seven participants were excluded due to cases of drift, or irregular eye movements during the eyetracker’s calibration task before the experiment, resulting in a total of 52 participants.. 23.

(30) 3.2 Materials The reading material used in the study was the digital article adapted from Advanced magazine, titled “Fighting Cancer, with a Little Help” by George Diaz. Advanced magazine covers a wide range of topics from international magazines and is tailored for advanced L2 (EFL) learners. It has become a popular language learning resource for advanced EFL learners in Taiwan and China (Liou, Chang, Chen, Lin, Liaw, Gao, & You, 2006), and was therefore considered as an appropriate reading material for the participants. All the participants knew of this magazine, but had no prior knowledge of the specific article chosen for this study. This article was adapted for the purpose of the study, and included glosses, vocabulary focus, footnotes, illustrations, infographics, and photos. These multimedia supports are commonly seen in EFL digital learning materials, and can be clearly categorized into either micro- or macro-level supports for acquisition and comprehension purposes. Figure 1 above shows the layout of the digital content and the multimedia supports of a digital page. The body of the text consisted of words within the 3000 frequency level, according to the Corpus of Contemporary American English (COCA). Words below this level are considered high frequency words, and are thus assumed to be within advanced L2 learners’ vocabulary knowledge. The ten target words input into the text were selected based on GRE vocabulary lists so as to ensure the participants do not have prior knowledge of those words. To further warrant this, the participants were asked to identify words they recognized from a list of both the target and filler words, and those who only identified the filler words were allowed to participate. The reading material was deliberately designed in a way so that the advanced-level participants should easily understand the body of the text except for the target words. This way, the. 24.

(31) participants can focus on their literacy purpose in face of different presentation modes, multimedia supports, and novel target words; and provide reading profiles for the purpose of this study.. 3.3 Design The participants took part in the study at an eye-tracking lab. Each participant was randomly assigned to one of four groups: SmV, SmC, SqV, and SqC (Sm: simultaneous mode; Sq: sequential mode; V: vocabulary acquisition purpose; and C: comprehension purpose). Those in the SmV and SmC groups were exposed to the simultaneous presentation mode of both texts and supports, and read for acquisition and comprehension purposes, respectively. Participants in the SqV and SqC groups were exposed to the sequential presentation mode, in which all multimedia supports were displayed while the text was serially presented – word by word, from left to right – and read for acquisition and comprehension purposes, respectively. The design of two input presentation methods was motivated by the practice of existing instructional e-books (e.g., Brown, 2008; Press, 2011). A concise illustration of the design is shown in Figure 2, and snapshots of the simultaneous and sequential presentation modes are illustrated in Figures 3 and 4 below, respectively. Figure 2. Illustration of the study’s design. 25.

(32) Figure 3. Snapshot of the simultaneous mode Figure 4. Snapshot of the sequential presentation mode. 3.4 Procedure A nine-point calibration grid was implemented at the start of the experiment and after each one-minute pause between the pages to ensure accuracy of the data collection. Six areas of interests (AOIs) were created with rectangle boundaries around each of the 6 multimedia supports (glosses, vocabulary focus, footnotes, illustrations, infographics, and photos) to collect the total fixation. 26.

(33) duration within the areas. Each participant was given a few practice trials with different materials in order to familiarize with the eye-tracker. The participants reading for comprehension purposes were explicitly told to read for comprehension. To ensure this, they were asked to write one word or draw a simple picture on paper of what they read during a one-minute pause between each page – a time constraint determined by the maximum time it took for the participants in a pilot test – so as to give them sufficient time to write. The pilot test was administered to a similar population prior to the study. The participants reading mainly for acquisition purposes were told to read for learning new words. They were encouraged to write down any new words they deemed to be important during a oneminute pause between each page, and, if possible, remember those words. The participants were notified of a one-minute time limit to read each page – a time constraint determined by the average time it took for the participants in a pilot test to finish each page – before they took the one-minute pause to draw a picture to illustrate their understanding (for the participants reading for comprehension) or to write down important novel words (for the participants reading for acquisition). A time constraint was administered in order to analyze the total fixation duration based on how the participants divide their conscious attention among the text and multimedia supports as they read over each page once. The above reading events were repeated for each digital page.. 3.5 Apparatus The Eyelink 1000 eye-tracker used to monitor the participants’ monocular eye movements was placed on a desktop mount and collected data from the right eye. The sampling rate was 500 Hz, and the display resolution of the computer screen was 1024 (width) by 768 (height). The total. 27.

(34) fixation duration data was of interest to this study. A fixation is the area at which our eyes pause at. When reading, our eyes do not move in smooth progression, but in a non-linear fashion, jumping from one area to another (Rayner, Slattery, & Bélanger, 2010). In this study, the six multimedia supports are the target AOIs. Total fixation duration, usually measured in milliseconds, is the sum of all the eye fixations on a given area of interest during an online viewing/reading task (Holmqvist, Nyström, Andersson, Dewhurst, Jarodzka, & Van de Weijer, 2011; Poole & Ball, 2006; Slattery & Rayner, 2010). The longer the total fixation duration for each AOI, the more conscious processing and attentional resources the readers devoted to that AOI. The analysis of this duration metric is therefore indicative of readers’ conscious reliance and attentional investment on a particular multimedia support, especially in cases of reading difficulty (Kotval & Goldberg, 1998). The duration data is captured and presented in milliseconds in the present study.. 3.6 Data Analysis The study involved three independent variables: (1) multimedia supports (the aforementioned six major supports); (2) literacy purpose (acquisition vs. comprehension); and (3) presentation mode (simultaneous vs. sequential). Literacy purposes and presentation modes are the between-subject independent variables, multimedia supports are the within-subject variables, and total fixation duration is the dependent variable. Accordingly, a 6 (multimedia supports) x 2 (literacy purposes) x 2 (presentation modes) mixed ANOVA was applied to the eye-movement data in hopes of determining whether the relative effects of the six multimedia supports varied significantly between the SmV, SmC, Sqv, and SqC groups.. 28.

(35) CHAPTER FOUR RESULTS The main effect of the multimedia supports variable was statistically significant [F = 45.11, p < .001, η2 = 0.48]. This means that the total fixation duration (TFD henceforth), in milliseconds (ms), of the multimedia supports significantly differed regardless of presentation mode and literacy purpose. The main effect of the literacy purpose variable was statistically significant [F = 13.29, p < .01, η2 = 0.22]. This entails that the TFD significantly differed when the participants read for vocabulary acquisition and for comprehension. The TFD for all multimedia supports was significantly higher when reading for vocabulary acquisition than for comprehension (M = 633882 ms vs. M = 403419 ms, respectively, p < .01). This indicates that more fixation time was spent on (some) multimedia supports when reading for vocabulary acquisition than for comprehension. The main effect of the presentation mode variable was statistically significant [F = 9.49, p < .01, η2 = 0.17]. This means that the TFD for all multimedia supports significantly differed between the simultaneous and sequential presentation modes. The TFD was significantly higher when reading under the sequential presentation mode than the simultaneous presentation mode (M = 610780 ms vs. M = 426521 ms, respectively; p < .01), suggesting that the participants seemed to spend more time processing (some) multimedia supports when reading under the sequential mode than under the simultaneous mode. There was a significant interaction between the multimedia supports and literacy purpose variables [F = 14.41, p < .001, η2 = 0.23]. The simple main effects analyses indicated that the TFD of the six multimedia supports significantly differed between reading for vocabulary acquisition and for comprehension purposes (M = 633882 ms vs. M = 403419 ms, respectively; p < .001), with. 29.

(36) TFD being much longer when reading for acquisition. Post-hoc comparisons were conducted to further testthe simple main effects, which showed that the TFD for glosses was significantly higher than other supports when the participants were reading for vocabulary acquisition (M = 9055 ms, SD = 6550 ms) than for comprehension (M = 5057 ms, SD = 3410 ms), with p < .01. Similarly, the TFD for vocabulary focus was also significantly higher than other supports when the participants were reading for vocabulary acquisition (M = 24931 ms, SD = 18297 ms) than for comprehension (M = 8316 ms, SD = 7937 ms), with p < .001. This means glosses were fixated slightly longer than vocabulary focus, but there was no significant difference between the two. This also entails that glosses and vocabulary focus are two major supports that the participants relied on when reading for acquisition. The TFD for illustrations was significantly higher when the participants read for comprehension (M = 3732 ms, SD = 2406 ms) than for vocabulary acquisition (M = 2401 ms, SD = 2184 ms), with p < .05; this finding suggests that illustrations are the key support that the participants drew on when reading for comprehension. The aforementioned input processing preferences under the two literacy modes are collaborated by the collapsed heat maps of the participants’ fixation durations across all pages when reading for vocabulary acquisition (Figure 5) and comprehension (Figure 6).. 30.

(37) Figure 5. Collapsed heat map of the participants’ total fixation duration across all pages when reading for vocabulary acquisition Figure 6. Collapsed heat map of the participants’ total fixation duration across all pages when reading for comprehension 31.

(38) The participants’ differential attention to the multimedia supports under investigation was also observed in post-hoc comparisons under different presentation modes. Specifically, the analysis also showed that there was a statistically significant interaction between the multimedia supports and presentation mode variables [F = 6.98, p < .01, η2 = 0.13]. Simple main effects analyses showed that the TFD of the multimedia supports significantly differed between the simultaneous and sequential presentation modes, with the latter receiving higher TFD than the former (M = 426521 ms vs. M = 610780 ms, respectively, p value < .01). Further testing of the simple main effect showed that the TFD for vocabulary focus was significantly higher under the sequential presentation mode (M = 22804 ms, SD = 15630 ms) than the simultaneous presentation mode (M = 10443 ms, SD = 14751 ms), with p < .01. This indicates that reading under the sequential mode invoked the participants to fixate longer on the vocabulary focus – the most prominent multimedia support they relied on when reading for vocabulary acquisition. On the other hand, no significant differences were found between multimedia supports under the simultaneous presentation mode, suggesting the simultaneous mode allows the participants to evenly distribute their attention toward all supports. Figures 7 and 8 are collapsed heat maps of the participants’ fixation durations across all pages when reading under the sequential and simultaneous presentation modes, respectively.. 32.

(39) Figure 7. Collapsed heat map of the participants’ total fixation duration across all pages when reading under the sequential mode Figure 8. Collapsed heat map of the participants’ total fixation duration across all pages when reading under the simultaneous mode 33.

(40) Finally, post-hoc analysis did not find any interaction between the literacy purpose and presentation mode variables (p value > .70). There was also no interaction between the multimedia support, presentation mode, and literacy purpose variables (p value > .70). Table 1 below presents the interaction results noted above. Table 1 Summary of interaction results between multimedia supports, literacy purpose, and presentation mode df. F. Sig.. Effect. Post-hoc comparison. size η2 Multimedia. Literacy. Supports. Purpose. Multimedia. Input. Supports. Presentation. 2.09 14.41 p < .001. 0.23. Acquisition* (633882 ms) > Comprehension (403419 ms). 2.09 6.98. p < .001. 0.13. Sequential* (610780 ms) > Simultaneous (426521 ms). Mode. 34.

(41) CHAPTER FIVE DISCUSSION AND CONCLUSION 5.1 Discussion The goal of this study was to examine whether literacy purpose (1st research question) and presentation mode (2nd research question) influence what and how L2 learners process micro-level and macro-level multimedia supports. The ensuing paragraphs will discuss findings in relation to the two research questions and VanPatten’s (2004) Input Processing Principles in the context of ebook reading. In response to the first research question, micro-level supports – glosses and vocabulary focus – were consciously heeded to for a significant amount of time when the participants read for vocabulary acquisition, regardless of presentation mode. On the other hand, macro-level supports – illustrations – were heeded to significantly longer when the participants read for comprehension, regardless of presentation mode. Two interesting observations emerge from the above holistic picture and deserve further discussion:. . 1st observation: Differential attention to the three micro-level supports when reading for acquisition (vocabulary focus = glosses > footnotes). Reading for vocabulary acquisition guided the L2 learners to pay significantly more attention to vocabulary focus and glosses, but not to another micro-level support under investigation (i.e., footnotes). This fixation pattern is clearly shown in the collapsed heat map of fixation durations of the participants reading for vocabulary acquisition (See Figure 5 above). The heat map indicates that the intensive fixations were mainly attributed to vocabulary focus and glosses; but footnotes were not intensively processed to the same extent.. 35.

(42) In regard to the first observation, researchers have claimed lexical knowledge – such as those provided in vocabulary focus and glosses – is not only beneficial for word-level acquisition, but also for global comprehension, and there may even be a causal connection between the two (Nassaji, 2006; Lervåg & Aukrust, 2010; Qian, 2002). Notably, of the two micro-level supports, vocabulary focus, in particular, was consistently processed for a significant amount of time among all multimedia supports when analyzed under both the literacy purpose and presentation mode variables. This suggests that the participants consistently deemed vocabulary focus as a key contributing factor to their understanding of the text regardless of whether or not this leads to better reading outcomes. What deserves further explanation is why footnotes did not receive significant amount of attention from the participants compared to vocabulary focus and glosses. In fact, as noted in the result section, footnotes, in general, received the lowest amount of conscious processing time among not only the micro-level supports, but also among the macro-level supports, regardless of literacy purpose and presentation mode. This statistical analysis was confirmed by the heat map of the eye fixation duration data. The heat map in Figure 5 above indicated that the participants rarely processed the footnotes (as gleaned by the lack of fixation duration on footnotes compared to all other supports). The discrepancy between the amount of fixation time on footnotes and on the other two micro-level counterparts (vocabulary focus and glosses) might have to do with what has been included in the footnotes (nature of information) and how the micro-level information in footnotes is positioned on a digital page (layout). Specifically, glosses and vocabulary focus provide direct foreground (lexical) information, whereas footnotes incorporate indirect background information of specific words; therefore, footnotes may not have directly facilitated. 36.

(43) the participants’ lexical knowledge for acquisition purposes. In terms of layout, AbuSeileek (2011) found micro-level supports located at the bottom of a text (as in the case of footnotes) were significantly outperformed by other multimedia supports located closer to the text. In this vein, the distance between footnotes and the text may impose greater cognitive load due to split attention effect. Split attention effect occurs when the participants need to divide their conscious attention between two physically separated information in order to integrate them into their learning process; thus increasing their cognitive load and diminishing their learning efficacy (Sweller, 2006).. . 2nd observation: The participants paid more attention to the three macro-level supports when reading for comprehension, but to different degrees (illustrations > photos > infographics). Reading for comprehension induced the L2 learners to intensively attend to illustrations, but not the other two macro-level supports (infographics and photos). Figure 6 above – a collapsed heat map of fixation durations of the participants reading for comprehension – shows that illustrations were intensively fixated on, whereas infographics and photos were rarely scanned.. In regard to the second observation noted above, the participants spent significantly more time fixating on illustrations than on photos when reading for comprehension, indicating a higher reliance on illustrations to assist their understanding. The differential effects of illustrations and photos can be attributed to the nature of the pictorial format of the two macro-level supports. Specifically, although both illustrations and photos convey information in pictorial format, photos include real-life snapshots, and perhaps self-evident visual details, which would therefore be less effective in prompting the learners to process further. Illustrations, on the other hand, include less. 37.

(44) visual details and may be open to different interpretations than real-life photos. This ‘insufficiency’ or ‘ambiguity’ probably has the potency to induce the participants to ‘fill in the gap’ with their schemata and prior knowledge, and strive to interpret the (macro-level) comprehension supporting information conveyed by illustrations, thereby enhancing the depth of input processing and fostering stronger memory traces for the information read (see Ally, 2012; Carpenter & Olson, 2012). In an informal post-study debriefing session, several participants clearly indicated that they made mental effort to ‘study’ the illustrations and thought about ‘the relationship between the illustration and the corresponding passage’ while reading the e-book. Additionally, the graphic nature of illustrations speaks directly to universal and language independent concepts (Jones, 2004). In this regard, illustrations are more likely to bypass the linguistic representation and processing of information, and are directly constructive toward the graphic representations of mental models which L2 learners use when comprehending the gist of the text (Author, 2015). The aforementioned idiosyncratic features of the pictorial qualities of illustrations partially explain why illustrations are consciously processed more than other macro-level supports (i.e., photos) when it comes to understanding the gist of the text. This is confirmed by studies reviewed previously, who believe information displayed in pictorial format aids L2 learners’ overall comprehension (AlSeghayer, 2001; Shalmani & Sabet, 2010; Yanguas, 2009). In stark contrast to the superior status of illustrations, infographics – being not the optimal support for enhancing macro-level understanding – convey information via combination of textual and pictorial formats which require the participants to process and convey both linguistic and graphic representations of information into their mental models. According to Mayer and Anderson (1991) and Schnotz (2008), information presented in two different formats (e.g., textual and graphic) that are both processed through the same channel can compete with each other for. 38.

(45) cognitive resources; thus, instigating L2 learners to split their attention in order to process and integrate them into their mental models. In the case of infographics, macro-level supporting information is simultaneously conveyed through two formats (e.g., keywords and graphic schematization), both of which are processed through a common channel (i.e., visual channel). Since infographics are built upon the interaction between textual and pictorial information, L2 learners would need to simultaneously process both formats of information in order to grasp what the infographics are trying to illustrate. In this sense, infographics would require more conscious processing time and effort to process and comprehend, yet outcomes in this study revealed infographics received significantly less fixation time. That is, the participants – who were all advanced L2 readers – chose not to invest too much focal attention on infographics (as reflected in their fixation time). The discrepancy between the finding of this study and Mayer and Anderson’s account can be explained by the findings of Ariew and Ercetin’s (2004) study. As reviewed earlier, Ariew and Ercetin found that spending too much time on all available macrolevel supports debilitates, rather than facilitates, L2 learners’ reading experiences. In face of this, the advanced L2 learners in this study might have deliberately and efficiently invested their attention on a macro-level support that requires less cognitive effort (i.e., illustrations) over a support that appears more time-consuming or perhaps require more effort to understand (i.e., infographics). In this regard, infographics turned out to be the less-optimal multimedia support. Although this study found literacy purpose to be a significant modulating factor in L2 learners’ reading behaviors, Author (2015) and Yanguas (2009) found results indicating otherwise. The discrepancy may be due to the difference in proficiency level of the participants examined. Lower-proficiency L2 learners, as examined by the above two studies, may not be as competent in reading as advanced-level learners, and would thus need to read at the word-level before reading. 39.