Numerous CALL studies have operated various multimedia supports to investigate L2 learners’ reading behaviors and (written) vocabulary acquisition in digital research (Akbulut, 2007;
Al-Seghayer, 2001; Ariew & Ercetin, 2004; de Jong & Bus, 2003; Yanguas, 2009). Despite the availability of various micro- and macro-level multimedia supports in L2 digital learning materials, existing studies have mainly focused on the following two supports: glosses and pictures. As a result of this skewed focus, the general understanding of the relative efficacy and importance of the six micro- and macro-level supports under investigation is still very limited. The ensuing section will review major CALL studies that are set out to investigate the relative efficacy of glosses and pictures (referring either to illustrations or photos) in promoting (written) vocabulary acquisition.
15
A notable early study conducted in the area of multimedia learning for written vocabulary acquisition was the one endeavored by Chun and Plass (1996). Three separate experiments were undertaken based on a within-subjects design to examine the relative weight between multimedia supports for incidental written vocabulary acquisition when the primary task purpose is to read a short story for comprehension. Supports used in the short story were glosses, glosses with picture, and glosses with video. All participants were L1 English university students enrolled in second-year L2 German courses. Acquisition scores from vocabulary tests showed significantly higher scores for glosses with pictures and with videos than glosses alone, with pictures having the highest scores. This finding indicated that both pictures (i.e., photos) and videos were effective in promoting vocabulary gains, with pictures being the most beneficial support.
Parallel to the results from the immediate vocabulary tests, again, learners showed a preference for glosses with pictures (i.e., photos) as the major retrieval cue compared to other types of support. In light of this finding, the authors speculated that pictures have a constant quality allowing learners to form a vivid mental image in their memories, whereas videos consist of dynamic animations giving learners less time to establish a constant mental model. The authors concluded that pictures were the most influential support.
Chun and Plass’s (1996) finding – the prominent role of picture – is later challenged by Al-Seghayer’s (2001) experiment. The study was based on a within-subjects design exposing all 30 intermediate ESL participants to three conditions for the purpose of written vocabulary acquisition in an English narrative: glosses alone, glosses with pictures (i.e., illustration), and glosses with videos. Two types of tests were administered, recognition and production, to measure vocabulary acquisition. Overall results from both tests yielded the conclusion that pictures were not the optimal multimedia support in facilitating written vocabulary acquisition; a direct contradiction to
16
Chun and Plass’s (1996) study. The authors suggested videos build better mental images due to its multitudinous combination of modalities (dynamic animations, sound, and textual support) and hence, better retention of unknown written words.
In line with Al-Seghayer’s (2001) results were findings collected by a more current study, Lin and Tseng (2012), in which pictures (i.e., illustrations) were not found to be the optimal multimedia support for vocabulary acquisition. Lin and Tseng examined 88 beginner-level EFL students who were randomly assigned to one of three designs: glosses, glosses with pictures, and glosses with videos. Immediate and delayed post-tests in both recognition and production of newly learned vocabulary words revealed significant differences between the three groups, with video supports leading to the highest scores and glosses alone resulting in the lowest scores. Drawing on Weiss, Knowlton, and Morrison’s (2002) interpretation on videos, Lin and Tseng (2012; p.351) reasons that still pictures such as illustrations “can neither illustrate complicated meanings embedded in the target words nor serve as a retrieval channel [for the forms of the target vocabulary]. On the other hand, the dynamic animations, such as videos, offer richer contexts to facilitate learners’ construction of schema, and are thus better able to generate form-meaning connections of unknown vocabulary when reading L2 digital content.
However, Akbulut (2007) offers findings distinctive from those previously mentioned.
This study found no significant differences between videos and pictures (i.e., photos) as multimedia supports for vocabulary acquisition. In this study, 69 advanced EFL students were randomly assigned to one of the following three multimedia support conditions: glosses alone, glosses with pictures, and glosses with videos. Results were analyzed based on three categories:
form recognition, meaning recognition, and meaning production. Immediate and delayed post-tests from all three categories collectively revealed that both glosses with pictures and with videos were
17
significantly more effective than glosses alone, but that no significant difference was found between pictures and videos. In other words, the two visual multimedia supports, videos and pictures, both facilitate incidental vocabulary learning and retention; but neither type of visual support is superior than the other.
Yanguas’s (2009) study of low-intermediate L2 learners of Spanish yielded findings that seem to be suggestive of the superior role of pictures (a mixed use of illustration and photos) to gloss. In this study, 94 participants were assigned to read a digital text under four conditions: 1) gloss-only; 2) picture-only; 3) gloss + picture; and 4) control. Yanguas found that the participants’
comprehension scores were significantly higher under the gloss-plus-picture group, indicating that comprehension was best enhanced by combination of pictorial and textual information. Although no significant differences were found among the other groups, many participants indicated in the think-aloud protocol that pictures are “weird,” “distracting,” and, “not helpful”. Such negative comments, however, were not found in the gloss group. This discrepant finding leads Yanguas to conclude that (textual) gloss seems to be more helpful than pictures.
Recently, using illustration as a macro-level support, Author (2015) examined high-intermediate EFL learners’ L2 (English) vocabulary retention performance under the following reading conditions: (1) illustration + L2 gloss; (2) illustration; (3) illustration + L1 pictographic gloss; and (4) illustration + L1 gloss. Author found that the participants had the best recall of novel L2 words under the “illustration” and “illustration + L2 gloss” conditions; however, the difference between the two are not statistically significant. The ‘additive’ contribution of (L2) glosses thus seems very limited. This in turn entails that illustration appears to be a more effective comprehension support compared with (L1 and L2) glosses; but its relative effect in comparison to other macro-level supports such as infographics and photos are yet to be established.
18
The above review shows that researchers have loosely used the generic term “picture” to refer to either graphic “illustration” (e.g., Al-Seghayer, 2001) or “photo”, i.e., image made using a camera (e.g., Chun & Plass, 1996; Yanguas, 2009). In this regard, photos and illustrations are both subsumed under the same (i.e., “picture”) multimedia support category. Because of this conceptual or methodological conflation, the net and relative effects of photos and illustrations remain unclear. It is likely that illustrations and photos may invoke very different underlying input processing profiles. What further exacerbates the picture regarding the relative effects of illustrations and photos is that the efficacy of glosses and ‘pictures’ (referring to either illustrations or photos) are typically examined in combination with other supports (e.g., gloss + illustration or gloss + video). Consequently, findings of existing studies do not clearly reveal the relative contribution of glosses, illustrations and photos.
Last but not least, with the exception of Author (2015), ‘pictures’ (either photos or illustration) are generally operationalized by researchers as micro-level comprehension supports (e.g., pictorial gloss) to assist learners’ understanding of novel L2 vocabulary, rather than as macro-level supports to aid learners’ global understanding, as seen in many learning materials designed for adolescent or adult L2 learners. Due to this ecological validity issue in research, the efficacy of photos and illustrations as macro-level supports – a common practice in actual L2 learning materials – is yet to be established. The following section will review research on using multimedia supports for promoting reading comprehension.