DISCUSSION
5.1 Research Question 1: Overall effects of different caption modes
The first research question aimed to explore whether caption presentation mode affects L2 learners’ listening comprehension when L2 learners’ modality preferences are not considered. The quantitative results showed that when individual differences are not considered, the participants’ comprehension performances under the four caption conditions (i.e., no, full, partial, and real-time captioning) were comparable.
The analysis indicated that neither the presence nor the presentation mode of captions significantly influenced L2 learners’ comprehension performance. To unravel the reasons behind such a finding, these two aspects (i.e., the presence and presentation mode of captions) will be discussed in more depth in the ensuing paragraphs.
The current study found that when L2 learners’ modality preferences are not considered, the presence or absence of captions exerted little impact on L2 learners’
comprehension as no insignificant difference was observed between the no and full caption conditions. This finding, however, contradicts most existing studies which found full captioning helpful in aiding L2 learners’ comprehension (e.g., Guichon &
Mcloran, 2008; Huang & Eskey, 1999; Perez et al., 2013; Winke et al., 2010). This discrepancy may be attributed to differences in the participants’ proficiency profile between existing research and the current study; namely, while extant research mainly targeted intermediate L2 learners, the current study focused on high-intermediate L2 learners as participants. It is thus reasonable to extrapolate that L2 learners’ proficiency may modulate the extent to which they may benefit from captioning—a view suggested by Chen, Liu, and Todd (2018). Precisely, the current finding suggested that full captions might not be as beneficial for higher-proficiency (i.e., high-intermediate) L2 learners as for their less proficient (i.e., intermediate) counterparts in promoting their
comprehension. Since higher level L2 learners tend to be more sophisticated in decoding rapid speech as well as utilizing strategies to assist their L2 listening comprehension, their reliance on full captions for comprehension would not be as much as lower level L2 learners. It is possible that as L2 learners’ proficiency increases, the aid provided by full captions becomes gradually insignificant. This finding, though conflicting with a few existing studies (Garza, 1991; Taylor, 2005), is in accordance with Leveridge and Yang’s (2013) discovery that “the more a learner progresses, the less they are reliant on (full) captions as a learning support” (p. 211). It also echoes Lwo and Lin’s (2012) claim that captions may not facilitate L2 learners with different profiles and, meanwhile, underscores the importance of considering L2 learners’
proficiency level when investigating and implementing various types of captions as listening comprehension aids.
In addition to the lack of difference between full captioning and no captioning in promoting L2 comprehension, this study found that partial and full captioning helped participants achieve similar performance. The lack of significant difference between these two caption conditions corroborates the results of most existing partial caption studies (Gillory, 1998; Hsu et al., 2013; Mirzaei et al., 2017; Perez et al., 2014; Perez et al., 2014). While some partial caption studies attributed the lack of difference to inappropriate difficulty level of the experimental materials (Perez et al., 2014; Perez et al., 2014), some regarded it as an endorsement of partial captioning as it helped L2 learners perform equally well with limited textual support (Gillory, 1998; Mirzaei et al., 2017). In the current study, aside from the proficiency issue mentioned above, L2 learners’ modality preference was identified as another factor contributing to the result.
Last but not least, real-time captioning, which has not been investigated as an L2 listening comprehension aid in the existing literature, also led to similar performance to that under the full caption condition. While both full and real-time captioning offer
the same amount of textual transcription of the oral text, the transcription is presented to L2 learners in different ways: static vs. incremental/dynamic display. However, this display difference, as shown by the finding of this study, did not lead to significant differences in the two captions’ efficacy in promoting the participants’ comprehension.
Since full captioning has been proved to be an effective L2 comprehension aid, the comparable performance under the full and real-time caption conditions entails that real-time captioning neither optimized nor impaired the effects of full captioning on high-intermediate L2 learners. The questionnaire data also corroborated this view: both full and real-time captioning did not cause distractions nor cognitive overload to the participants’ viewing experience.
Based on the aforementioned findings, it appears to suggest that the presence or display modes of captions exerted little impact on L2 learners’ comprehension and that captions are not a necessary comprehension aid. It is important to note that such a view should be interpreted with caveat. As will be seen in the discussion in subsection 5.2, the efficacy is significantly modulated by L2 learners’ modality preference.
Differentiated (video) input is warranted for optimal comprehension outcomes.
5.2 Research Question 2: Effects of L2 learners’ modality preference under different caption modes
The second research question is concerned with whether L2 learners’ modality preference in real-time input processing affects their comprehension under different caption conditions. The insight into this question sheds light on whether L2 learners’
modality preference modulates the efficacy of various caption modes in promoting L2 comprehension. The statistical interaction observed between caption mode and modality preference revealed that individual L2 processing profile did play a significant role in determining the efficacy of captions. L2 learners’ modality preference alone also
led to substantial difference in L2 listening comprehension, with auditory learners outperforming their visual counterparts. These two sets of data support the researcher’s hypothesis that L2 learners’ modality predilection might serve as a key factor modulating the efficacy or usability of captions.
In particular, among the four caption conditions, the modulating effect of L2 learners’ modality preference is particularly salient under the partial caption condition, where auditory learners performed significantly better than visual learners. This provides a possible explanation for the insignificant difference between full and partial captioning observed in many existing studies and the current study (Gillory, 1998; Hsu et al., 2013; Mirzaei et al., 2017; Perez et al., 2014; Perez et al., 2014), indicating that different caption modes selectively benefit learners of different L2 modality preferences.
It also confirms previous eye-tracking results that L2 learners’ viewing behaviors, which are a reflection of their individual input preferences, are highly sensitive to the alteration of input presentation (Choi, 2016; Perez, Peters, & Desmet, 2015). Since L2 learners’ modality preference has not yet been explored in studies exploring different caption displays, subsections 5.2.1 & 5.2.2 will provide an in-depth discussion on the selective beneficial effects of different caption modes on visual and auditory L2 learners.
5.2.1 Effects of different caption modes for visual L2 learners
While visual L2 learners under the full caption condition marginally outperformed those under no and real-time caption conditions, they significantly surpassed those under the partial caption condition. This suggests that visual L2 learners benefited the most from full captioning among the four caption modes. Existing studies suggested that full captioning helps L2 learners better parse the aural input through phonological visualization (Vanderplank, 1988; Guichon & Mcloran, 2008), which, according to the current finding, is especially true for visual L2 learners. Owing to the abundant textual
support provided by full captions, visual L2 learners were better able to keep up with the on-going speech stream and understand the video content more thoroughly.
Similarly, probably as a result of visual learners’ inclination to process visual input, real-time caption—where full transcription of oral text is provided—also significantly assisted their comprehension, albeit to a lesser extent compared with their comprehension under the full caption condition. Unlike full captioning, the transient verbatim caption exhibition in real-time captioning might have caused slight disruption in visual L2 learners’ input processing, which slightly dampened their comprehension.
Such a disruption possibly stemmed from the participants’ unfamiliarity with real-time caption mode as most of them stated in the interview that they had never experienced such captioning before. Even so, real-time captioning still proved to be almost as beneficial as full captioning for visual L2 learners’ listening comprehension, allowing them to have better comprehension than those assigned to the partial caption mode.
While full and real-time captions serve as the optimal listening comprehension scaffold for visual L2 learners, partial captions failed to assist visual L2 learners’
listening comprehension to similar extent. This implies that reducing the amount of textual support (as in the case of partial captioning) could dampen the beneficial effects of captioning on visual L2 learners. Since visual L2 learners rely more on visual and textual cues, the unpredictable nature of partial caption display might cause distractions which prevent them from fully attending to neither the images nor the captions. This finding echoes Perez et al.’s (2013) qualitative data revealing L2 learners’ negative perception of partial caption due to its distracting and abrupt nature.
Accordingly, the observation that L2 visual learners’ performance under full and real-time caption conditions were significantly better than that under the partial caption mode indicated that full, rather than partial, visual transcription of the oral text seemed to be more beneficial to L2 visual learners. This also entails that when caption display
mode is an option and caption is being turned on and displayed, full transcription of oral text—irrespective of static full caption mode or dynamic real-time caption mode—
offers visual L2 learners desirable comprehension support. In particular, static full captions provide the optimal comprehension scaffold to visual learners.
Notwithstanding, one interesting finding stands up and begs for further explanation. While full transcription of oral text proved to be useful in enhancing visual L2 learners’ comprehension, this study also found that visual L2 learners assigned to the no caption condition performed similarly well with those under full and real-time caption conditions. This finding, which appears to contradict the aforementioned account for the prominent role of captions for visual learners, begs for further explanation. It is possible that when captions were not available, visual L2 learners, due to their keen preference to visual input, would pay more attention to other available subtle paralinguistic or nonlinguistic clues (e.g., facial expression, hand gestures) that may not be attended to by other non-visual L2 learners. This explains why L2 visual learners, who seemed to rely more on captions, were still able to understand the video content even without the scaffold of captioning and exhibited comparable comprehension performance without the presence of captions.
Thus, it can be concluded that the optimal captioning viewing environments for visual L2 learners are the ones where full or real-time captions are provided. Or, alternatively, it is advised that visual L2 learners watch non-captioned videos for optimal comprehension outcomes by attending closely to visual input other than captioning. Partial captioning is the least effective aids for visual L2 learners’
comprehension due to its limited amount of text and its irregular presentation.
5.2.2 Effects of different caption modes for auditory L2 learners
On the other hand, while auditory L2 learners under the partial caption condition
scored slightly higher than those under the no caption condition, they significantly outperformed those under full and real-time caption conditions. Different from what has been revealed for visual L2 learners, partial captioning, which provided relatively less textual support than full and real-time captioning, appeared to be the most facilitative caption mode for auditory L2 learners’ comprehension. Based on existing L2 caption research, partial captions help reduce L2 leaners’ cognitive load, allow them to process the auditory input more attentively, and thereby assist their comprehension (Behroozizad & Majidi, 2015). However, according to the current finding, this can only be applied to auditory L2 learners. For auditory L2 learners, selective transcription of oral text (i.e., partial caption) give auditory learners’ just the ‘right’ amount of comprehension scaffold, without bombarding them with too much visual support that is not desired by them. The selective textual scaffold under the partial caption condition thus gave auditory L2 learners the optimal viewing environment.
Interestingly, despite auditory L2 learners’ optimal comprehension performance under the partial caption condition, they did not express positive attitude toward this caption mode. In fact, based on the qualitative data, auditory L2 learners reported that they focused more on the audio input and they considered partial captioning rather distracting. The Likert-scale data shows that auditory L2 learners rated 4.57 (out of the highest possible rating of 5 indicating “strongly agree”) on the statement “I focused more on the audio than the captions while watching the video” and 3.71 on “I think the captions in the video were distracting.” The discrepancy between the auditory learners’
(positive) comprehension performance and their (negative) oral report implies that although they found this attentional re-allocation distracting, auditory L2 learners’
attention was indeed re-oriented to attend to the selected keywords shown on the screen and this indeed optimized their comprehension—a view also shared by Guillory (1998).
Notwithstanding, auditory L2 learners’ negative mindset toward this ‘optimal’
captioning viewing environment also indicated that partial captioning should be carefully implemented for auditory L2 learners. Specifically, although partial captioning may optimize auditory L2 learners' comprehension of video content, their negative perception toward partial captioning may not foster the same desirable outcomes for other language domains (e.g., retention of the video content, vocabulary acquisition).
Next, it is observed that auditory L2 learners’ comprehension performance under the no caption condition, where the least amount of visual (i.e., textual) support was provided, did not significantly differ from that under the partial caption condition. This indicates that videos without captions might be an alternative choice for auditory L2 learners, especially when L2 instructors are constrained in time to produce partial captioning as L2 comprehension aids. While videos without captions are easily accessible, the production of partial captioning requires professional judgement and caption editing. Owing to the relatively time-consuming production of partial captioning, presenting videos without captions serve as an alternative to facilitate auditory L2 learners’ comprehension of multimodal materials when partial captioning is not available.
On the contrary, presenting auditory learners with full transcription of oral text under the full and real-time caption conditions debilitated, rather than facilitated, their comprehension outcomes—a finding contrary to what was observed in visual L2 learners. Whereas more textual support (i.e., full and real-time captioning) provides more scaffold for visual L2 learners, it is likely to cause more challenges for auditory L2 learners. Possible explanation can be drawn from Mayer’s (2005) assumption that humans are limited in their capacity of information processing. For auditory L2 learners, excessive visual/textual input presented in L2 multimodal materials might increase their cognitive load and thereby disrupt their information processing. This finding again
lends empirical support to the need to consider L2 learners’ preferred input modality when viewing captioned videos. However, more empirical evidence is warranted to shed light on whether less proficient auditory L2 learners would find it more challenging to attend to full or real-time captioned videos as well. Owing to their unsophisticated aural decoding skills, less proficient auditory L2 learners might benefit more under the full or real-time caption than partial caption condition.
To briefly conclude this chapter, different caption presentations did not seem to significantly affect high-intermediate L2 learners’ listening comprehension. Due to their adequate listening skills in the L2, they could understand the gist of the video content without relying too much on the ‘extra’ scaffold provided by different forms of captions. However, this across-the-board finding actually was challenged when L2 learners’ modality preference is considered; each caption mode was found to facilitate visual and auditory L2 learners to different extents. While visual L2 learners benefited more from full and real-time captioning and less from partial captioning, auditory L2 learners reacted to these two caption modes in the opposite way; auditory learners actually had the optimal comprehension outcomes when receiving limited (selective) captioning support, namely partial captioning. These findings call for the need to consider L2 learners’ individual profiles (modality preference) while choosing multimodal instructional or learning materials, especially in the context of differentiated teaching/learning setting.