RQ1: Does preferred modality modulate the effect of captioning?

The first research question concerns whether participants’ preferred modality (i.e., visual and auditory) would modulate the effect of captioning on L2 learners’ listening comprehension. Statistical results revealed that this input processing factor significantly modulated the effect of captioning. It was also evidenced that modality preference alone resulted in statistically significant difference in the L2 learners’ performance data. Both findings lent support to the hypothesis proposed in chapter 2, that modality predilections, as an input processing factor, could potentially modulate the extent to which the effect of captions is determined. Since preferred modality is theoretically and empirically pertinent to processing multimodal input, the following section will discuss

60

the finding vis-à-vis both Mayer’s CTML and existing empirical evidence from L2 caption research.

5.1.1 Effects of caption mode on auditory learners

The quantitative data revealed that L2 learners who were prescribed as auditory learners performed the best under the no caption condition. When compared to their captioned counterparts, the no-caption listening advantage became even more statistically salient. Therefore, it became clear that L2 auditory learners understood the audiovisual information best when immersed in no-captioned multimodal environment.

The prominent no-captioned effect manifested in the auditory learners may be accounted for by Mayer’s active processing assumption in CTML. In theory, multimodal processing requires efficient learner control in selecting the most relevant input to aid comprehension (Hasler, Kersten, & Sweller, 2007; Mayer, 2001). Similarly, L2 learning context also requires learners to the same—be efficient in allocating attention to cues that are directly beneficial to listening comprehension (Taylor, 2005).

In this case, L2 learners are most likely to depend on the input that matches their modality preference. Furthermore, Oxford (2003) hypothesized that in order to generate the most desirable learning outcome, what is preferred has to match with what is presented (Oxford, 2003). As the auditory learners in this study were found to be a case

61

in point, it is reasonable to extrapolate that the no-caption condition may be a more

favorable environment for them to optimize their multimodal listening outcome.

The observation that having no caption appears to be the optimal listening condition for the auditory learners indicates that visual support (in this case, any form of captions) disrupted, rather than facilitated auditory learners’ understanding of multimodal video content. This explains why their comprehension was impaired when captions were provided. This finding contradicts with the previous studies, where captions were found to aid the listening comprehension process (Chai & Erlam, 2008; Danan, 2004;

Markham & Peter, 2003; Sydorenko, 2010; Taylor, 2005; Vanderplank, 2010; Winke et al., 2010). In fact, prior research maintained that captions were widely used as a strategy to facilitate perceptual processing in listening (Goh, 2000). If captions were designed to facilitate the encoding of the acoustic message, why is it detrimental to the auditory learners’ listening comprehension?

One possible explanation may lie in the disturbance of L2 learners’ optimal processing channel. In the case of the auditory learners, their preferred processing channel is through what they hear rather than what they see. This speculation is established by the qualitative data reported earlier; namely when exposed to captioned video viewing environment, the presence of captions may have reoriented their attention to the visual input—the channel that is not of their processing preference.

62

Moreover, according to the questionnaire data, the auditory learners indicated that presence of captions may have divided the auditory learners’ attentional resources. With limited attentional resources at hand (Mayer, 2001), it is possible that such demand from captions imposed difficulty in processing information from their optimal and most preferred channel. Furthermore, two thirds of the auditory learners did not express positive attitude towards receiving captions during their video-watching. Both sets of data sources, quantitative and qualitative, jointly indicated that the presence of captions may not lead to the optimal multimodal processing environment for the auditory learners. This finding, in turn, helped shed light on recalibrating the listening strategy suitable for those who preferred to “listen” in a multimodally enhanced environment.

5.1.2 Effects of caption modes on visual learners

Notwithstanding the notable effect captions had on the auditory learners, it did not manifest in the visual learners’ listening outcome. This study found that the listening scores from the visual leaners in the caption condition did not differ significantly from those in the no caption condition. Without the assistance of captions, visual learners were able to achieve similar level of understanding as those assisted with the visualized text. This illuminating finding indicated that for visual learners, captions may not be the only scaffold facilitative to their listening comprehension.

63

Since visual learners tend to rely on what they see, it is possible that even without captions, they would still rely on other available elements presented visually.

Nonlinguistic (e.g., facial expressions, body language etc.) input, for example, may contain subtle but rich contextual cues that may provide assistance equal to the effect of captions. As evidenced in Sueyoshi and Hardison’s (2005) study, paralinguistic elements, such as gestures and facial cues, can assist the understanding of videotaped lectures—a type of video content similar to the TED video used in this study. This may explain that without captions, L2 visual learners were still able to comprehend at comparable depth as their captioned counterparts in a multimodal listening environment.

Aside from the comparable performances between L2 visual learners under full and no caption conditions, closer examination at the standard deviation revealed a visible variance among the visual learners’ listening outcome. Such variance was also manifest in the qualitative records. Unlike the auditory learners whose qualitative data exhibited more consistency in rejecting the effect of captions, the data from the visual learners showed that they perceived captions in a positive lens: while some denounced the benefits of captions, others held a more positive mindset towards the online visual aid. Specifically, some reported that captions were (1) not distracting, (2) facilitative to their understanding of the novel words (see Winke et al., 2013), and (3) useful to resolve ambiguity which helped release their anxiety (see Vanderplank, 2010). Notably, of all

64

the participants in this study, only the visual learners reported that having captions on made them feel less cognitively demanded. The statistical variance along with the visual learners’ qualitative remarks posed a question as to whether the effect of captions was in fact, “selective” to L2 visual learners with other intra-learner factors.