Effects of caption modes on L2 learners with higher working

CHAPTER 5 DISCUSSION

5.2 RQ 2: Does working memory capacity modulate the effect of captioning?

5.2.2 Effects of caption modes on L2 learners with higher working

Contrary to the findings on lower working memory L2 learners, the interaction between the effect of captioning and modality predilections became salient in the listening performances of L2 learners with higher working memory capacity. Both auditory and visual learners with higher working memory performed the best under their most preferable input combinations. In other words, the visual learners significantly benefited from the captioned video; while the auditory learners performed most ideally without captions. The fact that the effect of captions was determined by their idiosyncratic modality preference again, reinforced Mayer’s active processing assumption in CTML: learners tend to rely on their most preferred input for the most desirable learning outcomes (Mayer, 2001, 2005a). This match-making phenomenon

generated the most optimal results for both types of learners with higher (i.e., average and/or above average) working memory capacity in this study, which again verified the hypothesis proposed in Oxford (2003). Specifically, the quantitative results obtained from the auditory learners with higher working memory capacity concurred with the findings discussed in the earlier paragraphs, as the presence of captions may have disrupted their listening comprehension, which was evidenced in their listening scores.

As for their visual counterparts, full textual support led to a more advantageous listening outcome—a finding contrary to what was observed in the auditory learners.

Such effect may be attributed to them reporting paying more attention to their most preferred sources of input, visualized text, rather than the aural stream—a match that generated the best multimodal input processing scenario for them. Aside from the advantage for immersing in their most preferred processing environment, having higher attentional control could also explain the visual learners’ achievement. Based on the qualitative data, some of them reported that captions were not distracting or cognitively demanding. They also self-assessed as more capable of attending to the multimodal input, which allowed them to dexterously utilize any given input during online processing (Colflesh & Conway, 2007). More control in processing implies more use

of captions to enhance form-meaning mapping that aids listening comprehension (Chai

& Erlam, 2008; Danan, 2004; Markham & Peter, 2003; Sydorenko, 2010; Taylor, 2005;

Vanderplank, 2010; Winke et al., 2010). Therefore, it is reasonable to suggest that higher working memory enabled the visual learners to efficiently use captions to their advantage, and consequently generated deeper input processing that led to a more optimal listening outcome.

6. CHAPTER 6 CONCLUSIONS 6.1 Conclusion and implications

This study showed that the effect of captions was significantly affected by intermediate-to-high intermediate L2 learners’ input processing profiles, namely modality preference and working memory capacity. Specifically, among the two variables investigated in this study, working memory capacity determined the effect of modality preference in regards to the extent to which L2 learners benefited from captions. While the effect of captions was found absent in the lower working memory L2 learners regardless of their modality preferences, it was evident in their higher working memory counterparts. Among L2 learners who have higher working memory

capacity, captions were significantly detrimental to the auditory learners and notably facilitative to the visual learners’ listening comprehension. Consequently, the findings

from this study jointly reinforced the importance to consider individual factors—highly related to L2 multimodal input processing—when providing multimedia support in differentiated teaching/learning settings.

In light of the findings of this study, the following specific suggestions are offered regarding the use of captions as a potential listening comprehension aid for L2 learners who are intermediate-to-high intermediate proficiency level. Frist, language instructors

should be sensitive to L2 learners’ input processing profiles when using captions to

enhance their understanding of the audiovisual input. As evidenced in this study, not all L2 learners benefited from captions equally. Thus, without considering L2 learners’

input processing profiles, the effect of captions in L2 listening comprehension may not be maximized according to different individual needs.

6.1.1 Differentiated instructions for optimal implementations of captions in multimodal

listening

As noted earlier, intermediate-to-high intermediate L2 learners benefited from captions the most when their input processing factors were taken into consideration.

Since individual factors were found highly crucial in determining the effect of captions, language instructors are encouraged to use the assessments of modality preference and working memory capacity (i.e., CRT and Reading Span Test) to diagnose L2 learners prior to exposing them to multimodal learning environment.

As evidenced in this study, those with average or above-average working memory capacity should be more aware of their real-time modality preference when using captioned video. In this group of L2 learners, captions are not recommended for those who prefer to listen, but are highly encouraged for those who are visually inclined. For the auditory L2 learners with better working memory capacity, videos with captions

may not be a desirable listening environment for the presence of captions may impose extra load to their information processing. They may also be advised that processing preference plays a huge role in affecting their general understanding when receiving multimodal input. Aligning with this principle, their visual counterparts may be informed that due to having more cognitive/attentional capacity in utilizing visual input, they are more likely to make better use of captions to assist their multimodal comprehension.

As for L2 learners with below-average working memory capacity, they should be informed that their modality preference does not affect the extent to which they benefit from captions. Also, captions may not provide overt assistance to their listening

comprehension. Nonetheless, qualitative data in this study showed that there is an attitudinal discrepancy between the visual and the auditory learners’ listening

preference. With the overall results suggesting that the visual learners are more prone-caption and auditory learners are more resistant towards prone-captions, this may be a referential indication for lower working memory L2 learners when exposed to multimodal listening environment, which still warrants more empirical evidence.

The above differentiated instruction may be more suitable for listening practice beyond the classroom settings, for L2 learners, in particular EFL learners, tend to use videos to extend their exposure to native oral input after class. However, this does not

mean that such a differentiated instruction has no practical place in the classroom setting. As shown in the participants’ qualitative comments, visual and auditory

learners use their focal attention for different targets in multimodal input. With this differences in mind, instructors can make good use of this point in a dictogloss dictation activity. Specifically, language instructors can pair up visual and auditory learners, inviting them to view the video materials first without captions and then immediately review the same video with captions; in both viewings, all learners have to take notes.

Note that initial video viewing should not include captions so that L2 learners － irrespective of their modality preference and working memory capacity－focus only on the essence of the content without being cognitively loaded. Next, both visual learners and auditory learners would view the video again, but this time with captions. In this review, the visual learners can either focus on the captions or resprocess other visual content in the video, while auditory learners, having had the chance to process the meaning of the video once without the distraction of captions, might not find the captions distracting or disruptive － a pedagogical tip to offload the constraint of working memory capacity for auditory learners. the distinctive processing preferences of the visual and auditory learners may make their notes complementary to each other, thereby optimizing the effect of the dictogloss activity.

6.1.2 The implementation of CRT and RST for pedagogical purposes

Importantly, both CRT and RST—instruments that were originally designed as research tools—can be adapted and used as pedagogical tools by language instructors,

L2 learners, or even online language learning developers. In the present study, both CRT and RST were crucial assessments that were used to determine L2 learners’

real-time preferred modality and working memory capacity, respectively. Specifically, CRT, which takes the form of a multiple-choice comprehension test (a common assessment

tool in the language classroom), can be easily adapted for diagnostic purposes (i.e., assessing L2 learners’ preferred modality) as in the case of this study, Leveridge and Yang’s (2013a, 2013b) studies and Liu and Todd’s (2014) study.

The RST, on the other hand, is a cognitive assessment that can be easily adapted into a pedagogical/diagnostic tool for language teachers. First, the making of RST is not technically demanding. It can be made and presented on Power Point Presentations.

Also, RST can be implemented on an individual or a group of participants. For L2 individuals with self-learning in mind, they can complete the assessment online on their own; for language instructors and researchers, RST can be implemented in a classroom or a laboratory setting.

Another added benefit of CRT and RST is that it can be incorporated into popular online language learning websites for individualized learning purposes. VoiceTube, for

example, is a popular online learning platform that offers the assistance of captions. In light of the major findings of this study, software developers from these websites can consider creating a series of assessments, with an aim to personalize multimodal learning. This way, L2 learners are able to not only understand their input processing profiles, but also find the most optimal learning path to maximize their listening comprehension.

6.2 Future suggestions and limitations

Despite the above insights, several limitations are identified for future references.

First, conclusions were drawn from small sample size in each learning conditions. For researchers interested in replicating this study, a larger sample should be used to generalize the findings in this study. Second, although the number of trials in the RST in this study is adequate (Friedman & Miyake, 2004), it can be increased to maximize the reliability of the span scores. Nevertheless, it should be noted that the length of RST should be tailored to avoid causing attention fatigue from the participants. Third, because this study focused on listening comprehension, the findings may not shed light on other long-term aspects of learning, such as listening acquisition. Whether individual factors also modulate the effect of captions on different aspects of L2 listening acquisition still warrants more future investigations.

References

Al-Shehri, S., & Gitsaki, C. (2010). Online reading: A preliminary study of the impact of integrated and split-attention formats on L2 students’ cognitive load. ReCALL, 22(3): 356–375.

Baddeley, A. D. (2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 4, 829–839.

Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 47–89). San Diego, CA: Academic Press.

Bird, S.A., & Williams, J.N. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling.

Applied Psycholinguistics, 23(4), 509–533.

Bloomfield, A., Wayland, S. C., Rhoades, E., Blodgett, A., Linck, J., & Ross, S. (2010).

What makes listening difficult? Factors affecting second language listening comprehension. College Park, MD: University of Maryland, Center for Advanced Study of Language.

Chai, J., & Erlam, R. (2008). The effect and the influence of the use of video and captions on second language learning. New Zealand Studies in Applied Linguistics, 14(2), 25–44.

Chang, C. C., Tseng, K. H., & Tseng, J. S. (2011). Is single or dual channel with different English proficiencies better for English listening comprehension cognitive load and attitude in ubiquitous learning environment? Computers &

Education, 57(4), 2313–2321.

Colflesh, G. J., & Conway, A. R. (2007). Individual differences in working memory capacity and divided attention in dichotic listening. Psychonomic Bulletin &

Review, 14(4), 699–703.

Danan, M. (2004). Captioning and subtitling: Undervalued language learning strategies.

Meta, 49(1): 67–77.

Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning & Verbal Behavior, 19, 450–466.

Engle, R. W., Cantor, J., & Carullo, J. J. (1992). Individual differences in working memory and comprehension: A test of four hypotheses. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 972–992.

Engle, R.W., Kane, M.J., & Tuholski, S.W. (1999). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and

executive control (pp. 102–134). New York: Cambridge University Press.

Friedman, N. P., & Miyake, A. (2004). The reading span test and its predictive power for reading comprehension ability. Journal of Memory & Language, 51, 136–

158.

Garza, T. J. (1991). Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24(3), 239–258.

Gebhard, J. (2000). Teaching English as a foreign or second language: A teacher self -development and methodology Guide. United States of America: The University of Michigan Press.

Goh, C. C. M. (2000). A cognitive perspective on language learners’ listening comprehension problems. System, 28, 55–75.

Gruba, P. (2004). Understanding digitized second language videotext. Computer Assisted Language Learning, 17(1), 51–82.

Guichon, N., & Mclornan, S. (2008). The effects of multimodality on L2 learners:

Implications for CALL resource design. System, 36(1), 85–93.

Hasler, B. S., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional animation. Applied Cognitive Psychology, 21(6), 713–729.

Hayati, A., & Mohmedi, F. (2011). The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of Educational Technology, 42(1), 181–192.

Huang, H.C., & Eskey, D. E. (1999). The effects of closed-captioned television on the listening comprehension of intermediate English as a second language (ESL) students. Journal of Educational Technology Systems, 28(1), 75–96.

Jong, T. D. (2010). Cognitive load theory, educational research, and instructional design:

Some food for thought. Instructional Science, 38(2), 105–134.

Leveridge, A. N. (2015). The removal of target language captioning supports. In Turel, V. (Ed), Intelligent design of interactive multimedia listening software (pp. 75–

101). Hershey: Information Science Reference.

Leveridge, A. N., & Yang, J. C. (2012). Effect of medium: A conceptual framework for the removal of supporting captions for EFL listening comprehension in multimedia instructional delivery. In: 15th International CALL Research Conference: Proceedings.Taiwan: Taichung.

Leveridge, A. N., & Yang, J. C. (2013a). Learner perceptions of reliance on captions in EFL multimedia listening comprehension. Computer Assisted Language Learning, 27(6), 545-559.

Leveridge, A. N., & Yang, J. C. (2013b). Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL, 25(02), 199-214.

Liu, Y. T., & Todd, A. G. (2014). Dual-modality input in repeated reading for foreign language learners with different learning styles. Foreign Language Annals, 47(4), 684–706.

Low, R., & Sweller, J. (2005). The modality principle in multimedia learning. In R. E.

Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 147–158).

New York: Cambridge University Press.

Lwo, L., & Lin, M. C. T. (2012). The effects of captions in teenagers’ multimedia L2 learning. ReCALL, 24(2), 188–208.

Markham, P. (1989). The effects of captioned television videotapes on the listening comprehension of beginning, intermediate and advanced ESL students.

Educational Technology, 29, 38–41.

Markham, P. L., & Peter, L. (2003). The influence of English language and Spanish language captions on foreign language listening/reading comprehension.

Journal of Educational Technology Systems, 31(3), 331–341.

Markham, P., Peter, L., & McCarthy, T. (2001). The effects of native language vs. target language captions on foreign language students’ DVD video comprehension.

Foreign Language Annals, 34(5), 439–445.

Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The SOI model for guiding three cognitive processes in knowledge construction.

Educational Psychology Review, 8(4), 357–371.

Mayer, R. E. (1999). Research-based principles for the design of instructional messages:

The case of multimedia explanations. Document Design Journal of Research and Problem Solving in Organizational Communication, 1(1), 7–19.

Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.

Mayer, R. E. (2005a). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 31–48). New York:

Cambridge University Press.

Mayer, R. E. (2005b). Principles for managing essential processing multimedia learning:

Segmenting, pretraining, and modality principles. In R. E. Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 169–182). New York:

Cambridge University Press.

Mayer, R. E. (2010). Applying the science of learning to medical education. Medical Education, 44: 543–549.

Miyake, A., & Friedman, D. (1998). Individual differences in second language proficiency: Working memory as language aptitude. In A. F. Healy & L. E.

Bourne, Jr (Eds.), Foreign language learning: Psycholinguistic studies on training and retention (pp. 339–364). Mahwah, NJ: Erlbaum.

Mirzaei, M. S., Meshgi, K., Akita, Y., & Kawahara, T. (2017). Partial and synchronized

captioning: A new tool to assist learners in developing second language listening skill. ReCALL, 29(02), 178–199.

Moreno, R. (2006). Optimizing learning from animations by minimizing cognitive load:

Cognitive and affective consequences of signaling and segmentation methods.

Applied Cognitive Psychology, 21, 1–17. listening and vocabulary learning: A meta-analysis. System, 41(3), 720–739.

Pujola, J.T. (2002). CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL,14(2), 235–262.

Reid, J. (1995). Learning styles in the ESL/EFL classroom. Boston: Heinle & Heinle.

Révész, A. (2012). Working memory and the observed effectiveness of recasts on different L2 outcome measures. Language Learning, 62(1), 93–132.

Scarcella, R. & Oxford, R. (1992). The tapestry of language learning: The individual in the communicative classroom. Boston: Heinle & Heinle.

Stewart, M. A., & Pertusa, I. (2004). Gains to language learners from viewing target language closed-captioned films. Foreign Language Annals, 37(3), 438–442.

Sun, Y., & Dong, Q. (2004). An experiment on supporting children’s English vocabulary learning in multimedia context. Computer Assisted Language Learning, 17(2), 131–147.

Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning and Technology, 14, 50–73.

Taylor, G. (2005). Perceived processing strategies of students watching captioned video.

Foreign Language Annals, 38(3), 422–427.

Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent?

Journal of Memory and Language, 28(2), 127–154.

Vandergrift, L. (2004). Listening to learn or learning to listen? Annual Review of Applied Linguistics, 24, 3–25.

Vanderplank, R. (2010). Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching, 43(1), 1–37.

Vigliocco, G., Perniss, P., & Vinson, D. (2014). Language as a multimodal phenomenon:

Implications for language learning, processing and evolution. Philos Trans R Soc Lond B Biol Sci, 369, 1–7.

Winke, P., Gass, S., & Sydorenko, T. (2010). The effects of captioning videos used for foreign language listening activities. Language Learning & Technology, 14(1),

65–86.

Winke, P., Gass, S., & Sydorenko, T. (2013). Factors influencing the use of captions by foreign language learners: An eye‐tracking study. The Modern Language Journal, 97(1), 254–275.

Appendix 1 Information Sheet and Consent Form STUDY INFORMATION SHEET:

I am a Master’s student at the Department of English, National Taiwan Normal University (NTNU) interested in second language listening and speaking. I would like to invite you to participate in a research study that examines how different online processing profiles modulate the effect of captioning on second language listening comprehension.

If you decide to participate, I will ask you to take part in two 1-hour sessions in a computer laboratory at NTNU. In the 1^st session, you will be asked to take a caption reliance test and a working memory test. In the 2^nd session, you will first be asked to view a caption/no caption video, and then take a listening comprehension test. A questionnaire will be administered to you at the end of the 2^nd session.

In return for your participation, I am able to offer you NT 100 at the end of the 2^nd session. As soon as the data analysis is completed, I will also share the overall results of the study with you.

Any data obtained from you will be kept securely. At every stage of the project and beyond, your name will remain confidential. Your identity will be anonymized by the use of a unique identifier. The overall results of the study will be used for my maters research project and not be shared with others. The results will also be presented at professional conferences and in research publications.

You are free to withdraw from the study at any time without reason and without any impact on you. If you decide to withdraw, any data collected from you will be destroyed. If you have any queries about the study, please feel free to contact Emily Kam at [email protected], 0988103504.

I would be very grateful if you would agree to take part!

Emily Kam

Master’s student at the Department of English, NTNU.

Consent Form

1. I have read and had explained to me by Emily Kam the Information Sheet relating to this project.

Yes No

□ □

2. I have had explained to me the purposes of the project and what will be required of me, and any questions have been answered to

在文檔中人人都需要 “字幕” 嗎?探討不同學習行為如何影響字幕對英語聽力理解之效度 (頁 76-0)