The goal of the present study was to investigate the effectiveness of metacognitive listening strategy instruction. The study aimed to discover whether metacognitive strategy instruction improved EFL learners’ listening performance and reduced listening anxiety. In addition, the study investigated whether EFL learners’
awareness of strategy use improved after receiving metacognitive strategy instruction.
The following sections explain the design of the research, including the participants, the instruments, data collection procedure, and data analysis procedure.
Research Design
The design of the study was quasi-experimental, with an experimental group (with intervention) and a comparison group (without intervention). The duration of intervention lasted for eight weeks, from April to June, 2015. The experimental group received metacognitive strategy instruction and did listening comprehension exercises in the experiment. The comparison group received traditional listening instruction which merely provided listening comprehension exercises, without any instruction on listening strategies. The listening comprehension exercises the two groups did were the same. The materials used for the listening exercises were derived from the listening section of General English Proficiency Tests-Elementary Level compiled by Zhang and Gu (2013), which contained mock listening tests. As listening practice materials, six questions were played to the participants in the experimental and the comparison groups each week in the duration of experiment. A pilot study on the instruments and instructional procedures was done before the formal study.
The Pilot Study
In order to make sure the instruments and instructional procedures used in the present study are suitable, as well as to measure the time needed for the tests and the metacognitive strategy instruction, the researcher conducted the pilot study in March, 2015. Twenty- six students in the same junior high school as those in the formal study were chosen by convenient sampling. They were required to complete the listening tests and the questionnaires as soon as possible so that the researcher could measure how much time would be needed in the formal study. The students were also asked to give feedback on the wording of the items. In this way, the researcher could modify the ambiguous words or questions in the questionnaires. In addition, the procedures of one lesson’s metacognitive strategy instruction were gone through in the pilot study.
Thus, the researcher assured good pacing of the instruction in the formal study. The students were also encouraged to give feedback after receiving the instruction. The researcher modified the instructional plans based on the results of the pilot study.
Participants
The participants in the formal study were 56 eighth grade students (28 males and 28 females) in a junior high school in Taoyuan City, Taiwan. It’s almost impossible to randomly arrange individual students into any groups because of the management policies of schools; therefore, the participants were chosen from two intact classes, one as the experimental group (n = 28) and the other as the comparison group (n = 28) in the same junior high school, whose performance on monthly English listening comprehension tests did not differ significantly (t = 1.04, p = .30). The participants had learned English as a foreign language in school for at least five years and they had five English classes per week and the instructional time in each class period was 45 minutes. The traditional listening practices provided for the students in the school included listening practices in the workbook and textbook. In every monthly test, 20 to 30 marks out of 100 were allocated to listening comprehension questions.
Data Collection Procedures
One week before the experiment, both of the experimental and comparison groups took pre-tests to provide baseline data on the participants’ listening proficiency, anxiety level, and metacognitive awareness. After the eight-week intervention, both of the two groups took post-tests on listening proficiency, anxiety, and metacognitive awareness. The pre-tests and post-tests were mostly the same, except that the pre-tests contained one more questionnaire on the participant’s background and that the tests used to assess listening proficiency differed. The two sets of tests provided quantitative data for the researcher.
In the eight-week intervention, students in the experimental group were asked to write their reflections of learning on a listening worksheet, right after each period of strategy instruction. The worksheets provided qualitative data. Further qualitative data were provided by interviews with students that made greatest progress or regress in each of the two groups.
Instruments
The instruments used in the present study included two listening comprehension tests, a questionnaire that included the participants’ background information, Vandergrift, Goh, Mareschal, and Tafaghodtari’s (2006) Metacognitive Awareness
Listening Questionnaire (MALQ) translated by Li (2009), and Cheng’s (2014) Second Language Listening Anxiety Scale (SLLAS). In addition, semi-structural interviews and the worksheets which were used in the intervention also served as instruments for data collection in the present study.
The Listening Comprehension Tests
The listening tests applied in the pre-test and post-test of the present study were derived from the listening section of General English Proficiency Tests (Elementary Level) published by the Language Training and Testing Center in 2009 and 2011. The difficulty of these two tests was similar, and both of them contained 30 multiple-choice items. Each test took 20 minutes to finish.
The General English Proficiency Test (GEPT) is constructed by the Language Training and Testing Center in Taiwan and has been widely applied in Taiwan’s studies of language learning as the instrument for assessing participants’ English proficiency level. According to the Language Training and Testing Center (LTTC), the listening tests of GEPT Elementary Level have a high reliability (KR-20 = .80) (LTTC, 2008). In addition, a review of the GEPTs indicated that the reliabilities of the GEPTs are mostly in the high .8 range (Roever & Pan, 2008), which is similar to other standardized proficiency tests like TOEFL and IELTS.
The GEPTs contain five proficiency level tests: Elementary, Intermediate, High-Intermediate, Advanced, and Superior. According to the descriptions of the GEPTs, graduates of junior high schools with basic English proficiency can pass the elementary level of the GEPT, and the question types of the listening section of the GEPT are also similar to the listening comprehension questions of the Comprehensive Assessment (CA) for junior high school students. As a result, the researcher used the elementary level of GEPT to determine the participants’ listening proficiency. The listening comprehension section of the elementary level of the GEPT contains three parts, including picture description, response to the statement or question, and short conversation. In picture description section, listeners have to listen to the description of a picture and select the correct description for a picture printed on the test paper.
The second part is response to the statement or question. Listeners have to select a suitable response to a question. The third section is short conversation. Listeners listen to short conversations between people, and answer the questions based on their comprehension of the conversations.
Participants’ Background Questionnaire
The background questionnaire (See Appendix A) used in the study was adapted from Tsai (2010) and Hung (2010). The questionnaire included participants’ gender, exposure time to English per week, the length of studying English, whether the participants had stayed or lived in English-speaking countries, and whether the participants had passed the elementary level of GEPT. The questionnaire was written in Chinese so that the participants could have better understanding of the questions and their answers would be more reliable. This questionnaire was used only in the pre-test.
Metacognitive Awareness Listening Questionnaire (MALQ)
The Metacognitive Awareness Listening Questionnaire (MALQ, Appendix B) constructed by Vandergrift, Goh, Mareschal, and Tafaghodtari (2006) is a 6-point-Likert scale with 21 items. The MALQ is a self-report questionnaire and is constructed for researchers to assess second language learners’ metacognitive awareness and the use of metacognitive listening strategies. It was used as part of the pre-test and the post-test in the present study. To align with the response format of the listening anxiety scale, a 5-point-Likert scale was used instead of the 6-point scale.
The MALQ assesses five factors of listening strategies, including planning and evaluation, directed attention, person knowledge, mental translation, and problem-solving. Table 1 presents the categorization of the items in MALQ.
According to Li (2009), the Cronbach’s alpha reliabilities for the MALQ were acceptable, ranging from .68 to .78. The MALQ was originally written in English; it was translated into Chinese by Li (2009) to ensure the participants in her study could understand the questions. The Cronbach’s alpha reliabilities of the Chinese version as a whole was .90, and its split-half reliability for the first section was .83 and for the second section was .82 (Li, 2009). All of the values indicate that the Chinese version of the MALQ is reliable.
The participants’ responses to the MALQ were transformed into points and the scores they got were assumed to reflect their metacognitive awareness in listening.
Stronger agreement with the statements got more points (i.e. “strongly agree” equaled five points, “agree” equaled four points, “neutral” equaled three, “disagree” equaled two, and “strongly disagree” equaled one). But items 4, 8, 11, 16 and 18 were reverse scored; that is, a stronger agreement with these items was transformed into fewer points and vice versa.
Table 1
The Factors of the MALQ
Factors Description of the Factor Items Planning & evaluation Prepare for listening.
Evaluate the results of their listening efforts.
1, 10, 21 14, 20
Problem solving Make inference (guess at the unknown part) and monitor the inference.
5,7, 9, 13, 17, 19
Directed attention Concentrate or maintain concentration on tasks.
2, 6, 12, 16
Mental translation Translate heard information into L1. (Listeners must learn to avoid this if they want to be skilled listeners.)
4, 11, 18
Person knowledge Listeners’ perceptions and self-efficacy in L2 listening.
3, 8, 15
Second Language Listening Anxiety Scale (SLLAS)
The listening anxiety scale applied on the pre- and post-tests of the present study was Cheng’s (2014) Second Language Listening Anxiety Scale (SLLAS). The SLLAS includes only nine questions (Appendix C), measuring three dimensions of listening anxiety: cognitive, somatic (physiological), and behavioral. According to Cheng (2014), the construct validity of the SLLAS was ascertained based on a confirmatory factor analysis, which yielded fit indexes meeting the recommended criteria (normed χ² = 1.24; SRMR = .03; RMSEA = .04; CFI = 1.00; GFI = .97). The internal reliability of the SLLAS (Cronbach’s alpha = .83) was also satisfactory.
Listening Worksheets
The listening worksheets for the experimental group (Appendix D) provided guidance for students to practice four major metacognitive strategies while doing listening exercises: planning, monitoring, problem-solving, and evaluation. These four strategies were chosen based on Vandergrift and Goh’s (2012) metacognitive approach to listening. The strategies were presented on the worksheet in the sequence
of planning, monitoring, problem-solving, and then evaluation. Details about these metacognitive strategies are shown in Table 2.
Table 2
Descriptions of the Metacognitive Strategies Instructed Metacognitive
Strategy
Description
Planning Preview the content and prepare for listening.
Predict key words and phrases of the listening material.
Consider strategies for dealing with the possible challenges.
Monitoring Check understanding of messages by applying appropriate language knowledge (e.g., contextual and linguistic) and world knowledge.
Determine accuracy of understanding between old and new information.
Determine whether the approach adopted to understand the text is effective or not.
Problem-solving Adjust their approach by applying more appropriate strategies (e.g. revise predictions or adjust pervious inference to new possibilities).
Make inference from the understood content.
Evaluation Check overall acceptability of understanding of information.
Confirm comprehension based on a transcript of the text.
Check accuracy of understanding between old and new information.
Assess the effectiveness of strategies for listening practice.
Note. Adapted from Vandergrift and Goh (2012, p.106).
At the end of the worksheet, the participants were asked to write down their reflections on the listening tasks. The worksheets were collected after each class.
Semi-structured Interview Guide
In order to explore the reasons why participants in the experimental or the comparison groups made progress or regress greatly after the intervention, and to
supply data gathered from the questionnaires and the worksheets, interviews were applied in the study. From each group, five participants whose listening performance improved the most in the eight-week experiment and five participants whose performance regressed the most were chosen as the interviewees. The interviews were guided by pre-determined questions (Appendix E) that explored details of the listening process, including the difficulties the participants encountered, the anxiety they experienced, and the strategies they used in listening.
Metacognitive Strategy Instruction Procedure
In the eight-week treatment, the experimental group received instruction of metacognitive listening strategies, a process-based instruction which focused on four major metacognitive strategies: planning, monitoring, problem-solving, and evaluation (Vandergrift & Goh, 2012). The instruction was conducted twice a week, 45 minutes each time.
Figure 1 presents the sequence of metacognitive listening instruction, followed by descriptions of the instructional sequence.