Metacognitive Strategy Instruction Procedure
CHAPTER FOUR RESULTS
This chapter reports the results drawn from the collected data, which include the questionnaires, the listening comprehension tests, the worksheets, and the interviews. The findings of the present study are presented in the following five sections. The first section presents the background information of the participants.
The second section reveals the results of the pre- and post- listening comprehension tests. The findings of the change of participants’ listening anxiety level before and after the intervention are shown in the third section. In the fourth section, the participants’ change of metacognitive awareness of listening strategies before and after the intervention is presented. The last section summarizes the descriptive data which were collected from the worksheets in the eight-week intervention and the post-test interviews.
Results of the Background Information Questionnaire
The valid questionnaires collected from the participants were 56 in total.
According to the questionnaires, all of the participants had studied English for over 6 years. Among them, 5 students (17.9%) in the experimental group and 7 students (25%) in the comparison group reported that they had studied English for over ten years. In addition, most of the participants had never lived in English-speaking countries, except for one student in the experimental group, who reported having lived in America for one year when she was 12 years old.
In terms of the materials of practicing English listening comprehension outside of school, 17 students (60.7%) in the experimental group and 21 students (75%) in the comparison group reported that they did not have extra English listening practices after class. As can be seen in Table 3, 11 students (39.2%) in the experimental group and 7 students (25%) in the comparison group reported that they practiced listening comprehension through listening to English magazines or radio programs, and only 4 students (14.3%) in the experimental group listened to English songs or watched English movies to improve their listening comprehension performance.
44
Table 3
Participants’ Reported Materials for English Listening Practice Outside of the Classroom
Items The Experimental Group
(n = 28)
The Comparison Group (n = 28)
No extra practice 17 21
English magazines and English radio programs
11 7
Online sources 0 2
English songs or movies 4 0
Note: The materials that each participant reported might be more than one. Therefore, the total of items chosen by each group may exceed the number of students in each group.
Table 4 lists the length of time that the participants spent on extra listening practice out of class in a week. As can be seen, most of the participants were exposed to extra English listening practice for about two to four hours per week.
Table 4
The Time Spent on English Listening in a Week
Items The Experimental Group
(n = 28)
The Comparison Group (n = 28)
Less than one hour 0 3
One to two hours 4 3
Two to four hours 7 13
Four to six hours 6 2
Six to eight hours 5 7
Over eight hours 6 0
45
Table 5 presents the difficulties that the participants often encountered while listening to English comprehension questions. As shown in the first row of Table 5, both the experimental group and the comparison group agreed that their insufficient knowledge of English vocabulary, phrases, and grammar was the difficulty that they most frequently encountered while listening to English comprehension questions.
Also, more than half of the participants reported that their lack of the background knowledge and terms of unfamiliar topics was the difficulty they often encountered while listening to English. Furthermore, half of the participants in the experimental group were also bothered by fast speaking rate.
Table 5
The Participants’ Listening Difficulties
Items The
Lack of background knowledge and terms
13 16 29 51.8%
Speakers’ unfamiliar accents 13 11 24 42.9%
Note: The difficulties that each participant reported might be more than one.
Therefore, the total of difficulties chosen by each group may exceed the number of students in each group.
To further investigate whether the participants had taken and passed the GEPT tests, the participants were asked to report their test experiences and the results were as follows: five students (17.9%) in the experimental group and two students (7.1%) in the comparison group had the experience of taking the GEPT tests. Among them, only three participants (10.7%) in the experimental group and two students (7.1%) in the comparison group had passed the GEPT elementary level, and two participants
46
(7.1%) in the experimental group had passed the first stage of the GEPT tests, which means they passed listening and reading sections but failed in writing and speaking sections.
Results of the Listening Comprehension Tests
The listening tests applied in the study were derived from the listening section of General English Proficiency Tests (Elementary Level), which contained 30 items in each test. The participants would get 4 points for each correct answer, and the full marks in the listening comprehension test were 120. The tests were applied before and after the intervention in order to measure the participants’ listening comprehension. In particular, the t-test was conducted on the pre-test and the post-test to see if there were significant differences between the two groups in listening ability before and after the experiment. Also, to investigate the effect of metacognitive listening strategy
instruction on enhancing the participants’ listening comprehension performance, paired-t test was applied to analyze the pre-tests and post-tests within groups.
The descriptive statistics of listening pre-test and post-test results are presented in Table 6. As can be seen, the mean score (M = 82.14, SD = 25.38) of the experimental group in the listening pre-test was higher than that of the comparison group (M = 71.57, SD = 20.69). In addition, the experimental group had higher scores of the listening post-test (M = 78.14 SD = 23.29) than that in the comparison group (M = 68.00, SD = 18.98). In both pre-test and post-test, the standard deviation of the experimental group was wider than that of the comparison group.
Table 6
Descriptive Statistics of the GEPT Listening Test
Comparison (n = 28) Experimental (n = 28)
M SD M SD
Pre-test 71.57 20.69 82.14 25.38
Post-test 68.00 18.98 78.14 23.29
To further investigate if the difference of the pre-tests between the two groups reached the level of significance, independent t-test was applied to analyze the data.
The results of the t-test are presented in Table 7. As can be seen, there was no significant difference between the two groups in the listening comprehension pre-test
47
(t = -1.71, p > .05). In other words, the two groups’ listening performance was similar before the intervention.
In addition, the experimental group’s score on the listening post-test (M = 78.14, SD = 23.29) appeared higher than that of the comparison group (M = 68.00, SD = 18.98). However, with the analysis of independent t-test, the listening comprehension post-test score of the experimental group was not significantly higher than that of the comparison group (t = -1.79, p > .05). The results indicate that the experimental group did not significantly outperform the comparison group in the post-tests.
Table 7
Between-Group Comparison of Listening Comprehension Tests
Comparison Experimental t(54) p 95% CI Cohen’s D
M SD M SD LL UL
Pre-test 71.57 20.69 82.14 25.38 -1.71 .09 -22.98 1.84 0.46 Post-test 68.00 18.98 78.14 23.29 -1.79 .08 -21.53 1.24 0.48
To investigate whether the metacognitive listening strategy instruction had a significant effect on the participants’ listening comprehension performance, paired-t tests were conducted to analyze the data. The within-group comparisons of the listening pre-test and post-test scores are shown in Table 8. As can be seen, the comparison group’s pre-test score (M = 71.57, SD = 20.69) was higher than its post-test score (M = 68.00, SD = 18.98). However, the paired-t test revealed that the difference was not statistically significant (t = 1.53, p > .05). In other words, even though the pre-test score of the comparison group was higher than its post-test, the difference between the two tests did not reach the significance level.
On the other hand, the experimental group’s pre-test mean score (M = 82.14, SD
= 25.38) was higher than its post-test score (M = 78.14, SD = 23.29) and the drop in score reached the significance level (t = 2.16, p < .05). It shows that the experimental group’s listening scores decreased significantly from the pre-test to the post-test.
48
Table 8
Within-Group Comparison of Listening Comprehension Tests
Group Pre-test Post-test t(54) p 95% CI Cohen’s
M SD M SD LL UL D
Comparison 71.57 20.69 68.00 18.98 1.53 .14 -1.23 8.37 0.29 Experimental 82.14 25.38 78.14 23.29 2.16 .04* .20 7.80 0.41
*p < .05
Because both groups showed a drop in listening performance from the pre-test to the post-test, the independent t-test was used to check if there was a significant difference in the drop between the two groups. As can be seen in Table 9, the comparison group’s decrease in score (M = -3.57, SD = 12.38) was slightly smaller than that of the experimental group (M = -4.00, SD = 9.80), but the difference did not reach the significance level, t = .14, p > .05.
Table 9
Between-Group Comparison of Listening Test Score Drop
Group Score Drop t(54) p 95% CI Cohen’s
M SD LL UL D
Comparison -3.57 12.38 .14 .89 -5.55 6.41 -0.04 Experimental -4.00 9.80
In sum, the two groups had no significant difference on the listening pre-test or post-test. In addition, the comparison group did not have significant difference between its listening pre-test and post-test, even though its scores of the post-test were lower than that of the pre-test. In contrast, although the experimental group received the metacognitive listening strategy instruction, its post-test was significantly lower than its pre-test. In other words, the listening comprehension of the experimental group regressed significantly after receiving the metacognitive strategy instruction.
However, the learning loss between the two groups did not reach the significance level.
49
Results of the Second Language Listening Anxiety Scale
Second Language Listening Anxiety Scale (SLLAS) was applied to examine the change of the participants’ listening comprehension anxiety levels before and after the intervention. SLLAS is a 5-point-Likert scale with 9 items; the maximum score of the scale is 45 and the minimum score is 9. The higher the score is, the more anxious the participant is. The data collected from SLLAS were analyzed by independent t-test first to check if the pre-tests of the two groups had significant difference. Paired-t test was administered to each group’s pre-test and post-test to investigate whether the two groups had made significant progress after the intervention.
Table 10 presents the descriptive statistics of the SLLAS pre-test scores and post-test scores between the two groups. As can be seen, the comparison group’s SLLAS mean in the pre-test (M = 26.36, SD = 6.49) was higher than that of the experimental group (M = 23.21, SD = 5.52).
Table 10
Descriptive Statistics of the SLLAS
Comparison Experimental
M SD M SD
Pre-test 26.36 6.49 23.21 5.52
Post-test 22.82 6.25 23.43 6.69
To investigate whether the difference between the two groups reached the significance level, independent t-test was conducted to analyze the data and the results are shown in Table 11. As can be seen, the SLLAS pre-test scores between the two groups did not have significant difference (t = 1.95, p > .05). The result reveals that the listening anxiety level of the two groups before the intervention was similar. After the intervention, the comparison group’s SLLAS mean in the post-test (M = 22.82, SD
= 6.25) was lower than that of the experimental group (M = 23.43, SD = 6.69), but the difference between the two groups’ post-tests did not reach the significance level (t = -.35, p > .05). The result indicates that there was no statistically significant difference between the two groups on the post-tests of SLLAS. In other words, compared with the comparison group, the experimental group’s listening anxiety was not lowered significantly after receiving the metacognitive listening strategy instruction.
50
Table 11
Between-Group Comparison of SLLAS
Comparison Experimental t(54) p 95% CI Cohen’s
M SD M SD LL UL D
Pre-test 26.36 6.49 23.21 5.52 1.95 .06 -.09 6.37 -0.52 Post-test 22.82 6.25 23.43 6.69 -.35 .73 -4.08 2.86 0.09
Table 12 presents the paired-t test results of SLLAS within the same group. As shown in Table 12, for the comparison group, the mean score (M = 26.36, SD = 6.49) of the SLLAS pre-test was higher than the mean of SLLAS on the post-test (M = 22.82, SD = 6.25). Paired-t test showed that the post-test score of the comparison group was significantly lower than its own pre-test score (t = 2.98, p < .05), which indicates that the participants in the comparison group significantly lowered their anxiety levels after receiving the traditional listening instruction.
On the other hand, the experimental group’s SLLAS pre-test score (M = 23.21, SD = 5.52) was similar to its post-test score (M = 23.43, SD = 6.69). The paired-t test further confirmed no significant difference between the two test scores (t = -.18, p
> .05). The results show that the participants’ anxiety level in the experimental group was not lowered significantly after they received the metacognitive listening strategy instruction. In contrast, the comparison group’s anxiety level was significantly lowered after the instruction.
Table 12
Within-Group Comparison of SLLAS
Group Pre-test Post-test t(54) p 95% CI Cohen’s
M SD M SD LL UL D
Comparison 26.36 6.49 22.82 6.25 2.98 .01* 1.10 5.97 0.56 Experimental 23.21 5.52 23.43 6.69 -.18 .86 -2.61 2.19 -0.03
*p < .05
Further analyses on the three dimensions of SLLAS (i.e., cognitive anxiety, somatic anxiety, and avoidance behavior) were also conducted. Independent t-test was first conducted on the pre-tests of both groups. When there was a significant
51
difference in pre-test score between groups, ANCOVA was conducted to compare the post-test scores by group with the pre-test score statistically controlled. Otherwise, independent t-tests were used to compare the two groups’ post-test scores. Besides, paired-t test was administered to investigate whether each group made significant progress in the three dimensions of anxiety after the intervention.
Table 13 shows the descriptive statistics for the three dimensions of the SLLAS between the two groups. According to Table 13, the comparison group’s pre-test scores on the three dimensions of SLLAS were higher than those of the experimental group. Similarly, the comparison group’s post-test scores were higher than those of the experimental group except somatic anxiety.
Table 13
Descriptive Statistics of the Three Dimensions of SLLAS
Pre-test Post-test
Comparison Experimental Comparison Experimental
M SD M SD M SD M SD
Cognitive anxiety 12.39 2.56 10.93 2.68 11.25 2.80 11.14 2.95 Somatic anxiety 7.82 3.59 6.71 2.90 5.68 2.67 6.64 3.26 Avoidance behavior 6.14 3.33 5.57 2.36 5.89 3.28 5.64 2.60
To investigate whether the differences reached significance level, independent t-test was conducted to analyze the data and the results are presented in Table 14. As can be seen, cognitive anxiety in the pre-test between the two groups had significant difference (t = 2.09, p < .05). The result reveals that the cognitive anxiety level of the comparison group in the pre-test was significantly higher than that of the experimental group. In the post-test, the three dimensions of SLLAS did not show significant difference between the two groups. In other words, the anxiety level of the comparison group and the experimental group did not show significant difference after the intervention.
52
Table 14
Between-Group Comparison of the Three Dimensions of SLLAS
Comparison Experimental t(54) p 95% CI Cohen’s
Because there was a significant difference in the two groups’ cognitive anxiety level in the pre-test, to verify the above independent t-test result on post-test cognitive anxiety, ANCOVA was further conducted on the two groups’ cognitive anxiety in the post-test with the pre-test score controlled. Table 15 shows the ANCOVA results of cognitive anxiety in SLLAS. According to Table 15, with the pre-test scores controlled, the scores of cognitive anxiety between the two groups on the post-test were not significantly different, F= .85, p > .05. This result confirms the independent t-test result reported above, indicating that the two groups did not differ significantly from each other in cognitive anxiety after the intervention.
53
Table 15
ANCOVA of SLLAS—Cognitive Anxiety
Source SS df MS F p
Pre-test 100.04 1 100.04 15.30 .00*
Post-test 5.54 1 5.54 .85 .36
Error 346.63 53 6.54
Total 7467.00 56
*p < .05
Table 16 shows the paired-t test results of the three dimensions of SLLAS for the comparison group. As can be seen, the scores of the three dimensions in the post-test were lower than those in the pre-test. Paired-t test showed that the comparison group’s somatic anxiety in the post-test was significantly lower than that in the pre-test (t = 4.01, p <.05).
Table 16
Within-Group Comparison of the Three Dimensions of SLLAS—Comparison Group Pre-test Post-test t(54) p 95% CI Cohen’s
M SD M SD LL UL D
Cognitive anxiety
12.39 2.56 11.25 2.80 2.01 .05 -.02 2.31 0.38
Somatic anxiety
7.82 3.59 5.68 2.67 4.01 .00* 1.05 3.24 0.76
Avoidance behavior
6.14 3.33 5.89 3.28 .42 .68 -.97 1.47 0.08
*p < .05
The paired-t test results of the three dimensions of SLLAS for the experimental group are presented in Table 17. As shown in Table 17, only somatic anxiety in the post-test appeared lower than that in the pre-test. Paired-t test showed that there was no significant difference between pre-test and post-test in any of the three dimensions.
The result indicates that the participants in the experimental group did not lower their anxiety significantly after they received the metacognitive listening strategy
54
instruction.
Table 17
Within-Group Comparison of the Three Dimensions of SLLAS—Experimental Group Pre-test Post-test t(54) p 95% CI Cohen’s
M SD M SD LL UL D
Cognitive anxiety
10.93 2.68 11.14 2.95 -.43 .67 -1.24 .81 -0.08
Somatic anxiety
6.71 2.90 6.64 3.26 .13 .90 -1.07 1.21 0.02
Avoidance behavior
5.57 2.36 5.64 2.60 -.15 .88 -1.07 .92 -0.03
To further investigate whether the decrease in SLLAS from pre-test to post-test between the two groups reached the significance level, independent t-tests were conducted on the drop of scores (i.e., the post-test scores minus the pre-test scores).
As can be seen in Table 18, the two groups differed significantly in the drop of somatic anxiety level, t = -2.69, p < .05. In other words, the comparison group’s decrease in somatic anxiety was significantly larger than the experimental group’s. In addition, the drop in SLLAS total scores between the two groups also had significant difference, t = -2.25, p < .05. The results show that the comparison group’s decrease in overall listening anxiety was significantly larger than the experimental group’s.
Table 18
Between-Group Comparison of SLLAS Score Drop
Score Drop t(54) p 95% CI Cohen’s
Comparison Experimental LL UL D
M SD M SD
Cognitive anxiety
-1.14 3.00 .21 2.64 -1.80 .08 -2.87 .16 0.48
Somatic anxiety
-2.14 2.82 -.07 2.94 -2.69 .01* -3.62 -.53 0.72
55
Table 18. (continued)
Score Drop t(54) p 95% CI Cohen’s
Comparison Experimental LL UL D
M SD M SD
Avoidance behavior
-.25 3.15 .07 2.57 -.42 .68 -1.86 1.22 0.11
SLLAS total
-3.54 6.28 .21 6.19 -2.25 .03* -7.09 -.41 0.6
*p < .05
In sum, the two groups had significant difference on cognitive anxiety in the pre-tests, but both the independent t-test and ANCOVA results show that the two groups did not have significant difference in the overall SLLAS or in any of the three dimensions of SLLAS after the intervention.
After intervention, the comparison group’s SLLAS total score and somatic anxiety score were significantly lowered than the scores on the pre-test. The results indicate that the comparison group had significantly lower listening anxiety after the intervention (i.e., traditional listening instruction). In contrast, although the experimental group had received the metacognitive listening strategy instruction, their listening anxiety level was not significantly lowered. Besides, the difference of the SLLAS post-test scores between the two groups did not reach significance level. In other words, the two groups’ listening anxiety levels in the post-test did not have statistically significant difference.
Results of the Metacognitive Awareness Listening Questionnaire
To examine the change of the participants’ metacognitive listening awareness, Metacognitive Awareness Listening Questionnaire (MALQ) was applied before and after the intervention. MALQ is a 5-point-Likert scale with 21 items in the present study; the highest score of the scale is 105, and the lowest score is 21. If the participants get higher scores on the MALQ, it represents that they have higher awareness of metacognition while listening to English.
The data collected from MALQ were analyzed by independent t-test first to check if the pre-tests of the two groups had significant difference. Paired-t test was
56
administered to each group’s pre-test and post-test to investigate whether the two groups had made significant progress in the post-tests after the intervention.
Table 19 reveals the descriptive statistics of MALQ. As shown in Table 19, the comparison group’s MALQ mean in the pre-test (M = 69.46, SD = 10.09) was lower than that of the experimental group (M = 70.54, SD = 8.78).
Table 19
Descriptive Statistics of MALQ
Comparison Experimental
M SD M SD
Pre-test 69.46 10.09 70.54 8.78
Post-test 70.11 10.48 75.54 8.35
The between-group comparisons of MALQ are presented in Table 20. As can be
The between-group comparisons of MALQ are presented in Table 20. As can be