The aims of the current study are to: (1) determine if captioning, the visually-presented text, can facilitate EFL middle school students’ vocabulary acquisition in the aural aspects, and (2) discover whether the gains of vocabulary acquisition are modulated by learners’ linguistic competence. The independent variables are the two treatment conditions—captioned and non-captioned conditions, and three levels of linguistic competence—high, intermediate, and low levels1. The dependent variables are the participants’ scores on the overall form recognition and the overall vocabulary acquisition (i.e. the form-meaning mapping). The analysis of the collected data in this current study will be presented in the following sections to answer the proposed research questions.
This chapter contains three major sections. A two-way analysis of variance (ANOVA) was employed in the first two sections to answer the research questions in this study. Section one will discuss the comparison of the participants’ overall form recognition under captioned and non-captioned conditions; the relationship between the students’ linguistic level (high, intermediate, and low) and performance on overall form recognition will also be reported. Likewise, section two will first explore the comparison of the participants’ overall vocabulary acquisition under the two caption conditions. Then, the relationship between the students’ linguistic level and overall vocabulary acquisition will also be presented in this section. Section three will determine the caption effect on spoken word gains within groups by performing the paired samples t-test.
Results of Overall Form Recognition
In this three-month experimental study, ten video materials selected from the cartoon series Olivia were employed during the learning sessions, one video for one regular class period each week. However, since several students were absent in the second and eighth learning sessions due to the school activities and competition, the
1 This current study determined the students’ linguistic competence by the average scores on the achievement tests of previous three semesters and divided them into three linguistic groups—(1) high-level (i.e. the top-third of the participants), (2) intermediate-level (i.e. the middle-third), and (3) low-level (i.e. the last-third).
44
data collected in the two sessions were excluded from the final data analysis. At last, the data collected from the other eight learning sessions were used for analysis. In addition, it is worth to note that the data collected from the eight weekly measurements were collapsed into one for analysis because of the various features of each video material. Although the chosen videos were assumed to possess appropriate
difficulty-level (i.e., at least 90% lexical coverage for the participants), it was roughly estimated based on the information reported by six students in the pilot study. In fact, the video materials varied from one another in the length of time, the number of words, the novel word density, and the topic familiarity to the individual students. Thus, it seems inappropriate to make comparisons of the learning outcomes yielded from viewing the 8 videos which are not perfectly identical.
Therefore, instead of comparing the mean scores test by test, the current study averaged the scores in the eight vocabulary tests to examine the effects of captioning on overall spoken vocabulary acquisition. Then the scores were analyzed by a two-way ANOVA (with two independent variables— 1) caption condition and 2) level of linguistic competence) using SPSS Statistics. The results are presented in the following two subsections. First, the participants’ overall form recognition under captioned and non-captioned conditions will be reported and discussed in the first subsection. Then, the performance of students at different linguistic competence levels on overall form recognition will be presented and explored in the second subsection.
The Participants’ Overall Form Recognition under Captioned and Non-captioned Conditions
To have a clear picture of the results collected from form recognition tests, Figure 2 was presented below to illustrate students’ performance in the eight form recognition tests under captioned and non-captioned conditions. In general, the means in the eight tests did not change dramatically; instead, they changed rather gradually.
With regard to the captioned group, the mean scores remained quite similar from test one to test three and reached the highest in test four. Then it fell gradually until the last test, in which the mean score rose slightly again. Moreover, Figure 2 also
displayed that the captioned group consistently scored higher than the non-captioned group on average in form recognition tests.
45
0 2 4 6
1 2 3 4 5 6 7 8
Test
Mean scores
captioned non-captioned
Figure 2. Students’ performance in the eight form recognition tests under both conditions
As mentioned earlier, making direct comparisons of the results yielded from the 8 imperfectly identical videos could be problematic. In order to test and compare the participants’ overall form recognition under different treatment conditions (i.e., the captioned condition and non-captioned condition), their mean scores and standard deviations on overall form recognition were calculated and presented in Table 4 below.
As shown in Table 4, the mean scores of the phonologically recognized words under the two treatment conditions were 3.733 (the captioned condition) and 3.017 (the
non-captioned condition). Obviously, participants in the captioned group scored higher than those in the non-captioned group on overall aural form recognition. But whether this difference is statistically significant, a further test is warranted.
Table 4. Descriptive statistics of overall form recognition
Condition N M SD
Captioned 60 3.733 1.163
Non-captioned 58 3.017 1.034
Total 118 3.381 1.154
Note. Maximum score = 7.
To further examine if there was a statistically significant difference between the participants’ scores on form recognition under captioned and non-captioned conditions, and if there was an interaction effect between the two independent variables (i.e., the caption condition and level of linguistic competence), a two-way ANOVA was performed to analyze the data collected from form recognition tests. Levene’s test of homogeneity of variances for form recognition was not significant, p = .208. Therefore,
46
the assumption of homogeneity of variances between the groups is not violated and they can be compared by using ANOVA (see Table 5).
Table 5. Test of homogeneity of variances for overall form recognition Levene
Statistic df1 df2 Significance
Form Recognition 1.463 5 112 .208
The results of ANOVA are summarized in Table 6 below. Table 6 indicated that there was a significant caption effect on form recognition, F (1, 112) = 20.091, p = .000, and the interaction effect between the caption condition and the level of linguistic competence did not attain statistical significance, F (2, 112) = .589, p = .557.
Table 6. Summary of two-way ANOVA on overall form recognition
Source Type III SS df MS F Post-hoc
Caption 14.958 1 14.958 20.091*** C > N.C.
Level 56.504 2 28.252 37.948*** H > I, H > L I > L
Caption * Level .877 2 .044 .589
Residual 83.384 112 .745
Total 1505.000 118
Note. Caption = Caption Condition; Level = Level of Linguistic Competence.
C = Captioned; N.C. = Non-captioned; H = High; I = Intermediate; L = Low.
***p < .001.
The results in Tables 4 and 6 implied that when the participants in this study were exposed to the chosen video material twice with captioning, they were able to recognize substantially more words on the subsequent listening multiple-choice tests without explicit instruction. In addition, the insignificant interaction effect between caption condition and linguistic competence indicated that, for aural form recognition, the effect of captioning would not depend on learners’ linguistic level. In other words, viewing video with the presence of captioning tended to yield a more facilitative effect on form recognition than without captioning, regardless of learners’ linguistic level.
47
Differences among Learners at High, Intermediate, and Low Levels of Linguistic Competence on Overall Form Recognition
As mentioned earlier in Chapter Three, in this study, the participants’ linguistic competence level was determined by the average scores of the achievement tests in English subject in the previous three semesters. Based on the average scores, the students who were in the top one third were classified as those of high linguistic competence, the middle one third as intermediate linguistic competence, and the last one third as low linguistic competence. The results in Table 6 above also revealed that the participants’ linguistic competence had a statistically significant result on form recognition, F (2, 112) = 37.948, p = .000. To provide an overall picture on the relationship between form recognition and the level of linguistic competence, mean scores and standard deviations of the participants of different linguistic competence on overall form recognition are displayed in Table 7 below.
Table 7. Means and standard deviations for overall form recognition by level of linguistic competence
Level N M SD
High 39 4.282 .972
Intermediate 40 3.275 .987
Low 39 2.590 .818
Total 118 3.381 1.154
Note. Level = Level of linguistic competence; The maximum score = 7.
As shown in Table 7, the mean scores of the phonologically recognized words obtained by participants at different linguistic levels were 4.282 (the high level), 3.275 (the intermediate level), and 2.590 (the low level). Taken together, the participants at the upper-level of linguistic competence scored significantly better than those at lower linguistic levels. In sum, although vocabulary gains on form recognition occurred at all levels, these data suggested that students of higher competence in English tended to make significantly more vocabulary gains through incidental learning than others from the same viewing experience.
Results of Overall Vocabulary Acquisition
In order to examine the participants’ vocabulary acquisition under captioned and
48
non-captioned conditions at a deeper level, the other vocabulary test, form-meaning mapping, was administered immediately after the form recognition test in each learning session. Same as the data analysis procedures for overall form recognition, this study averaged participants’ scores on the eight form-meaning mapping tests and performed a two-way ANOVA to (1) investigate the participants’ overall vocabulary acquisition under captioned and non-captioned conditions, and (2) explore the performance of students at different linguistic competence levels on overall vocabulary acquisition.
The Participants’ Overall Vocabulary Acquisition under Captioned and Non-captioned Conditions
To have a clear picture of the results collected from vocabulary acquisition tests, Figure 3 was first provided below to illustrate students’ performance in the eight vocabulary acquisition tests under captioned and non-captioned conditions. In general, the ogive patterns of the captioned and non-captioned group were similar to those in Figure 2. Overall, although there were some ups and downs during the eight tests, the means in the eight tests changed gradually. For both captioned and non-captioned group, the mean scores were quite flat from test one to test three and reached the highest in test four. Then they fell gradually until the seventh and the eighth tests in which the mean scores rose slightly again. In addition, as shown in Figure 3, the captioned group, again, consistently scored higher than the non-captioned group on average in the eight vocabulary acquisition tests.
0 2 4 6 8
1 2 3 4 5 6 7 8
Test
Mean scores
captioned non-captioned
Figure 3. Students’ performance in the eight vocabulary acquisition tests under both conditions
In order to test and compare the participants’ overall vocabulary acquisition under
49
different treatment conditions (i.e., the captioned condition and non-captioned
condition), their mean scores and standard deviations on form-meaning mapping tests were calculated and presented in Table 8 below. As shown in Table 8, the mean scores under the two treatment conditions were 4.233 (the captioned condition) and 3.707 (the non-captioned condition). These raw scores showed that for overall vocabulary
acquisition, the participants in the captioned group, again, seemed to score higher than those in the non-captioned group.
Table 8. Descriptive statistics of overall vocabulary acquisition
Condition N M SD
Captioned 60 4.233 1.140
Non-captioned 58 3.707 1.124
Total 118 3.975 1.158
Note. The maximum score = 7.
To further examine if there was a statistically significant difference between the participants’ scores on overall vocabulary acquisition under captioned and
non-captioned conditions, and if there was an interaction effect between the two independent variables (i.e., the caption condition and linguistic competence), a two-way ANOVA was performed to analyze the data collected from form-meaning mapping tests. Levene’s test of homogeneity of variances for overall vocabulary acquisition was not significant, p = .060, and hence, it could be assumed that the groups were similar in variance and could be compared by using ANOVA (see Table 9).
Table 9. Test of homogeneity of variances for overall vocabulary acquisition Levene
Statistic df1 df2 Significance
Vocabulary Acquisition 3.617 5 112 .060
The results of ANOVA are summarized in Table 10 below. The analysis indicated a significant caption effect on vocabulary acquisition, F (1, 112) = 10.312, p = .002, and that the interaction effect between the caption condition and linguistic competence did not reach statistical significance, F (2, 112) = 1.167, p = .315.
50
Table 10. Summary of two-way ANOVA on overall vocabulary acquisition
Source Type III SS df MS F post-hoc
Caption 8.058 1 8.058 10.312** C > N.C.
Level 59.184 2 29.592 37.869*** H > I, H > L I > L
Caption * Level 1.824 2 .912 1.167
Residual 87.521 112 .781
Total 2021.000 118
Note. Caption = Caption Condition; Level = Level of Linguistic Competence.
C = Captioned; N.C. = Non-captioned; H = High; I = Intermediate; L = Low.
***p < .001; **p < .01.
The results in Tables 8 and 10 revealed a similar trend in favor of captioning; that is, the captioned group scored significantly higher than the non-captioned group on overall vocabulary acquisition. This positive effect of captioning suggested that instead of just enhancing aural form recognition, the availability of captioning could also facilitate associations of word meaning and spoken representation. In addition, the insignificant interaction effect between caption condition and linguistic competence entailed that, for form-meaning mapping, the effect of captioning would not depend on learners’ linguistic competence. In sum, these data demonstrated that with the
availability of captioning, EFL middle school students appeared to make significant gains in word knowledge without explicit vocabulary instruction, and the visual representations of words on the screen could be a contributor to students’ enhanced vocabulary acquisition.
Differences among Learners at High, Intermediate, and Low Levels of Linguistic Competence on Overall Vocabulary Acquisition
The results in Table 10 revealed that the participants’ linguistic competence had a statistically significant effect on overall vocabulary acquisition, F (2, 112) = 37.869, p
= .000. To offer an overall picture on the relationship between the vocabulary acquisition and learners’ linguistic competence, the mean scores and standard
deviations of participants at different linguistic levels are displayed in Table 11 below.
51
Table 11. Means and standard deviations for overall vocabulary acquisition by level of linguistic competence
Level N M SD
High 39 4.872 1.080
Intermediate 40 3.925 .971
Low 39 3.128 .656
Total 118 3.975 1.158
Note. Level = Level of linguistic competence; The maximum score = 7.
As shown in Table 10, the mean scores obtained by participants at different linguistic levels were 4.872 (the high level), 3.925 (the intermediate level), and 3.128 (the low level). As mentioned earlier, the insignificant interaction effect between caption condition and linguistic competence showed that vocabulary gains did take place at all linguistic levels with the availability of captioning; however, the results of ANOVA along with Tables 7 and 11 suggested that, students of high linguistic
competence tended to make significantly more vocabulary gains on both form
recognition and vocabulary acquisition than others from the same viewing experience without formal instruction. In addition, it was noteworthy that the participants scored higher on overall vocabulary acquisition than overall form recognition. This result was unexpected because it was hypothesized that form-meaning mapping might be more challenging than form recognition for the participants. It turned out that the students performed better on the overall vocabulary acquisition than they did on the overall form recognition under both captioned condition (M = 4.233 for vocabulary acquisition; M = 3.733 for form recognition) and non-captioned condition (M = 3.707 for vocabulary acquisition; M = 3.017 for form recognition). Detailed explanation of this unexpected outcome will be presented in the discussion section in Chapter Five.
The Caption Effects on Spoken Word Gains within Groups
To determine whether or not captioned video materials had any impact on spoken word gains within the three linguistic groups, the paired samples t-test was performed on the two types of vocabulary tests (i.e., form recognition and vocabulary acquisition) respectively. In this current study, scores obtained in the first learning session were used as the baseline and were compared with the average scores obtained from session two to session eight to test the differences within linguistic groups after the participants
52
received the weekly treatment.
Impacts on Form Recognition
Table 12 below presented the results of paired samples t-test for the three linguistic groups’ form recognition under both treatment conditions. By comparing the differences between the mean scores of the first test and the follow-up tests, the data in Table 12 demonstrated that under the captioned condition, the average score of high-level group’s follow-up tests, M = 4.550, is obviously higher than the mean score in the first test, M = 3.250, while the mean scores of the first and the follow-up tests of intermediate-level (M = 3.900 and M = 3.900) and low-level groups (i.e., M = 2.800 and M = 2.750) remained almost the same. Moreover, the t-value further indicated that after receiving the follow-up seven-week learning sessions, the high-level students were able to aurally recognize substantially more words, t (19) = 3.213, p = .005, than they did in the first session. However, there was no statistically significant differences in the means of form recognition tests obtained by intermediate-level as well as low-level students between session one and the follow-up sessions.
On the other hand, under the non-captioned condition, Table 12 displayed that while there was a decrease in the mean scores from the first test to the follow-up tests for both high-level and intermediate-level groups, the low-level group made a slight improvement in the follow-up tests. However, all the three linguistic groups
demonstrated no significant differences in form recognition between the first and the follow-up sessions. Taken together, Table 12 suggested that after the participants received the 7-week repeated learning sessions after the first one, captioned video materials tended to markedly and exclusively benefit students of high linguistic competence in aural form recognition.
53
Table 12. Summary of paired samples t-test for form recognition by students' linguistic levels under both conditions
Condition Level
Test 1 Test 2-8
M SD M SD df t
Captioned H 3.250 1.743 4.550 1.050 19 -3.213**
I 3.900 2.125 3.900 1.165 19 0.000 L 2.800 2.262 2.750 0.716 19 0.100 Non-captioned H 4.263 1.996 3.739 0.806 18 1.191 I 3.100 1.410 2.750 0.716 19 1.234
L 2.053 1.545 2.316 0.671 18 -0.620
Note. H = High; I = Intermediate; L = Low.
** p < .01
Impacts on Vocabulary Acquisition
Table 13 below provided the results of paired samples t-test for the three linguistic groups’ vocabulary acquisition in the first and the follow-up sessions under both conditions. The data in Table 13 indicated that under the captioned condition, the mean scores of high-level group’s vocabulary acquisition improved slightly from the first session (M = 5.000) to the follow-up sessions (M = 5.200), although not to a significant extent. This finding might be attributed to the ceiling effect since some participants in the high-level group gained full marks (i.e., 7 points) in the first vocabulary acquisition test. In contrast, the mean scores of intermediate-level and low-level groups dropped in the follow-up sessions.
With regard to the performance of students under the non-captioned condition, the low-level group, again, made a slight improvement in the average score of the
follow-up tests, M = 3.000, compared with the mean of the first test, M = 2.632, while the high-level group dropped slightly in the average score of the follow-up tests and the intermediate-level group remained almost the same in terms of the mean scores.
54
Table 13. Summary of paired samples t-test for vocabulary acquisition by students' linguistic levels under both conditions
Condition Level
Test 1 Test 2-8
M SD M SD df t
Captioned H 5.000 1.523 5.200 0.951 19 -0.639 I 4.750 1.209 4.200 0.894 19 1.927 L 3.950 1.504 3.150 0.671 19 2.179*
Non-captioned H 4.895 1.792 4.579 1.170 18 0.809 I 3.400 1.353 3.450 0.887 19 -0.175
L 2.632 1.571 3.000 0.667 18 -1.022
Note. H = High; I = Intermediate; L = Low.
* p < .05
55