In this chapter, the experimental design of the present study will be provided.
Detailed information of the research method will be presented in the following sections:
1) the background information about the participants of this study, 2) the design of the study, 3) the materials, 4) the procedures of data collection, 5) the instruments, and 6) data analysis.
Participants
One hundred and eighteen students in a public junior high school in Taipei City participated in this study. They were all eighth graders recruited from four intact classes (Class A, B, C and D), twenty-nine students in Class A, thirty-one students in Class B, twenty-seven students in Class C, and thirty-one students in Class D. They had four regular English class periods plus one extra English class period per week, forty-five minutes each. Their English teacher was the researcher in the current study. Among the students, half of them were female, and the others were male. Each class had an equal distribution of boys and girls. All the participants had learned English for more than five years since they were third graders. In addition, all the students’ first language is Chinese, and none of them had lived in English-speaking countries for more than a half year.
To investigate the relationship between EFL middle school students’ linguistic competence and vocabulary acquisition through watching video materials, the
participants were assigned into three linguistic groups—high-level, intermediate-level, and low-level of linguistic competence according to their scores on the achievement tests of the previous three semesters (Please see the Instrument Section for more detail).
The criteria for the three linguistic groups are based on the scoring guidelines employed by the school, which are presented as follows: Students with average marks in the top-third of all the participants on the achievement tests of the previous three semesters were assigned to be high-level students, the middle-third were intermediate-level students, and the last-third were low-level students. Students from the three linguistic groups were randomly and evenly assigned to the captioned group and the
non-captioned group (control). Based on the above design, in group with captioning,
30
there were 60 students in total, including 20 high-level students, 20 intermediate-level students, and 20 low-level students while in group without captioning, there were 58 students in total, including 19 high-level students, 20 intermediate-level students, and 19 low-level students.
There are certain reasons why the students were recruited as the participants in this study. First, the four classes were of similar English competence based on their average scores on the achievement tests of the previous three semesters, suggesting that they had comparable academic ability. Second, all the participants were the pupils of the researcher. Before the experiment, the researcher could familiarize the students with the presence and the absence of captioning when viewing videos in class. Therefore, it was expected that during the learning sessions of the experiment, the students might accept the caption modes (i.e., with or without captioning) and the researcher’ video
instruction more easily. Third, the participants were all eighth graders, who were assumed to have less difficulty understanding the cartoon videos than the seventh graders. Therefore, the selected videos should not cause too much trouble for the students to comprehend during the experiment.
Design of the Study
The aim of this study is to investigate spoken vocabulary acquisition through incidental learning from watching video materials with or without captioning. In order to provide a general picture of captioning effect and examine word gains from video viewing in a more reliable manner, the duration of the experiment is twelve weeks. In the first meeting, the participants were informed of the three-month video-viewing experiment without being notified of the aim of the study (i.e. captioning effect on word gains) and asked to complete several pre questions before the experiment to provide brief background information. The questions included two main sections—1) demographic data (e.g., age, gender, L1, length of study of English, and previous learning experience), and 2) the attitudes towards learning from video viewing including their perception of watching video in class and out-of-class exposure to English video material.
The experiment was conducted in the regular English class periods. During the learning sessions, ten videos were employed, and each was viewed twice. This current study adopted a two times of viewing design for both theoretical and practical reasons.
31
First, from theoretical perspective, Zahar, Cobb, and Spada’s study (2001) provided empirical evidence that incidental vocabulary acquisition could take place when the unfamiliar words were met two times in the level-appropriate text. One hundred and fourteen ESL seventh graders, whose vocabulary size was about 2000 words,
participated in their study. All the participants were subdivided into five groups based on their scores in the vocabulary pretest, Nation’s Vocabulary Levels Test. After reading the text, The Golden Fleece, in which 91 percent of the words were from the 2000 word frequency level, the participants took a vocabulary posttest consisting of 30 items in which the students were required to match the words to brief definitions. The frequency of the 30 target words in the text ranged from one to fifteen times. Results yielded from their study suggested that frequency played a more important role in acquisition for lower-level students as it was for higher-level students. Moreover, according to the results of the number of students in each group that learned words with different frequency in the text, the study also revealed that when the words were encountered two times in the level-appropriate text, incidental learning could occur across different vocabulary-level groups.
Second, students in this current study watched the selected video episode twice (about 10 minutes long each), instead of three or more times due to the practicality issue.
To be more specific, since this study was conducted in regular class periods, of which the length of time of one class period was 45 minutes, the procedures for data collection in one learning session including the viewing session and posttest-taking session must be completed in 45 minutes. Accordingly, two times of viewing, which took about 20 minutes, would be the maximum time that could be used for the viewing session in one class period. Given the empirical support from Zahar et al’s (2001) study and the time constraint of this current study, this study examined the foreign word gains by
providing learners with two times of exposure to the level-appropriate video materials.
The participants were divided into two groups: one with captioning, one without.
Within each group, the participants were further divided into high-level,
intermediate-level, and low-level linguistic groups based on the achievement in English for examining whether the vocabulary gains were modulated by students’ linguistic competence levels. Except for the availability of captioning, the two groups viewed the video materials with the same number and in the same sequence to ensure they received equal input. To put it more specifically, the only difference between the two groups was
32
that the captioned group, received three types of input simultaneously: images, audio, and text while the non-captioned group only received two types of input which were the imagery and auditory input.
To examine whether the availability of captioning could facilitate EFL young learners’ spoken vocabulary acquisition while the main focus was on material
comprehension, three types of tests were developed—one comprehension test, and two vocabulary tests. While the comprehension test in this study was adopted as a tool to direct the learners’ focus on messages, the two vocabulary tests were used to assess the word gains at different levels. The first type of vocabulary test, the form recognition, was conducted to answer the unresolved question—whether the presence of captioning was beneficial to aural recognition. More specifically, the aural recognition test was used to examine whether the students in captioned group could recognize more target words aurally than those in non-captioned group. The second type of test, the
vocabulary acquisition, was administered to investigate the vocabulary knowledge at a deeper level—the acquisition of word meaning, for examining whether the participants in captioned group could have a better performance in mapping the aural form to its meaning than those in non-captioned group. More details about the two types of tests will be expounded in the section of instruments. After the ten-week video viewing sessions, the participants were required to complete several follow-up questions in the last meeting based on their perception of the 10 weekly learning sessions (see the instrument section for detailed information).
Materials Video Selection
To explore the effects of learning novel words aurally through viewing videos, the researcher selected 10 episodes from the Olivia series, a British-American
children's animated television series based on Ian Falconer's books and produced by media company Chorion. This series was selected for three main reasons.
First, the storyline of Olivia series is clear-cut and easy to follow; hence, the students are expected to easily grasp the ideas conveyed in each video episode.
Moreover, for the affective domain, the Olivia series is very likely to be target audience appropriate because the interesting and imaginative presentation of family and school life in the cartoon videos is relevant and assumed of interest to the average
33
middle school students in Taiwan. Second, based on Lin’s (2010) suggestions about the criteria of material selection, the video materials have to be of similar length, number of words and speech rate. Since the Olivia cartoon series seems to take care of the above criteria, selecting ten video episodes consisting of roughly comparable features may not cause too much trouble. Third, the language of the cartoon series is generally of appropriate difficulty to the participants of this study. Since the aim of the current study is to examine vocabulary acquisition through incidental learning, it is important to provide the learners with appropriate lexical coverage of the material.
In fact, because the target audience of the cartoon series is native-speaking children, the difficulty-level of the videos is taken care of to some extent. This study further estimated the difficulty-level of the video materials to the target participants by pilot-testing another video on six 8th graders of varying competence at the same school a month prior to the experiment. By using another video (the 11th video) of Olivia series which was assumed at the difficulty-level comparable to the chosen 10 videos in terms of length of time, speech rate, and novel word density, the six students were asked to count the number of unknown words when watching the video. It turned out that in the video contains 806 words in total, the unknown words reported by the six students ranged from twenty-four to sixty-three. Namely, the percentage of known words of the 11th video was from 92 % to 97 %. Therefore, it was estimated that at least 90 percent of the words in the selected videos were familiar to the target participants, which met the threshold of text coverage proposed by Hu and Nation (2000).
In the end, ten video episodes, each represents a topic, were selected from the cartoon series. After the material selection was done, the captioning was added to each video material for captioned group by the researcher with the software, Power Director 10, since the original versions displayed no on-screen text. Complete information about the ten video materials is shown in Table 2 below.
34
Table 2. Information about the 10 video materials
Episode Topic Words Time (s) Speech Rate
1 The Might Five 815 510 1.60
2 Olivia Gets Fit 776 575 1.35
3 Olivia Plans a Tea Party 921 513 1.80
4 Olivia's Christmas Surprise 666 505 1.32
5 Olivia's Kite Party 635 505 1.26
6 Olivia's Road Race 786 509 1.54
7 Olivia's Meteor Mania 739 500 1.48
8 Olivia's Tip Top Tapper 727 509 1.43
9 Olivia's Dog Wash 886 505 1.75
10 Olivia's Hiking Adventure 788 511 1.54
Note. Words = number of words; Time = length of time Speech rate = number of words divided by length of time
Target Word Selection
In each episode, the words that were most likely to be unfamiliar to the
participants were first selected by the researcher and an experienced English teacher at the same school respectively, and then those selected by both judges became target words. Since the researcher was the participants’ English teacher, the researcher was supposed to know, most of the time, whether the words may or may not be familiar to the participants.
During the target word selection process, parts of speech of the words were also taken into account. According to Harley (2006), there are two main categories of English words: function and content words. Content words, which include nouns, verbs, adjectives, and adverbs, carry the messages we want to deliver while function words such as conjunctions and determiners, serve to provide the structures that help facilitate better communication. Since the aim of this study is to investigate whether the students could recognize the aural word forms and infer their meanings from contextual clues when their attention is directed to the messages of the cartoons with a high correlation of audio and video, it is much more appropriate to measure the acquisition of content words. In the end, seven words were selected from each episode as target words (seventy in total), including 42 nouns, 14 verbs, and 14 adjectives. Information about the target words is shown in Table 3 below.
35
Table 3. Information about the target words
Episode Topic Target Words (part of speech)
1 The Might Five rattle (n.), villain (n.), grab (v.), tug (v.), split (v.), castanet (n.), mastermind (n.)
2 Olivia Gets Fit demonstrate (v.), toss (n.), dash (n.), athlete (n.), jumping jack (n.), headband (n.), canopy (n.)
3 Olivia Plans a Tea Party pale (adj.), sniffle (n.), splendid (adj.), proper (adj.), dainty (adj.), bounce house (n.), delightful (adj.) 4 Olivia's Christmas Surprise squeeze (v.), peek (v.), hamper (n.), spin-dry (v.),
slobber (n.), paddle (n.), teepee (n.)
5 Olivia's Kite Party blowy (adj.), blustery (adj.), rip (v.), streamer (n.), string (n.), yarn (n.), refreshing (adj.)
6 Olivia's Road Race pit crew (n.), refreshment (n.), course (n.), lean (v.), hardboard (n.), ruin (v.), shovel (n.)
7 Olivia's Meteor Mania meteor (n.), dazzling (adj.), jester (n.), bagpipe (n.), stardust (n.), tickle (v.), catchy (adj.)
8 Olivia's Tip Top Tapper slip (v.), recital (n.), cast (n.), sprained (adj.), crutch (n.), rolling-walking (n.), invention (n.)
9 Olivia's Dog Wash splashing (adj.), scrape (n.), puddle (n.), hose (n.), grooming (n.), guarantee (v.), rinse (v.)
10 Olivia's Hiking Adventure badge (n.), baton (n.), compass (n.), squishy (adj.), muddle (n.), skunk (n.), halt (v.)
Note. There are 7 target words in each video material.
Procedures for Data Collection
This study was conducted in the second semester of the participants’ second school year. One month before the learning sessions, the participants took the vocabulary pretest to measure their prior knowledge of the target words. The
experiment lasted three months. The participants had one learning session per week.
The first and the twelfth sessions were used to collect qualitative data by asking the participants to complete several pre-learning and post-learning questions respectively for further explanation. From the second to the eleventh meetings were the learning sessions. In each learning session, the participants in different caption groups watched
36
one video episode twice with different caption modes (with or without captioning).
There was no explicit vocabulary instruction during the viewing sessions. Three immediate posttests were given to the two groups of students in exactly the same order.
That is, after the first viewing, a reading comprehension test was given to the students, and two vocabulary posttests (form recognition test was conducted first followed by vocabulary acquisition test) were administered immediately after the second viewing.
The whole data collection process took place in regular classrooms with an overhead projector and an interactive whiteboard on the left in front of the classroom. The procedure for data collection in the current study is illustrated in Figure 1 below.
Figure 1. Procedures for data collection
Instruments The Achievement Test
As mentioned earlier, to look into the relationship between captioned video and linguistic competence, the average achievement test scores of the previous three
Vocabulary Pretest (one month before)
Pre-learning questions (W1)
Captioned condition
Non-captioned condition
After the 1st viewing: a reading comprehension test;
After the 2nd viewing: two vocabulary tests (form recognition and form-meaning mapping)
twice
Post-learning questions (W12) Repeat
10 weeks
(W2- W11)
37
semesters were used to determine the participants’ linguistic competence level in this study. Specifically, based on the school’s scoring guidelines, the students with average scores in the top-third of the participants were assigned to be the high-level students, the middle-third were the intermediate-level students, and the last-third were the low-level students. Detailed information about the achievement tests are provided as follows.
At the middle school, the English achievement tests, which cover multiple aspects of linguistic knowledge, are composed of three main sections. One of them is the listening section comprising 15 multiple-choice questions (5 for answering the
questions according to the pictures, 5 for choosing the best response to each question or statement, and 5 for answering the questions after listening to a conversation). The maximum possible score for this section is 15. The other two sections are reading and writing, respectively. The reading section involves 30 multiple-choice questions on vocabulary, grammatical structures, cloze, and reading comprehension. The maximum score for this section is 70. The writing section includes vocabulary spelling and Chinese-to-English translation practices. The maximum score for this section is 15.
The achievement test scores were used in this study to determine learners’
language competence due to certain reasons. First, JURKOVIČ (2010) indicates that learners’ preexisting linguistic competence could be a positive predictor of
achievement test scores. Undoubtedly, students’ varying degrees of success in achievement tests depend on a number of factors; nevertheless, JURKOVIČ’s study empirically proves that there is a high correlation between learners’ linguistic competence and their academic performance.
Second, the achievement tests conducted in the middle school measure both listening and reading skills. Since the aim of the study is to investigate the effects of the audiovisual medium offering multisensory input, it is more appropriate if learners’
abilities of reading and listening are taken into account simultaneously.
Finally, this current study determined students’ linguistic profiles by achievements in English due to the practicality. For EFL teachers in Taiwan, students’ performance on achievement tests at school is frequently instructors available. In other words, adopting students’ performance on achievement tests is very likely to be the most convenient and economic way to determine learners’ linguistic profiles for in-service teachers.
Hopefully, the results yielded from this current study may provide some practical
38
implications for EFL teachers when dealing with middle school students of varying linguistic competence.
As mentioned in Chapter Two, although a few of the existing studies have investigated the relationship between learners’ linguistic competence and language learning from captioned video (Lwo & Lin, 2012; Neuman & Koskinen, 1992; Taylor, 2005), there are some issues in previous research needing to be addressed. To be specific, only one of the studies reviewed above determined learners’ linguistic profiles by the performance on achievement tests in English (Lwo & Lin, 2012), and there were some methodological issues in the previous study. One of the problems was the small sample size in their study (N = 32).
In addition, to collect learners’ oral responses scene by scene, their study adopted the interruptive “pause scene-by-scene” method. However, this could be considered as a weakness because the experimental procedure rendered the viewing experience far from the real-life one where general comprehension is usually the priority. Therefore, to address these issues, this study determined the 118 EFL middle school students’
In addition, to collect learners’ oral responses scene by scene, their study adopted the interruptive “pause scene-by-scene” method. However, this could be considered as a weakness because the experimental procedure rendered the viewing experience far from the real-life one where general comprehension is usually the priority. Therefore, to address these issues, this study determined the 118 EFL middle school students’