Three intact classes of ninth graders (about 32 people in each class) in junior high school in northern Taiwan were invited to participate in the research. They were chosen because of their similar English proficiency according to the average scores of their previous two midterm exams. In other words, an analysis of their performances in the previous exams was conducted prior to the experiment to ensure the homogeneity among classes.
The three classes were all normal class groupings with both high and low proficient learners. They received five hours of general English regularly every week. The materials used and activities in general English class followed the nine-year curriculum guidelines from the Ministry of Education (M.O.E.). No additional supplement materials were provided for the students in these three classes.
In terms of their proficiency level, they were supposed to be ‘elementary’,
corresponding to General English Proficiency Level Test (GEPT)4. The three classes were randomly assigned to one of the three treatments (STG, MCG, and MCG-E).
3.2 Materials
3.2.1 Target Words and Vocabulary Pre-test
There were a total of eight target words adopted from the selected reading material. According to the two experienced in-service teachers and the writer, these words are supposed to be unfamiliar to the participants. To further ensure that the participants have little pre-knowledge of the target vocabulary, participants had to take a vocabulary pre-test (Appendix A) before the experiment session.
The parts of speech of the vocabulary included nouns, verbs, and adjectives.
Adverbs and propositions were not considered in this study for they were much harder for beginners to infer the meaning from context (Waring and Takaki, 2003).
So as not to alert the participants on the eight target words which were going to be used in the following treatments, eight other words which were supposed to be unknown to the participants were selected and mixed in the vocabulary pre-test.
In the pre-test, participants were asked to provide the Chinese meanings of the given target words. As soon as they finished the pre-test, the test sheets were collected but not returned. The answers of the pre-test were not given either. In this way, the learners had little chance to intentionally memorize the target words. The results of the vocabulary pre-test were used to modify and decide the target words appearing in the reading materials which the current study was going to adopt in the
4 According to the general level description of ‘Elementary’ in GEPT, an examinee who passes this level has basic ability in English and can understand and use rudimentary language needed in daily life. His/her English ability is roughly equivalent to that of a junior high school graduate in Taiwan.
following experiment session.
3.2.2 Reading passages and comprehension questions
A reading material (Appendix B) adapted from Guided Practice for Smart Vocabulary for Senior High 2000 was manipulated as an instrument. According to
the Ministry of Education (M.O.E.), students in junior high school need to acquire at least 1200 words before they graduate. Reading materials of 2000-word level contain some vocabulary which is not included in junior high textbooks but also most of the words students had already known. Most important of all, the grammars and syntaxes embedded in the readings are familiar to students at junior high level.
The readability of readings was examined using the Flesch Kincaid Grade Level, which is extensively used by teachers in the U.S. to judge the readability level of books and texts in the field of education. Research on readability in BCTest has indicated that the appropriate level for ninth graders in Taiwan was about three to five5. Therefore, the adopted reading material in the current research was modified to match the appropriate readability level.
Besides, according to Liu and Nation (1985), 96% of words are suggested to be known by readers to ensure the fluency of reading and reasonable chances of inferencing of unknown words to occur. Therefore, the length of the reading passage was at least 200 words for there were eight target words in the current research.
Students were required to answer reading comprehension questions after
5 The data come from 飛揚專刊 (65), P2-P8. This magazine is published by The Committee of the Basic Competence Test for Junior High School, whose objective is to promote the Multiple Entrance Program and respond to the publics’ questions about the BCTest for the M.O.E. The magazine is written by educators and specialists in the field and is published regularly.
finishing the reading. In this way, the participants were supposed to engage in the reading tasks instead of intentionally memorizing the target words, following Krashen’s (1989) notion that lexical learning as a ‘by-product’ of a task or activity.
These comprehension questions involved the general reading content, for example, the main idea, skimming, details scanning, or content inferencing. Since the current study aimed at exploring lexical learning through reading in a more incremental and unintentional way, no vocabulary-focused or vocabulary inferencing questions were specifically dealt with at this part.
Participants in different treatments groups, in fact, received the same reading passage but with different gloss treatments. The STG group received the passage with single-translation gloss (Appendix B) while MCG and MCG-E groups received the multiple-choice glossed passage (Appendix C). Considering the suggestion of reducing the amount of distractors from the previous study (Watanabe, 1997) and the participants’ English proficiency in the current study, there were one correct answer and only one distractor provided as the choice items for the multiple-choice glossing groups.
3.2.3 Immediate posttest and delayed posttest
To investigate the effects of different gloss types on word recognition, participants received an unexpected immediate posttest (Appendix D) after the treatment and a delayed posttest (Appendix E) two weeks later. A total of eight target words were included in the test. The goal of these tests was to investigate the amount of vocabulary the participants recognized after the reading activity with glosses.
In the immediate posttest, participants were required to choose the correct Chinese meanings of the given target words in the form of multiple choices. To be more specific, the receptive aspect of vocabulary knowledge was concerned. Though vocabulary knowledge includes both receptive and productive aspects (Laufer, Elder, Hall, and Cogndon, 2004; Schmitt, 2010), the meaning recognition test in the current research was to investigate only the receptive aspect of vocabulary knowledge. The reasons were as follows. First, productive knowledge is generally believed as involving higher degrees of familiarity of words, with only one encounter;
participants in the current research were not expected to be able to memorize the forms of the words. Besides, junior high school students are not required to produce English words in their further Basic Competence Test; therefore, recognizing the meaning of words, especially words not included in the 1200 word list, is regarded as sufficient for their learning need at the present stage.
To elicit the ‘pure’ learning effect, no contexts were provided for they might provide clues for learners to infer the word meanings in the posttest. There were one correct answer and one distractor for each target item. To eliminate the chance of wild guessing which might degrade the validity of the study, a third choice item, ‘I don’t know,’ was the alternative. An unexpected delayed posttest was implemented two weeks later to understand the retention of the recognized vocabulary. The form of the delayed test was actually identical to the immediate posttest but with different distractors and different item orders.
In terms of the scoring the two posttests, each correct choice of the target word meaning obtained 1 mark whereas a wrong choice got 0 mark. A full mark was eight points in both the immediate and delayed posttests.
3.2.4 Survey and focused-group interviews
A survey was distributed to the participants in the MCG and MCG-E groups.
Since little literature has revealed the implementation of multiple-choice glosses in the junior high level, the purpose of the survey was to preliminarily explore the attitude of the junior high participants towards using multiple-choice glosses in reading. To be more specific, whether they favored the multiple-choice glossed method or not would be checked in the survey. As mentioned, the target participants concerned in most of the previous studies on comparing STG and MCG were adults and senior high school learners (Huang, 2007; Hulstijn, 1992; Nagata, 1999; Rott, 2005; Wang, 2010; Watanabe, 1997). Since junior high school students are at the initial phase of forming their reading ability and the ability to learn words through reading, it seems to be of great importance to understand their perceptions while manipulating multiple-choice glosses in reading. Their opinions might be valuable to instructors or textbook writers since the enhancements the instructors or textbook writers adopted have direct impacts on these learners.
After the survey, two focus groups with five participants from the MCG and MCG-E groups respectively were carried on. The participants were chosen according to the richness of the participants’ responses in the survey. The richness of their responses might suggest that these candidates had more thoughts about the treatment and had the ability to express themselves clearly in the interviews.
Whether the participants like the method or not would not be the criterion in choosing the participants for the survey. Besides, it was also commonly believed that the heterogeneity of the group members generated more fertile results.
The purpose of the two focused-group interviews was to have a deeper
investigation of the participants’ perceptions toward the MCG method and MCG-E method. Several issues were concerned, including their prior experiences of using multiple-choice glosses, their thoughts or feelings when using multiple-choice glosses, and their experiences of using multiple-choice glosses in the current study.
Through probing these issues, their reasons for the liking or disliking of the MCG were understood. The interview data were also used to explain or supplement the quantitative part of the current research.