This chapter reports the results and discusses factors involved. The first section presents the VKS scores the students obtained in the pre- and post-test. The changes of the scores are further analyzed to see the changes of the students’ performances before and after the teaching experiment. Following the quantitative data, the second section contains the analyses of the writing task. First of all, the changes of the use of target word are presented. Then, qualitative analyses of the students’ products are included to see the actual word use by the students from different levels, which helps better understand the students’ performance in the productive task. The last section includes the observations made by the teacher-researcher during the experiment.
Analyses of the VKS
The section first presents the scores of the VKS in the pre-test to show the students’ initial knowledge level of the target vocabulary. After that, the scores of the VKS in the post-test are included. The changes between the two scores are further compared and analyzed.
The Pre-Test
The students’ VKS scores indicated different levels of word knowledge through five descriptors. The VKS scores were counted and averaged for each target word. For the pre-test, 25 words fell in the range of 2.01-3.00, 16 words in 1.00-2.00, 11 words in 3.01-4.00, and only one word above 4.01. According to the descriptors, score 2 meant that the students had seen the word but could not recall it, and 3 meant that the students could roughly guess the meaning of the word. The average scores for all the target words were 2.41, which indicated insufficient knowledge of the target words when doing the pre-test. The following table showed more detail of the scores, the target words, and word counts of the VKS in the pre-test. The new words that the
40
students had never learned before have been boldfaced.
Table 3
The Results of the VKS in the Pre-Test
Words &
Counts Scores
Target Words Counts
1.00-2.00 abuse, access, apply, attitude, benefit, convey, device, efficient, identity, privacy, remark, responsible, site, switch, violate, virtual
16 words
2.01-3.00 account, clip, comment, community, connect, contact, describe, environment, essential, express, factor, file, incident, information, influence, issue, major, necessity, negative, opinion, product, regular, situation, social, visual
25 words
3.01-4.00 bully, content, control, delete, event, focus, message, personal, relax, surf, technology
11 words
4.01-5.00 download 1
word The results of the VKS indicated that the average scores did not necessarily correspond to whether a word had been learned from a textbook before. Of the 53 words, 40 words had been taught in the previous textbooks. However, the scores showed that the students may have forgotten most of the words. This highlighted the importance of repetition (Hatch & Brown, 1995; Hunt & Beglar, 2005; Lee &
Muncie, 2006; Zimmerman, 1997). As for the remaining 13 new words that the students had not learned, the students got even lower scores. Most of the scores fell in
41
the range of 1.00-2.00, three of them 2.01-3.00. Only two common words, “bully” and
“delete”, were above 3, indicating that most of the students were familiar with these two words only. The reason for this may be their frequent exposure to the word
“bully” and “delete” in their daily life, especially when using the computer.
The Post-Test
After the 12-week experiment, the teacher researcher administered the post-test and analyzed the data. The same method of calculation was adopted, so that there were no other factors affecting the average scores.
Table 4
The Results of the VKS in the Post-Test
Words &
Counts Scores
the Target Words Counts
1.00-2.00 none 0
2.01-3.00 abuse, access, apply, attitude, benefit, comment, community, connect, contact, content, convey, describe, device, efficient, essential, express, factor, identity, incident, issue, opinion, privacy, regular, remark, responsible, situation, social, switch, violate, virtual, visual
31 words
3.01-4.00 account, clip, environment, event, file, information, influence, major, necessity, negative, product, site, surf, technology
14 words
4.01-5.00 bully, control, delete, download, focus, message, personal, relax
8 words
42
Table 4 showed the detail of the scores, the target words, and word counts of the VKS after the experiment. The new words that the students had never learned before have also been boldfaced. For the post-test, 31 words fell in the range of 2.01-3.00, 14 words in 3.01-4.00, 8 words above 4.01, and no word below 2.00. According to the descriptors, 2 meant that the students have seen the word but could not recall it, and 3 meant that the students could roughly guess the meaning of the word. The average scores for all the target words were 3.09, which indicated that the students not only recognized most of the words but knew their meanings when doing the post-test.
Compared with the students’ performance in the pre-test, the results in the post-test indeed improved a lot. Table 5 shows the VKS scores in the two post-tests and the changes of the levels of each word. There were no words falling into level 1 in the post-test, which meant that the students could at least remember seeing these words.
Moreover, up to 22 words got above 3.01, showing that the students knew these words in the post-test stage. The improvement was obvious in both the old words and the new words. In the pre-test, the majority of the new words and some old words were completely unknown to the students. However, these words moved to the next level, and some even improved more. Overall, 28 words moved one level up. What’s more, two target words, “abuse” and “site,” moved two levels higher in the post-test. A total of eight target words became the productive level, an impressive achievement
compared to the results in the pre-test. Such progress suggested that the experiment succeeded in improving the students’ knowledge for the target words.
43
Table 5
The VKS Scores of Target Words4
No Word Pre- Post- No Word Pre- Post-
4 Following the previous chapter, the 13 new words have been boldfaced here. The circles highlight important points for discussion. Besides, the numbers in the last column indicate the changes of the levels of each word after the experiment.
44
Although the VKS scores of both the old and new words increased after the teaching experiment, some old words remained at the same level. Both types of words were repeated the same times during the experiment, but the old words seemed to improve less. On the other hand, the students remembered the newly learned words better: five out of the 13 new words were marked as level 3 or level 4 words, which meant that the meanings of the words were known to the students. This may result from the design of the current study. Since the experiment was incorporated with the ongoing lesson, the students studied and reviewed the new words for their mid-term exam, which helped improve their knowledge of these words.
From the results, it is evident that the students have improved significantly after the experiment. Table 6 showed the comparison of the VKS scores, and the scores of post-test at all levels were higher than those of the pre-test. To see how the students’
scores in the post-test differed from those in the pre-test, the teacher-researcher used EXCEL to conduct a paired T-test on the average VKS scores of the pre- and post-test.
The result of the one-tailed T-test showed remarkably significant differences between the two scores (p = 2E-185).
Table 6
Comparison of the VKS
Scores Tests
1.00-2.00 2.01-3.00 3.01-4.00 4.01-5.00 Average score Pre-test 16 words 24 words 12 words 1 word 2.41 Post-test 0 word 31 words 14 words 8 words 3.09
The most striking improvement was the change in the range of 1.00-2.00. In the pre-test, 16 words fell in this level. According to the descriptors, the results indicated
5 The value 2E-18 means that two times ten to the negative 18th – 18 zeroes after the decimal point.
45
that most of the students did not know nearly one-third of the target words or could not recall the meanings. However, this changed greatly in the post-test, with no target words remaining in this range. The drastic improvement in this level provided
evidence that the teaching experiment has succeeded in increasing the students’ word knowledge, aiding them in keeping the vague impression of seeing the target words.
In addition to that, changes could also be seen in the next level. In the pre-test, 25 words belonged to the range of 2.01-3.00, while in the post-test, the number increased to 31 words. This meant that the students remembered seeing the words, and could even guess some of the meanings correctly. The number changed from 25 words to 31 in the post-test, which was over half of the target words. Interestingly, in both the pre- and post-test, the students could not recall most of the target words. The phenomenon provided important information about learning words. Despite the fact that the students had been exposed to the words for at least six times during the experiment, they still found it hard to recall the exact meanings of the words they learned. This provided evidence that though repetition was indeed necessary for the students to know words better (Hatch & Brown, 1995; Hunt & Beglar, 2005; Lee &
Muncie, 2006; Zimmerman, 1997), repeating merely six times may not be sufficient for the students to gain a deeper knowledge of nearly half of the target words.
In the next level of 3.01-4.00, the number increased from 12 words in the pre-test to 14 in the post-test. The change might seem small, but closer inspection revealed great differences of the words falling in this range. According to the descriptor, the range indicated that the students could remember the exact word meanings without making any guesses. Of the 12 words appearing in this level in the pre-test, only three of them remained in this level in the post-test. The others upgraded to the next level.
This showed that in the post-test, most of the 14 words improved from a lower level to this level. It was quite evident that the students’ vocabulary knowledge enhanced
46
from roughly knowing or guessing the word meanings to exactly recalling the meanings after the experiment. What’s more, of the 14 words in the post-test, three words, “account,” “file,” and “site,” were new words, while the others were old words from the previous textbooks. It can be proved that the experiment was effective for both the words the students just learned and those they had learned before.
An interesting point was the word “content,” the only word that got a lower score in the post-test. This may result from the learning sequence of the target word. Since the students just learned one of the meanings of “content,” which was “be happy with,” in the beginning of the new semester, they could answer the item on the VKS correctly. However, when doing the post-test, many students may have forgotten the adjective meaning, neither could they recall the noun meaning reviewed in the
experiment. As a result, they got a lower score on this particular word. This suggested students memorized newly-learned vocabulary better in the short term. As time went by and the students’ memory faded, repetition of both of the meanings were required for the students to remember and use the word productively in the long term.
Lastly, in the level of 4.01-5.00, major differences also existed between the pre- and post-test. In the pre-test, only one word “download” fell on this range, while in the post-test, there were eight words. The word “download” still remained in this range in the post-test, and the other seven words were added to this level. The result indicated a significant gain in vocabulary knowledge. The students were familiar with these words in the pre-test, since they could recall the meanings. However, their knowledge of these words improved greatly when taking the post-test in using these words to actively construct sentences.
Analyses of the Writing Task
This section contains two parts. The first part analyzes the students’ writing products and presents the changes of target word use in the pre- and post-test. Then,
47
the second part includes the qualitative analyses of three sample writing products of the students to help understand the actual word usage by the students of different levels.
Changes of Target Word Use
In addition to the VKS, the students’ writing products also provided critical information regarding the students’ word knowledge, especially active production of words. The changes of target word use in the pre- and post-test are discussed based on the changes in the three categories: correct usages, related words, and mistakes.
The Pre-Test. After the administration of the pre-test, the teacher-researcher
gathered the students’ compositions and calculated the results of correct usages, related words, and mistakes. Table 7 shows the results of the writing task in the pre-test. In total, only 9 out of 53 words were used correctly by the students in the pre-test stage. The most frequently used were “surf” and “bully,” ten times and nine times respectively. Both words were highly common and related to the prompt. When compared with the results of the VKS, these two words were at the range of 3.01-4.00. Several other words were also used for only once or twice. Interestingly, though the students were quite familiar with the two words, four students still made mistakes with the spelling or usage of the word “surf”.
Another intriguing point of the writing prompt lay in the use of related words.
When carrying out the pre-test, a small number of the students used words relevant to the target words. Two of them used the word “website”, another one used “impact”, and another one used “upload”. They were compound, synonym, and antonym of the target words “site,” “influence,” and “download” respectively. These words did not belong to the target words, but the use of these words showed that the users had relevant knowledge of target words, since word knowledge included knowing the word parts and associating the words with other similar words as suggested in
48
literature (Nation, 2001).
Table 7
The Results of the Writing Task in the Pre-Test
Types Data
Correct usages Related words Mistakes
Words
Added together, the total number of words that were used correctly in the pre-test was 12 out of 53 words, including four new words (refer to Table 7 for the boldfaced words). These 12 words were used by 24 different students, each using one to three words in their compositions. The result indicated that in the pre-test, one-third of the students did not actively recall the words, and the rest of the students could merely use few words productively in their writing products. It seemed that the students possessed very little productive knowledge of the target words before the experiment.
The Post-Test. After finishing the post-test, the teacher-researcher followed the
same procedures of collecting and calculating the data. In Table 8, the results of the writing task in the post-test is reported. Out of expectation, the results were
surprisingly affirmative. In total, 25 out of 53 words, nearly half of the target words in the experiment, were used correctly by different students. The most frequently used were still the words used in the pre-test, “bully” and “surf”. The only difference was that they were used more often than they were in the pre-test. Aside from these two words, there were a variety of words used several times by the students. The words
“abuse” and “event” were used a number of times, and many other words by the
49
students, as shown in the following table.
Table 86
The Results of the Writing Task in the Post-Test
Types
Despite making several mistakes, there was a considerable increase in the number of the target words used productively in the post-test. Added together, the number of words used correctly was 31 target words by 34 different students. In contrast, only 12 words appeared in the pre-test. This provided evidence that the students acquired the words and could use them to produce sentences. Compared with the results in the pre-test, the results of the writing task in the post-test showed great improvement in terms of target word usage. In the pre-test, there were only 9 correct usages, but this improved to 25 correct usages in the post-test. Also, there were only 3
6 Here, following the previous chapter, the new words have been boldfaced.
50
related words used in the pre-test, but in contrast, 6 related words were used in the post-test. Interestingly, there was also an increase in mistakes, from one mistake in the pre-test to 8 mistakes in the post-test, which may be the evidence that more attempts were made by the students to use the words productively.
Another point worthy of noticing is the students’ word use in the pre- and post-test. In both tests, the most frequently used words were “bully” and “surf,” almost ten times used. That meant nearly one-third of the students actively recalled these two words. This may result from the nature of the writing prompt, which required the students to write about their experiences on the Internet. These two words were highly related to the prompt, in that “surf” was the most common verb used in this kind of topic, and that “bully” was a familiar issue to most of the students. The result also proved a close prompt-word relationship in the experiment.
A closer examination of the related words used in the pre- and post-test also provided insights into the effects of the experiment. In the pre-test, there were only three related words used. The words were “website,” a compound of “site,” “impact,”
a synonym of “influence,” and “upload,” the antonym of “download”. Particularly, since “download” was the only word that got above 4.01 in the pre-test, the use of its antonym showed that the students could apply the word knowledge to learning new words.
Just as the correct usage of target words, mistakes could also provide insight into the students’ gain of vocabulary knowledge. In the pre-test, four students misspelled the word “surf”. To better understand their proficiency level, the individual students’
scores on the VKS were examined. It was found that their average scores were in the range of 2.01-3.00, suggesting the students’ unfamiliarity with most of the target words. After the experiment, two of the four students still misspelled “surf.” Closer inspection showed that their VKS scores in the post-test showed slight improvement.