In order to measure whether the subjects had made improvement in vocabulary acquisition, the researcher used two measures of lexical richness to assess the vocabulary development. One was the ratio of different words (Types) to the total number of words (Tokens), known as type-token ratio (TTR) to measure vocabulary variation, and the other was VocabProfile analysis (Cobb, 2001). TTR, seemingly
the most widely used measure of lexical richness, has been described as a measure of language variation (Laufer, 1994; Laufer & Nation, 1995), “a measure of vocabulary
‘flexibility’ or variability,” (Johnson, 1944, p. 1), and a measure of “vocabulary richness” (Andolina, 1980, p. 373). A high figure means that the text contains a wide range of different words and a low one indicates that the writer or speaker has relied on a small amount of words that are frequently repeated. The figure is known to vary with text length. As Carroll (1964) stated, “This ratio will tend to decrease as sample size increases, other things being equal, because fewer and fewer of the words will not have occurred in the sample already counted” (p. 54). For example,
“the type-token ratio of a 500-word text is lower than that for a 300-word one” (Read, 2000, p. 201). While the type/token ratio distinguishes between the different words used in speaking or writing, it does not show the quality of different words used.
The VP analysis, however, can compensate for such inadequacy. Cobb’s (2001) VocabProfile (VP) which is available online and allows color as well as numeric visualization of output compares words in a text with words lists that accompany the program. It divides any text into four categories by word frequency including the most frequent 1000 words of English, the second most frequent thousand words of English, i.e. 1000 to 2000, the academic words of English, and the remainder which are not found on the other lists (Cobb, 2001). The lists are from West’s (1953) General Service List, which contains 2000 word families and the Academic Word List (AWL) consisted of 570 word families which occurred with high frequency and wide range in the academic corpus. Here, the most frequent words refer to the base forms of the words. Under each headword are its inflected and derived forms. Therefore, the most frequent 2000 words include far more than this number and so does the Academic Word List. These lists are mainly based on printed texts, but also have been used quite often to evaluate richness of spoken language (Donzelli, 2002; Meara,
1993; Meara, Lightbown, & Halter, 1997). Laufer & Nation’s (1995) study validated VP analysis a reliable and valid measure of lexical use in writing. The results showed that the VP analysis is reliable across two pieces of writing by the same person provided the topics are of general nature and require no expert knowledge of the particular subject matter. It also discriminates between learners of different proficiency levels as well as correlates with an independent measure of vocabulary knowledge.
As for research on evaluating lexical richness in speaking, Meara’s (1993) study analyzed the vocabulary richness of BBC English broadcast by using VP. No difference was found between the lexical profiles of easy program series and more difficult ones. Just over half the words came from the Nat1 list, the 1000 most frequent words in English. A further 10 % came from the Nat2 list, the second thousand most frequent words. Meanwhile, Meara et al. (1997) investigated whether classrooms can be characterized as rich lexical environment. VP was used to analyze the vocabulary in the speech of ten teachers in intensive communicative ESL classes for children. Findings showed that over half of the vocabulary offering across these classrooms consisted of basic items from the 0-1000 frequency level. It was concluded that classrooms as rich lexical environment cannot be conclusively proved. Another research using VP to evaluate richness of spoken language is Donzelli (2002), which examined the lexical richness of teacher talk in three different class-levels of EFL classes. The results suggested that the level of words difficulty in the teacher’s speaking was found to be equal in the different class-levels. These studies showed that VP analysis has been viewed as a useful instrument to analyze the spoken data. Another study using written corpora for oral vocabulary analysis is Brown, Sagers & LaPorte’s (1999) research. Brown et al.’s (1999) research on the examination of incidental vocabulary acquisition from oral and written dialogue
journals found that the texts that advanced university EFL learners produced had no significant difference in the level of the vocabulary in the two modes by using the Brown University Corpus, a written corpus for word frequency analysis.
Accordingly, VP analysis is considered an appropriate measure to analyze vocabulary richness.
The level of most of the subjects in this case was intermediate level, so it was assumed that they may use simple words to express their ideas. Besides, the data are spoken language, rather than written, so the chance of the words being used beyond the most frequent 1000 words becomes even less. Based on these reasons, in this study, the original four levels (the most frequent 1000 words of English, the second most frequent thousand words of English, i.e. 1001 to 2000, the AWL, and the remainder which are not found on the other lists) were divided into two levels, one is basic vocabulary including the most frequent 1000 words and the other is non-basic vocabulary including the words beyond the 1000 words including the second most frequent 1000 words, the AWL and the words not found on the other lists. The oral journals at the entry (the 1st & 2nd journals), mid (the 13th & 14th journals), and end (the 25th & 26th journals) stages of the project were analyzed with the two measures.
To perform the VP analysis, the oral journals were transcribed and entered into the computer program for calculation. Because every time for the same subject, his or her utterances had different lengths, the present study following Laufer’s (1991) method by taking everyone’s least production length for analysis. Take one of the students, Sheree as an example, the total number of words she spoke in the 1st, 2nd, 13th, 14th, 25th, and 26th oral journals are 387, 302, 392, 390, 432, 459 respectively.
In order to make the calculation reliable and consistent, only the first 302 words for each oral journal were selected. Finally, comparisons of means (F-test) on the journals at entry, mid, and end stages were carried out between the percentages of
words at the two levels.
To sum up, the measurements in this present study were numerous. For fluency, the five variables were chosen including the rate of speech, the number of filled and unfilled pauses per 100 words, repetitions per 100 words, the length of fluent speech runs and the ratio of hesitation pauses in the total time of speaking. As for grammatical accuracy, errors of verb tense, prepositions, plural forms and subject-verb agreement were calculated for analyses. In terms of vocabulary, two measurements of vocabulary richness (i.e. TTR and VocabProfile analysis) were used to analyze the subjects’ oral journals.