raters evaluated the composition samples. Their error feedback was calculated and categorized. Table 27 is a list of the 34 error types of all the errors identified in the 146 samples.
Table 27. The Types and Numbers of Error Messages Identified by Human Raters
1. Spelling errors (168) 18. Word order (27)
2. Missing/ unnecessary/ incorrect prepositions (129) 19. Run- on sentences (26)
3. Teacher's rewrite (127) 20. Unclear meaning (26)
4. Verb tense (119) 21. A vs. An (22)
5. Verb group consistency (87) 22. Word choice (21)
6. Missing/ unnecessary/ incorrect articles (86) 23. Causative verb (16) 7. Nouns: mass or count (72) 24. Infinitive or -ing form (13) 8. Subject-Verb agreement (68) 25. Comparative/ superlative (11)
9. Word confusion (67) 26. Pronoun reference (8)
10. Word form (63) 27. Possessive errors (7)
11. Misused words (63) 28. Capitalization (6)
12. Clause errors (53) 29. Compounding errors (6)
13. Noun phrase consistency (46) 30. Informal usages (6)
14. Punctuation (44) 31. Very vs. very much (6)
15. Redundant expressions (39) 32. Pronoun errors (3)
16. Idiomatic expressions (32) 33. Homonyms (3)
17. Relative pronoun errors (32) 34. Double negative (1)
To see the distribution more clearly, Figure 8 presents the occurrence of the error feedback messages provided by human raters.
54 0
20 40 60 80 100 120 140 160 180
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 error types
number of errors
Figure 8. Occurrence of the Error Types Provided by Human Raters
In the following section, the top twenty error messages identified by human raters were examined and compared with the results provided by Correct English.
4.3.1 The analysis of the top twenty error types identified by human raters and the comparison with feedback provided by Correct English
In order to find out the differences of the results between Correct English and human raters, the top twenty error types identified by human raters were selected for further analysis and discussion.
1. Spelling errors: In this category, two human raters identified 168 errors, while Correct English provided 237 spelling errors (excluding the false alarms). It is obvious that Correct English performed better because teachers might be influenced by fatigue and thus were unable to single out all the errors.
2. Missing/ unnecessary / incorrect prepositions: Human raters identified 129 prepositions errors, whereas Correct English detected only 18 (excluding the false
55
alarms). This shows that the system was insensitive to the collocation of words and unable to provide proper feedback on prepositions.
3. Teacher’s rewrite: When there were too many errors in a sentence that would influence the readability of a sentence, the human raters rewrote the sentence and provided some suggestions for further revision. There were 127 errors belonging to the category of teacher’s rewrite. This category is quite unique because Correct
English did not give feedback on sentence level. Some examples are shown in Table
28. See Appendix for the complete list of teacher’s rewrite.Table 28
Examples of Teacher’s Rewrite by Human raters and the Comparison with Feedback by Correct English
1. And every year will publish the new commercial,
Teacher’s rewrite: Every year the company will release a new commercial,
Correct English: And <clause error> every year will publish the new commercial, 2. I saw a deep impression of advertisement.
Teacher’s rewrite: I saw an advertisement that impressed me the most.
Correct English: Not available 3. I just want to be a welcome man.
Teacher’s rewrite: I just want to be a man that is welcomed/ popular among people.
Correct English: Not available
4. The company is success to be remember by people.
Teacher’s rewrite: The company is successful because it’s easy to remember.
Correct English: The company is success to be remember <infinitive or -ing> by people.
5. Drinking it is just like becoming more thin.
Teacher’s rewrite: Drinking it makes you become thinner.
Correct English: Drinking it is just like becoming more thin <comparative>.
6. Also, they will be the best choice to take with you during the journey.
Teacher’s rewrite: They will be the best choice when it comes to a proper companion.
Correct English: Also <adverb placement>, they will be the best choice to take with you…
7. I am scared not knowing what to do.
Teacher’s rewrite: I am too scared to do anything.
Correct English: Not available
8. I am very like to listen music, no matter what kind of them.
Teacher’s rewrite: I like to listen to music very much, regardless of its type.
56
Correct English: I am very like to listen <missing prepositions (ESL)> music, no matter what kind of them.
9. I can relief my unease and make a memory in words.
Teacher’s rewrite: I can relieve my anxiety and turn memories into words.
Correct English: I can relief <word form> my unease and make a memory in words.
4. Verb tense: Human raters identified 119 errors in this category, while Correct English provided only 12 verb tense error messages. Among the 119 errors, many of them were related to the conditional sentences because one of the composition topics is “If you were to go abroad for a year, what would you bring along with you?” Many students failed to utilize the conditional sentences in their writing. Correct English seemed unable to judge whether conditional sentences were necessary or not in the context.
5. Verb group consistency: 87 errors were identified by the human raters, while
Correct English detected 32 errors (excluding the false alarms) in this category.
6. Missing/ unnecessary/ incorrect articles: Human raters pointed out 86 errors in articles, where as Correct English identified 29 errors (excluding the false alarms).
7. Nouns: mass or count: In this category, human raters identified 72 errors, while Correct English detected only 15 (excluding the false alarms).
8. Subject-Verb agreement: Human raters identified 68 subject-verb agreement errors, whereas Correct English provided 80 error messages (excluding the false alarms). In this category, Correct English performed slightly better than human raters.
9. Word confusion: In this category, human raters identified 67 errors, but
Correct English detected only 9 (excluding the false alarms). There are many easily
confused words when ESL learners write compositions. See Table 29 for some57
examples.
Table 29
Examples of Word Confusion Errors Identified by Human Raters
1.I am very diligent and patient, I am finding a job now (look for vs. find) 2. A real cow! Can you, can I, can we image? (imagine vs. image)
3. The picture showed that the cancer present can live happily like common people (patient vs. present) 4. because it was neutral, so it won’t hurt our hair. (natural vs. neutral)
5. Because of the beer has a great favor, (flavor vs. favor)
6. if you don’t have anyone who can company you. (accompany vs. company)
7. Thanks to the step counter, I would not gain much weight after I finish my journal. (journey vs. journal) 8. The travel maybe is happy, sad, interesting (may be vs. maybe)
9. I can enjoy the trip and never fell tedious. (feel vs. fell)
10. You just can’t forgive if and it let me want to drive. (forget vs. forgive)
10. Word form: In this category, 63 errors were identified by human raters, while Correct English detected only 14 (excluding the false alarms). The following words are some of the examples: success /successful, impressive/ impressed, lose/ lost, France/ French, and live/ life.
11. Misused words: Human raters identified 63 errors in this category, but
Correct English detected only two errors (excluding the false alarms). Some of the
examples are shown in Table 30.Table 30
Examples of Misused Words Detected by Human Raters
1. And during those time, maybe I have no one can depend on. (periods) 2. A digital camera records my past life and my now life.(present)
3. At this time, my small pillow can be a protector continuatively at every night. (again) 4. Mom can grasp my message so that she can relieve herself. (feel relieved)
5. So I always turned on the TV to the loudest, hoping to scare the quiet away. (silence)
12. Clause errors: In this category, human raters identified 53 errors, while
Correct English provided 86 clause error messages (excluding the false alarms).
Correct English seemed to do a better job for detecting fragment in sentences.
58
13. Noun phrase consistency: 46 errors were categorized as noun phrase consistency errors by human raters, and Correct English detected 35 errors (excluding the false alarms).
14. Punctuation: In this category, 44 errors were identified by human raters and 31 were detected by Correct English (excluding the false alarms).
15. Redundant expressions: 39 errors were regarded as redundant expressions by human raters, while Correct English detected only three errors (excluding the false alarms when there were words like the reason why).
16. Idiomatic expressions: Human raters identified 32 errors in this category, while Correct English only found four errors (excluding the false alarm like the word
alone). Some examples are shown in Table 31.
Table 31
Examples of Detection of Idiomatic Expressions Errors by Correct English and Human Raters
1. Fortunately, they finally met and love in the first sight Human raters: fell in love at the first sight
Correct English: love at the first sight
2. Many people called the number after watching the commercial because of curiosity.
Human raters: out of curiosity Correct English: not available
3. what the better is I can not only choose many channels I like Human raters: more importantly
Correct English: what the better <comparative> is I can not only choose many channels I like
17. Relative pronoun errors: Correct English did not have this category because relative pronoun errors were put into the category of pronoun errors. However, this category is one of the most common errors in learners’ writing. It is essential to specify relative pronouns for further revision. Human raters identified 32 relative pronoun errors. Some of the examples are shown in Table 32.
59
Table 32
Examples of Detection for Relative Pronoun Errors by Human raters
1. Evyeryone <relative pronouns> has watched this commercial has the same feeling (suggestion: who) 2. I could take pictures when I see a beautiful view or anything <relative pronouns> interested me.
(suggestion: that)
3. I always appreciate <relative pronouns> the elder had done for me (suggestion: what)
18. Word order: There were 27 errors identified by human raters, while Correct
English detected 10 errors (excluding the false alarms). Some examples are shown in
Table 33.Table 33
Examples of Detection for Word Order by Human raters and Correct English
1. They did not know how could they do and very anxious.
human raters: They did not know what they could do and were very anxious.
Correct English: They did not know how could they <word order> do and very anxious.
2. a dog which looked like just as mine
human raters: a dog which looked just like mine Correct English: not available
3. People may ask what could I do with the newspapers
human raters: People may ask what I could do with the newspapers
Correct English: People may ask what could I <word order> do with the newspapers
4. Her parents spent money on giving her anything they could and hard worked to make money
human raters: …and worked hard to make money Correct English: not available
19. Run-on sentences: Human raters identified 26 errors, while Correct English detected only ten errors. Some examples are shown in Table 34.
Table 34
Examples of Detection for Run-on Sentences by Human raters
1. He is willing to come to my aid whenever I am in trouble, I am glad that I have such an older brother.
2. All of these will be in my collection book, by doing so , I can recall my memory when I go back Taiwan.
3. I have to live in foreign country one year therefore I bring my iphone together .
60
20. Unclear meaning: The sentences which were unable to be understood or rewritten would be put into this category. Some of the errors are probably due to Chinese English. There were 26 errors identified by human raters. Some examples are shown in Table 35.
Table 35
Examples of Detection for Unclear Meaning by Human raters
1. Therefore, I bring my favorite coffee pig doll with me to travel other national such as homelike.
2. They sing for mother, sing for the nutrition.
3. A digital camera lets world master my hand.
4.3.2 Comparison of the top twenty error feedback types provided by human raters and Correct English
If we see the top twenty error types provided by Correct English and human raters listed in Table 36, it can be found that they focused on different types of errors.
Table 36. The Top Twenty Types and Numbers of Error Messages Identified by Correct English and Human Raters
Error Types of Correct English Error Types of Human raters
1. Spelling errors (256) 1. Spelling errors (168)
2. Clause errors (152) 2. Missing/ unnecessary/ incorrect prepositions (129) 3. Subject -Verb agreement (104) 3. Teacher's rewrite (127)
4. Word form (62) 4. Verb tense (119)
5. Punctuation (ESL) (46) 5. Verb group consistency (87)
6. Noun phrase consistency (40) 6. Missing/ unnecessary/ incorrect articles (86) 7. Infinitive or -ing form (35) 7. Nouns: mass or count (72)
8. Verb group consistency (35) 8. Subject-Verb agreement (68)
9. Weak / non-standard modifiers (34) 9. Word confusion (67)
10. Adverb placement (33) 10. Word form (63)
11. Capitalization (31) 11. Misused words (63)
12. Missing / unnecessary/ incorrect articles (30) 12. Clause errors (53)
13. Wordy expressions (26) 13. Noun phrase consistency (46)
14. Redundant expressions (25) 14. Punctuation (44)
15. A vs. An (24) 15. Redundant expressions (39)
16. Vague quantifiers (20) 16. Idiomatic expressions (32)
17. Missing/ unnecessary / incorrect prepositions(ESL) (19) 17. Relative pronoun errors (32)
18. Nouns: mass or count (17) 18. Word order (27)
19. Open vs. closed spelling (16) 19. Run- on sentences (26)
20. Word confusion (14) 20. Unclear meaning (26)
61
Both Correct English and human raters identified most errors in the category of spelling, which indicated that spelling accuracy is considered the most important part in writing. For the rest categories, there were great variations. For example, human raters identified many errors in word confusion, verb tense, misused words and countable or uncountable nouns, but the number of errors detected by Correct English was quite few. Besides, human raters rewrote the sentences to make the essays more fluent and readable, but most of the rewritten sentences were not identified as errors by Correct English. It is apparent that Correct English could check the basic elements in writing but was unable to provided further suggestions for revision.
4.3.3 Discussion of the error feedback messages provided by human raters and Correct English
Since the error feedback types provided by Correct English and human raters are different, it is not easy to compare the results directly. The error types both in
Correct English and human raters are first examined. It is found that there are 27
types shared by Correct English and human raters. Table 37 summarizes the 27 types of error feedback.Among these 27 categories, Correct English detected more errors (114 errors) than human raters in the following three—spelling, subject-verb agreement and clause errors. Their accuracy rates are 92%, 56% and 77% respectively. Besides, Correct
English and human raters identified the same number of errors in the following three
categories—a vs. an, pronoun errors, and compounding errors.62
Table 37. The Number and Error Types Shared by Correct English and Human Raters
Error Types No. of Errors by Correct English8
10. Missing / unnecessary/ incorrect articles 29 86
11. Redundant expressions 3 39
12. A vs. An 22 22
13. Missing/ unnecessary/ incorrect prepositions(ESL) 18 129
14. Nouns: mass or count 2 72
26. Comparative / superlative 4 11
27. Double negative 3 1
Total 712 1281
In the other 21 categories, human raters identified more errors than Correct
English, especially in the following six—word form, article errors, preposition errors,
noun: mass or count, word confusion and verb tense, which shows that the system could not identify those common errors produced by students. In sum, CorrectEnglish detected 712 errors, while human raters identified 1281 errors. That is, human
raters identified 569 more errors than Correct English. It is obvious that CorrectEnglish failed to recognize some common errors in students’ compositions.
There are still seven types of errors that were not provided in Correct English:
8 The number of errors by Correct English here has already excluded the false alarms confirmed by the researcher.
63
teacher’s rewrite, relative pronouns, unclear meaning, causative verbs, pronoun reference, possessive errors, and very vs. very much. In these seven categories, teacher’s rewrite is what Correct English did not provide, but it is very useful when learners need to do revision. As for pronoun reference and very vs. very much, both are common errors in ESL learners’ writing, but they were not detected by Correct
English either. One example of the pronoun reference errors is as follow, “I think I’ll
take my dog with me. Going on a journey alone may be a boring thing if you don’t have anyone who can accompany you.”Human raters put more emphasis on collocation, context and meaning of the compositions. Therefore, there were more errors identified in the categories of verb tense, word confusion, preposition errors and teacher’s rewrite, etc. Although Correct