The thesis proposed a novel methodology for scoring Chinese essays based on the extraction and analysis of conceptual frameworks in essays. Experimental results show that the performance of the methodology is quite close to that of current English AES systems. There are three characteristics in the methodology. First, it performs well based on the analysis of semantics in essays even if it does not employ surface features and syntax features. Second, the result of evaluation can be used for instructional feedback to the authors because it refers the conceptualization progress of authors to evaluate the quality of essays. Third, it overcomes the difficulties of applying current English AES systems to Chinese.
Many further studies can be developed based on the proposed methodology. T First, various figures of speech should be explored and employed to increase the performance of scoring essays. This thesis has shown the usefulness of figures of speech for scoring essays. The exaction of more figures of speech may be useful for scoring essays and instructional feedback. Second, the e-learning system for writing instruction based on the proposed methodology will be very useful for education.
Today, students cannot practice writing frequently because of lack of human raters.
Using the interactive learning systems based on the proposed methodology, students can improve their writing skills fast and effectively.
Reference
[1] Y. Attali and J. Burstein, “Automated Scoring Using With e-raterV.2.The Journal of Technology,”
Learning and Assessment, vol. 4,no. 3, 2006.
[2] M. H. Bai, C. J. Chen, and K. J. Chen, “Category Determination for Chinese Unknown Words Using Contextual Rules,” Proceedings of Research on Computational Linguistics Conference XI, pp.
253-272, Taiwan, 1998.
[3] S. T. Bai and A. W. Shi, ”A Comparative Study of Figures of Speech between Chinese and English,”
Journal of Xinzhou Teachers University, vol. 18, no. 1, pp. 70-71, 2002.
[4] J. Burstein, K. Kukich, S. Wolff, C. Lut, M. Chodorow, L. Braden-Harder and M. Dee Harri,
“Automated Scoring Using A Hybrid Feature Identification Technique,” Proceedings of the 36th Annual Meeting of the Association of Computational Linguistics, Montreal, Canada, 1998.
[5] J. G.. Cai, “A contrastive study of English and Chinese writing,” Fudan University Press. Shanghai:
China, 2001.
[6] J. G.. Cai, “Contrastive study of writing & rhetoric in English and Chinese,” Fudan University Press, Shanghai: China, 2003.
[7] T. H. Chang and C. H. Lee, “Topic Segmentation for Short Texts,” Proceedings of the 17th Pacific Asia Conference on Language, Information, and Computation, Singapore, 2003.
[8] T. H. Chang and C. H. Lee, “Automatic Chinese Unknown Word Extraction Using Small-Corpus-based Method,” Proceedings of IEEE International Conference on Natural language processing and knowledge engineering, pp. 459-464, Beijing, China, 2003.
[9] T. H. Chang and C. H. Lee, “Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech,” Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation, pp. 28-35, Wuhan, China, 2006.
[10] C. C. Chen, “A Study on Procedural Knowledge Learning for Mandarin Rhetoric Teaching,” Master Thesis, Graduate Institute of Taiwan Languages and Language Education, National Hsinchu University of Education, Hsinchu, Taiwan, 2004.
[11] C. J. Chen, M. H. Bai, and K. J. Chen, “Category Guessing for Chinese Unknown Words.”
Proceedings of the 4th Natural Language Processing Pacific Rim Symposium, pp. 35-40, Thailand, 1997.
[12] H. J. Chen, “Analysis of Students Use Figure-of-speech “Pi-yu,” Newsletter for Languages and Literature Education, vo. 15, pp. 48-55, 1997.
[13] K. J. Chen and M. H. Bai. 1998. “Unknown Word Detection for Chinese by a Corpus-based Learning Method,” Computational Linguistics and Chinese Languages Processing, vol. 3, no. 1, pp. 27-44, 1998.
[14] K. J. Chen and W. Y. Ma, “Unknown Word Extraction for Chinese Documents,” Proceeding of the 19th International Conference on Computational Linguistics, pp. 169-175, Taiwan, 2002.
[15] CKIP, “Chinese Part-of-speech Analysis,” Technical Report 93-05, Academia Sinica, Taipei, 1993.
[16] S. Dikli, “An Overview of Automated Scoring of Essays,” The Journal of Technology, Learning, and Assessment. vol. 5, no. 1, 2006.
[17] S. M. Elliot, “IntelliMetric: From Here to Validity,” Proceedings of the annual meeting of the American Educational Research Association, Seattle, WA, 2001.
[18] M. Hearst, “The Debate on Automated Essay Grading,” IEEE Intelligent Systems, vol. 15, no. 5, pp.
22-37. 2003.
[19] S. He and J. Zhu, “Bootstrap method for Chinese new words extraction,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 581-584, Salt Lake City, Utah, USA, 2001.
[20] H. J. Hsu and C. C. Wang, “The Effect of Reading-and-Writing Rhetoric Instructive Method on the Ability of Elementary Students’ Writing Rhetoric Manner,” Proceedings of Conference on Children's Languages and Literature Education, pp. 341-362, Taitung, Taiwan, 1999.
[21] Z. L. Hu, “Application of Discourse Analysis in Language Teaching,” Foreign Language Education.
vol. 22, no. 1, pp. 3-10, 2001.
[22] L. C. Huang, “Figure-of-speech “Pai-bi” (I),” Chinese Language Monthly. vol. 491, pp. 19-24, 1998.
[23] T. T. Huang, “Ten Figures-of-Speech Instruction Using Syntax,” Elementary Education Century, vol.
215, pp. 79-88, 2005.
[24] C. Y. Jiao, “A syntactic comparison and transformation between English and Chinese,” Journal of Yancheng Teachers College. vol. 22, no. 2, pp. 83-87, 2002.
[25] R. B. Kaplan, “Cultural Thought Patterns in Intercultural Education,” Language Learning, vol. 16, pp.
1-20, 1966.
[26] H. W. Ko, “The Evaluation Criteria of Expository and Narrative Writings,” Journal of Chinese Language Teaching, vol. 1, no. 2, pp. 15-32, 2004.
[27] Y. Ko, J. Park and J. Seo, “Improving text categorization using the importance of sentences,”
Information Processing and Management, vol. 40, no. 1, pp. 65-79, 2004.
[28] Y. S. Lai and C. H. Wu, “Meaningful term extraction and discriminative term selection in text categorization via unknown-word methodology,” ACM Trans. on Asian Language Information Processing, vol. 1, no. 1, pp. 34-64, 2002.
[29] W. Lam and K. S. Ho. “FIDS: an intelligent financial Web news articles digest system,” IEEE Trans.
on Systems, Man and Cybernetics: Part A, vol. 31, no. 6, pp. 753-762, 2001.
[30] T. K. Landauer, D. Laham and P. W. Foltz, “The Intelligent Essay Assessor,” IEEE Intelligent System, vol. 15, pp. 27-31, 2000.
[31] L. S. Larkey and W.B. Croft, “A Text Categorization Approach to Automated Essay Grading.
Automated Essay Scoring: A Cross-Disciplinary Perspective,” Lawrence Erlbaum Associates Inc., Mahwah New Jersey, pp. 55-69, 2003.
[32] G.. N. Lee, “Contrastive Studies of Figures of Speech in English and Chinese,” Fujian People’s Publishing House: Fuzhou, China, 1999.
[33] X. L. Lee and K. Zeng, “Heterogeneity and Homogeneity of Sentence Structure in English and
Chinese,” Journal of Shenyang University. vol. 12, no. 1, pp. 52-55, 2001.
[34] W. S. Li and D.Agrawal, “Supporting web query expansion efficiently using multi-granularity indexing and query processing,” Data and Knowledge Engineering, vol. 35, no. 3, pp.239-257, 2000.
[35] G. C. Ling, M. Asahara, and Y. Matsumoto, “Chinese Unknown Word Identification Using Character-based Tagging and Chunking,” Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, p.p. 197-200, Sapporo, Japan, 2003.
[36] L. J. Liu, “A contrastive study of Discourse Structure in English and Chinese,” Modern Foreign Languages, vol. 86, no. 4, pp. 408-419, 1999.
[37] K. Y. Liu and J. H. Zheng, “Research of Automatic Chinese Word Segmentation,” Proceedings of the 1st International Conference on Machine Learning and Cybernetics, vol. 2, pp. 805-809, Beijing, China, 2002.
[38] W. Y. Ma and K. J. Chen, “Introduction to CKIP Chinese Word Segmentation System for the First International Chinese Word Segmentation Bakeoff,” 2nd SIGHAN Workshop on Chinese Language Processing, Sapporo, Japan, 2003.
[39] R. Mandala, T. Tokunaga, and H. Tanaka, “Query expansion using heterogeneous thesauri,”
Information Processing & Management, vol. 36, no. 3, pp. 361-378, 2000.
[40] C. D. Manning and H. Schutze, “Foundations of statistical natural language processing,”
Massachusetts: MIT Press, 1999.
[41] A. Mikheev, “Automatic rule induction for unknown word guessing,” Computational Linguistics, Vol.
23, no. 3, pp. 405-423, 1997.
[42] M. F. Moens and R. de Busser, “Generic Topic Segmentation of Document Texts,” Proceedings of the 24th ACM SIGIR Annual International Conference on Research and Development in Information Retrieval, pp. 418-419, 2001.
[43] T Nakagawa, T. Kudoh, and Y. Matsumoto, “Unknown word guessing and part-of-speech tagging using support vector machines,” Proceedings of the 6th Natural Language Processing Pacific-Rim Symposium, pp. 325-331, Tokyo, Japan, 2001.
[44] E. B. Page, “Computer Grading of Student Prose, Using Modern Concepts and Software,” Journal of Experimental Education, vol. 67, pp. 127-142, 1994.
[45] J. M. Ponte and W. B Croft. “Text segmentation by topic,” Proceedings of the 1st European Conference on Research and Advanced Technology for Digital Libraries. pp. 120-129. Pisa, Italy, 1997.
[46] L. M. Runder and T. Liang, “Automated Essay Scoring Using Bayes’ Theorem,” The Journal of Technology, Learning, and Assessment, vol. 1, no.2, 2002.
[47] R. Scollon, S. W. Scollon, and A. Kirkpatrick, ”Contrastive Discourse in Chinese and English: A Critical Appraisal,” Foreign Language Teaching and Research Press: Shaihai, China, 2000.
[48] M. D. Shermis, and J. C. Burstein (Eds), “Automated Essay Scoring: A Cross Disciplinary Perspective,” Lawrence Erlbaum Associates Inc.: Mahwah New Jersey, 2003.
[49] M. S. Sun, D. Y. Shen, and C. N. Huang, “CSeg&Tag1.0: A practical word segmenter and POS tagger for Chinese texts,” Proceedings of the 5th Conference on Applied Natural Language Processing, pp.
119-126, Washington DC, USA, 1997.
[50] S. M. Thede, “Predicting part-of-speech information about unknown words using statistical methods,”
Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics, pp.
1505-1507, Montréal, Canada, 1998.
[51] M. F. Tsai, “Figures-of-speech pi-yu, bi-ni and zhuan-hua”, World of Chinese Language and Literature, vol. 16, no. 9, pp. 81-87, 2001.
[52] T. Y. Tsai, “Rhetoric Theory and Writing Teaching,” Newsletter for Teaching the Humanities and Social Sciences, vol. 9, no. 3, pp. 52-62, 1998.
[53] H. H. Tseng, C. L. Liu, Z. M. Gao, and K. J. Chen, “A hybrid approach for automatic classification of Chinese unknown verbs,” Computational Linguistics and Chinese Language Processing, vol. 7, no. 1, p.p. 1-28, 2002.
[54] S. Valenti, F. Neri, and A. Cucchiarelli, “An overview of current research on automated essay grading,” Journal of Information Technology Education, vol. 2, pp. 319-330, 2003.
[55] J. P. Yamron, I. Carp, L. Gillick, S. Lowe and P. van Mulbregt, “A hidden Markov model approach to text segmentation and event tracking,” Proceedings of the 1998 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 333-336, 1998.
[56] R. Baeza-Yates and B. Ribeiro-Neto, Modern information retrieval. ACM Press, New York, 1999.
[57] H. P. Zhang, Q. Liu, H. Zhang, and X. Cheng, “Automatic recognition of Chinese unknown words based on roles tagging,” Proceedings of the 1st SIGHAN Workshop on Chinese Language Processing, pp. 71-77, Taiwan, 2002.
[58] X. H. Zeng, “Enhancing English Writing Ability by Comparing the Difference of Organization in Paragraphs between English and Chinese,” Journal of Nanchang Vocation-technical Teachers College, vol. 4, pp 75-77. 1997.