This study aimed to explore student responses to different types of feedback for writing, as well as the unintended consequences of the use of automated writing evaluation. The primary findings can be delineated into three main observations. First, students consistently preferred teacher feedback to either peer or AWE feedback, suggesting that the teacher’s role in the writing classroom is still
27
viewed as central. Second, although students tended to prefer peer feedback to AWE feedback, they nonetheless perceived the potential shortcomings of peer feedback, most particularly regarding the willingness—or lack thereof—of peers to provide feedback which might be perceived as negative or critical. However, they adopted AWE feedback more frequently than peer feedback when making revisions, and regarded the ubiquitous feature of AWE as helpful in their autonomous learning. Finally, students did not experience writing “anxiety” any more strongly than they experienced other emotions. If anything, they expressed at least equal levels of positive curiosity and of being disheartened as they did of anxiety.
The limitations of this study can be summarized as follows.
First, this study involved two writing classes, taught by different instructors. Although the teachers of each class followed the same curriculum, differences in teaching styles are almost certainly inevitable, and so it could be argued that these differences could not be completely ruled out as a factor which could affect the results.
However, in order to minimize the likelihood of this, a t-test was run on the data from the two classes, and no significant differences were found.
Second, only self-reported emotions based on the questionnaire were investigated. Actual behavior or other emotions experienced while using the software were not examined. Still, within this limitation were steps taken to provide triangulation. Since this study is predominantly a quantitative exploration of the students’ affective responses to an AWE system in English writing feedback, the researchers included an open-ended question in the questionnaire and a small-scale follow-up interview to probe more in-depth reflections
28
of the students’ emotions. It is hoped that this has helped to provide a fuller, more nuanced analysis, if not a complete analysis.
The implications of the study are as follows. First, teachers who hope to increase the use of peer and/or AWE feedback in the writing classroom need to pay more attention to how to raise the profile of each type of feedback. For instance, instead of expecting peer and AWE feedback to meet with students’ approval in all areas, it might be more realistic to focus on areas where that feedback is felt to be more helpful, such as using AWE feedback to address certain form-related areas. As C.-F. Chen and Cheng (2008) pinpointed, despite preferences for teacher feedback, the ways in which teachers integrate AWE into the writing class can influence student perception of AWE and help make them more accepting of its contribution. Second, peer feedback may require more coaching by teachers. In particular, students need help in giving constructive criticism to their peers in a manner that does not come across as overly negative. Hu and Lam (2010) also emphasized that certain training procedures should be provided to help students increase the provision and incorporation of valid suggestions addressing peer feedback. Otherwise, students will tend to be reticent in offering critical feedback. Finally, teachers need to beware of exaggerating the role played by any single affective reaction in the classroom. Teachers should better understand the complicated emotions which arise in students during the process of computer assisted learning so as to facilitate effective learning outcomes. In cases where there is a complex interplay of positive and negative emotions—such as curiosity and sadness in this study—
awareness can aid teachers who want to build on the strengths of positive emotions and mitigate the effects of negative emotions.
29
Based on the findings of this study, there are also some suggestions for software designers. The assistance and assessment functions of AWE are still being developed and have not yet reached maturity (C.-F. Chen & Cheng, 2008; H. J. Chen et al., 2009). Indeed, students in this study expressed negative feelings towards the vagueness of the machine feedback. Consequently, more effort should be spent on improving the reliability and validity of the scoring process. Moreover, students in this study reported feeling disheartened and dispirited when using Criterion. Software designers, therefore, should consider how the instructional and user-interface designs of AWE help students. Specifically, perhaps designers could consider taking more cues from the types of technological devices and applications that students make use of in their daily lives, so as to ease the transition into using these tools in an educational setting and to squarely address the motivation issues. If such changes were made, students would be in a much better position to make use of the AWE software, especially in their individual writing. In this way, AWE could better promote learner autonomy in EFL writing (Benson, 1997, 2001).
Suggestions for future studies include a more in-depth review of emotions which arise during the process of computer assisted learning. Obtaining contemporaneous information will require identifying how to access students’ emotional reactions as they write and receive feedback. Weekly journals, for example, could be adopted to collect information about the emotions involved in a specific writing activity and their sources. Future studies could also focus on the timing of teacher intervention (Wosnitza & Volet, 2005) after identifying students’ negative affective reactions. Timely adjustment
30
of teaching strategies and appropriate scaffolding could help students conquer their negative affective reactions and better allow them to benefit from computer-assisted learning. Last but not least, due to the anonymous nature of the survey, we did not explore the relationship between learner emotion and performance. Relevant studies should be conducted to reveal how emotions influence the learning process and learning outcomes so that teachers can learn how to adopt software for EFL writing in an optimal way.
ACKNOWLEDGMENT
This paper is partially supported by a grant from the National Science Council in Taiwan (NSC 100-2511-S-328-001). The authors would like to thank the students who participated in this project, and of course special thanks are in order for the reviewers and the editor for their insightful comments and valuable feedback.
REFERENCES
Bailey, P., Daley, C. E., & Onwuegbuzie, A. J. (1999). Foreign language anxiety and learning style. Foreign Language Annals, 32, 63-76.
Benson, P. (1997). The philosophy and politics of learner autonomy.
In P. Benson, & P. Voller (Eds.), Autonomy and independence in language learning (pp. 18-34). London: Longman.
Benson, P. (2001). Teaching and researching autonomy in language learning. London: Longman.
31
Chen, C.-F., & Cheng, W.-Y. (2006, May). The use of a computer-based writing program: Facilitation or frustration? Paper presented at the 23rd International Conference on English Teaching and Learning in the Republic of China, Kaohsiung, Taiwan. Retrieved March 12, 2010, from http://www2.nkfust.
edu.tw/~emchen/Home/Chen%20Papers/Computer-based%20 writing%20program_paper.pdf
Chen, C.-F., & Cheng, W.-Y. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2), 94-112.
Chen, C. M., & Wang, H. P. (2011). Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance. Library &
Information Science Research, 33, 244-255.
Chen, H. J. (2006, December). Examining the scoring mechanism and feedback quality of MyAccess. Paper presented at Tamkang International Conference on Second Language Writing, New Taipei City, Taiwan.
Chen, H. J., Chiu, S.-T., & Liao, P. (2009). Analyzing the grammar feedback of two automated writing evaluation systems: My Access and Criterion. English Teaching & Learning, 33(2), 1-43.
Cheng, Y.-S. (2004). EFL students’ writing anxiety: Sources and implications. English Teaching & Learning, 29(2), 41-62.
Cross, J., & Hitchcock, R. (2006, July). Differences, difficulties and benefits: Chinese students’ views of UK HE. Paper presented at the 2nd Biennial International Conference, Portsmouth, UK.
32
Dunkel, P. (1990). Implications of the CAI effectiveness research for limited English proficient learners. Computers in the Schools, 7(1-2), 31-52.
Erkan, D. Y., & Saban, A. I. (2011). Writing performance relative to writing apprehension, self-efficacy in writing, and attitudes towards writing: A correlational study in Turkish tertiary-level EFL. Asian EFL Journal, 12(1), 164-192.
Fang, Y. (2010). Perceptions of the computer-assisted writing program among EFL college learners. Educational Technology
& Society, 13, 246-256.
Guénette, D. (2007). Is feedback pedagogically correct? Research design issues in studies of feedback on writing. Journal of Second Language Writing, 16, 40-53.
Hay, V., & Isbell, D. (2000). Parallel on-line and in-class sections of
“writing for the professions”: A practical experiment.
Educational Technology & Society, 3, 308-316.
Hu, G. W., & Lam, S. T. E. (2010). Issues of cultural appropriateness and pedagogical efficacy: Exploring peer review in a second language writing class. Instructional Science, 38, 371-394.
Järvelä, S., Hurme, T. R., & Järvenoja, H. (2010). Self-regulation and motivation in computer-supported learning environments. In S.
Ludvigsen, A. Lund, I. Rasmussen, & R. Säljö (Eds.), Learning across sites: New tools, infrastructures and practices (pp. 330-345). New York: Routledge.
Järvelä, S., Lehtinen, E., & Salonen, P. (2000). Socioemotional orientation as a mediating variable in teaching learning interaction: Implications for instructional design. Scandinavian Journal of Educational Research, 44, 293-306.
33
Järvelä, S., Veermans, M., & Leinonen, P. (2008). Investigating students’ engagement computer-supported inquiry—A process-oriented analysis. Social Psychology in Education, 11, 299-322.
Järvenoja, H., & Järvelä, S. (2005). How students describe the sources of their emotional and motivational experiences during the learning process: A qualitative approach. Learning and Instruction, 15, 465-480.
Kay, R. H., & Loverock, S. (2008). Assessing emotions related to learning new software: The computer emotions scale.
Computers in Human Behavior, 24, 1605-1623.
Lai, Y. H. (2010). Which do students prefer to evaluate their essays:
Peers or computer program. British Journal of Educational Technology, 41, 432-454.
Min, H.-T. (2006). The effects of trained peer review on EFL students’ revision types and writing quality. Journal of Second Language Writing, 15, 118-141.
Nummenmaa, M. (2007). Emotions in a web-based learning environment. Retrieved 30 August, 2011, from: https://www.
doria.fi/bitstream/handle/10024/27232/B304.pdf?sequence=1 Nunnally, J., & Bernstein, I. (1994). Psychometric theory. New York:
McGraw-Hill Humanities.
Otoshi, J. (2005). An analysis of the use of Criterion in a writing classroom in Japan. The JALT CALL Journal, 1(1), 30-38.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement:
A program of qualitative and quantitative research. Educational Psychologist, 37, 91-105.
34
Schweiker-Marra, K. E., & Marra, W. T. (2000). Investigating the effects of prewriting activities on writing performance and anxiety of at-risk students. Reading Psychology, 21, 99-114.
Tsai, P. C., & Cheng, Y.-S. (2009). The effects of rhetorical task type, English proficiency, and writing anxiety on senior high school students’ English writing performance. English Teaching &
Learning, 33, 95-131.
Volet, S. E., & Järvelä, S. (Eds.). (2001). Motivation in learning contexts: Theoretical advances and methodological implications. Amsterdam: Elsevier Science.
Wosnitza, M., & Volet, S. (2005). Origin, direction and impact of emotions in social online learning. Learning and Instruction, 15, 449-464.
Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback in a Chinese EFL writing class. Journal of Second Language Writing, 15, 179-200.
Yang, N. D. (2004, March). Using My Access in EFL writing. Paper presented at the 2004 International Conference and Workshop on TEFL & Applied Linguistics, Taipei, Taiwan.
Yeh, Y., Liou, H. C., & Yu, Y. T. (2007). The influence of automated essay evaluation and bilingual concordancing on EFL students’
writing. English Teaching & Learning, 31, 117-160.
Zhang, S. (1995). Reexamining the affective advantage of peer feedback in the ESL writing class. Journal of Second Language Writing, 4, 209-222.
35