Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=caeh20
Download by: [National Chengchi University] Date: 05 July 2016, At: 16:59
Assessment & Evaluation in Higher Education
ISSN: 0260-2938 (Print) 1469-297X (Online) Journal homepage: http://www.tandfonline.com/loi/caeh20
Understanding learners’ self-assessment and
self-feedback on their foreign language speaking
performance
Shu-Chen Huang
To cite this article: Shu-Chen Huang (2016) Understanding learners’ assessment and
self-feedback on their foreign language speaking performance, Assessment & Evaluation in Higher Education, 41:6, 803-820, DOI: 10.1080/02602938.2015.1042426
To link to this article: http://dx.doi.org/10.1080/02602938.2015.1042426
Published online: 21 May 2015.
Submit your article to this journal
Article views: 204
View related articles
View Crossmark data
Understanding learners
’ self-assessment and self-feedback on their
foreign language speaking performance
Shu-Chen Huang*
Foreign Language Center, National Chengchi University, Taipei, Taiwan
This study examines university learners’ self-assessment and self-feedback on performance as captured in audiofiles from a foreign language speaking test. The learners were guided to listen, transcribe and analyse their own speaking samples, as well as propose future actions for improvement. Content of learners’ self-feedback was scrutinised against a feedback model, with data coded into various feedback categories as stipulated in the model for analysis. Results indicated that learners’ self-feedback was far reaching and multifaceted. Through self-feedback, learners identified discrepancies, answered feed up, feedback and feed forward questions, and inspected performance at task, process, self-regulation and self levels. Much of the feedback involved reflections on past learning history, other areas of learning, deviation of performance from prepara-tion and learner personality traits. The self-feedback went largely beyond most teachers’ feedback capacity and bore great potential for learning and instruction. In particular, contrary to theoretical presumptions, self-level feedback seemed quite enlightening. Whether the observed quality self-feedback could actually help learners improve their performance, however, was not clear. It was suggested that some teachers’ time and effort be directed to the endeavour of facilitating learners’ self-assessment and self-feedback. Learners’ self-feedback capability should also be explored further in the development of relevant pedagogies and theories.
Keywords: self-assessment; self-feedback; foreign language speaking
Introduction
Earlier self-assessment studies focus mostly on the correctness of learner judgment (Boud and Falchikov1989), with teacher marks as the major yardstick of examination. Results indicate that more advanced learners are usually more accurate (Boud and Falchikov1989; Lew, Alwis, and Schmidt2010). Self-assessment has also been shown as a good predictor of information recall in foreign language reading (Brantmeier
2005). In addition to reading, the more transient skill of speaking could also be self-assessed by elementary (Butler and Lee2006) and tertiary learners (Chen2008) alike, with on-task assessment permitting more accuracy than the off-task condition. Interestingly, cultural background has a role to play, too. For example, Japanese learners, possibly owing to their high value for modesty as a virtue, tend to be more severe in judging their own writing than judging that of their peers (Matsuno2009).
Learners’ self-assessment, however, is not just a matter of accuracy. Some researchers are interested in the power relationship inherent in the act of transferring
*Email:huang91@nccu.edu.tw
© 2015 Taylor & Francis
Vol. 41, No. 6, 803–820, http://dx.doi.org/10.1080/02602938.2015.1042426
agency from the teacher to the learner, and have investigated the potential of self-assessment activities in empowering learners (Milne 2009; Tan 2004). Yet, the core of learner self-assessment studies is not on assessment itself, nor on correctness or power, but on the facilitation and development of learners’ learning through engaging in self-assessment. As defined by Boud (1995), self-assessment is the act of judging oneself and making decisions about the next step by asking a series of questions– ‘How am I doing? Is this enough? Is this right? How can I tell? Should I go further?’ By answering these questions through self-assessment, learners are learning to self-regulate and gradually taking over the ownership of their learning.
Self-assessment for the purpose of enhancing learning
Self-assessment has been considered an important part of self-regulation (Butler and Winne1995). It is also an ability with which learners can benefit from instructional input they receive. Orsmond and Merry (2013) found that higher achievers, because of their better self-assessment ability, gained more from tutor feedback than did their peers. Rivers (2001) studied mature and successful foreign language learners, and dis-covered that these learners regularly engaged in self-assessment. English learners in Korea enhanced performance and confidence by conducting self-assessment (Butler and Lee 2010). College French learners in America obtained pedagogical benefits from self-assessment at both cognitive and affective levels (de Saint Léger2009).
With the advantages of self-assessment widely recognised, some studies look at how self-assessment could be better designed and carried out. For example, Andrade (2008) suggests that judgmental marking should be removed from self-assessment, so learners can focus on constructive comments. She emphasised that this simple removal of scores, although subtle as it appears, has a powerful impact. When scores and judgments are absent and the focus lies on answers to the series of questions in Boud’s definition of self-assessment (1995), the fine line between assessment and feedback blurs. Self-assessment and self-feedback can hardly be separated, as implied in the definition of self-assessment provided by Hattie and Timperley (2007, 94): ‘a self-regulatory proficiency that is powerful in selecting and interpreting information in ways that provide feedback’. Consequently, the focal point becomes the content of feedback that learners generate based on judgments, not their judg-ments themselves. In this sense, correctness studies of self-assessment do not inform. But to date, beyond judging and correctness, little is known about the nature and characteristics of learners’ self-assessment and self-feedback. Without a better understanding of what learners see and think in conducting self-assessment and generating self-feedback, our understanding of the potential of formative assessment is limited, and relevant theories and pedagogies are incomplete.
Peer assessment and teacher feedback studies
In the dearth of learner self-assessment and self-feedback research, two types of studies are relevant and informative. One is peer assessment and the other teacher feedback studies. Other than whether assessment is correct or how peers interact when they assess each other, peer assessment is usually discussed together with self-assessment/self-feedback (Orsmond, Merry, and Callaghan2004), and is a means to other ends, rather than an end itself. Those other ends include facilitating learner understanding of complex work and pertinent criteria (McConlogue 2012), and, as
is often the case, making learners better self-assessors. As demonstrated by Nicol, Thomson, and Breslin (2014), learners benefited more in actively furnishing feed-back comments for their peers than passively receiving peers’ feedback. It was through performing the task themselvesfirst, and subsequently evaluating peer work against the set of criteria already learned, that made them assess their own work more critically and professionally, which eventually contributed to the improvement of learners’ own work.
On the other hand, learners’ self-assessment/self-feedback could also be informed by research in teachers’ assessment feedback. Studies on formative assessment or assessment for learning have in recent years focused more specifically on feedback, probably because of feedback’s potentiality in taking learners further on the learning journey after assessment. Dissatisfaction with teacher feedback in higher education became an issue. There were then many endeavours trying to identify possible prob-lems. Some attributed it to teacher misperception of and their lack of readiness for giving quality feedback (Bailey and Garner 2010; Lee 2009). Others demonstrated the misalignment in perception and feedback provision between tutors and learners (Orsmond and Merry 2011). Still others argued that the trouble did not lie as much in the content as in the delivery of feedback. For example, Wingate (2010) pointed to tone, style and excessive amount of feedback as factors preventing learners of low motivation and low self-perception from taking advantage of teacher feedback. Furthermore, Price et al. (2010) analysed the underlying complexity of higher educa-tion and pinpointed the near impossibility of adequately evaluating the effectiveness of teacher feedback.
More recent studies on solutions of the problem seem to point in a similar direc-tion, that is, the involvement of learners in the feedback process. Orsmond and Merry (2013) believe that in the past, our understanding and pedagogy of feedback have been tilted and too heavily teacher oriented, neglecting aspects on the learner side. Many other researchers contend too that, instead of one-way teacher mono-logues, feedback should be provided, responded to, adjusted and reflected to form a cycle of teacher–learner dialogues (Beaumont, O’Doherty, and Shannon 2011; Bloxham and Campbell2010; Yang and Carless2013). Moreover, Boud and Molloy (2012) assert that, in order for feedback to be effective for learning, our understand-ing of teacher feedback should shift from the habitual conception of teacher tellunderstand-ing to one that depicts students actively seeking.
All the aforementioned literature implies a belief that learners have the ability to actively contribute to the assessment and feedback process. Despite the proliferation of formative assessment and teacher feedback studies, however, the nature of learner self-assessment/self-feedback is largely unknown. Not much has been reported about what learners could do when they are given a chance to assess their own perfor-mance and provide feedback accordingly. Without a qualitative understanding, i.e. how they self-assess and what they attend to in real learning situations, further learner self-assessment/self-feedback studies may not be well supported.
Feedback models
Models of feedback have been proposed and modified to reflect development in educational research. Although none focuses particularly on learner-generated self-assessment and self-feedback, these models provide a background for understanding self-feedback from learners’ self-assessment.
Butler and Winne (1995) proposed to integrate feedback into a self-regulation model. Their model has similar underpinnings to Black and Wiliam’s (2009) frame-work of assessment for learning, in that both regard the discrepancy between goals and the current state of performance as a critical starting point for learning. With careful monitoring or assessment, the gap is identified. With effective processing of feedback obtained which points to appropriate tactics and strategies, the identified gap can be bridged. As Butler and Winne synthesised from empirical studies, when the level of discrepancy perceived by the learner is high relative to expectation, it has a stronger impact on follow-up learning. Moreover, effects of external feedback are too oftenfiltered by learners’ existing beliefs, causing constructive external feed-back to be ignored or rejected. More than changing the products of learning with summative outcome feedback, they emphasised the more important role pro-cess/formative feedback can play in changing the processes of learning.
Hattie and Timperley’s (2007) feedback model also starts with the purpose of learning as reducing the discrepancy between current understandings/performance and a desired goal. To do that, effective feedback has to be more than feeding back, in which students obtain information to the question ‘how am I going?’ In addition, teachers clarify the learner’s question of ‘where am I going’ by feeding up. More importantly, feeding forward tells them‘where to next?’ Hattie and Timperley claim that each feedback question works at four levels: task, process, self-regulation and self. Feedback at the task level concerns how well tasks are understood or performed; feedback at the process level addresses the main processes needed to understand/perform the task; feedback at the regulation level is related to self-monitoring, directing and regulating of actions; andfinally, feedback at the self level is about personal evaluations and affect about the learner. They argue that feedback at the self level is the least effective as it is usually unrelated to performance on the task. Feedback at the process and self-regulation level are powerful for mastery of tasks and deep processing, while task-level feedback has the potential to improve subsequent strategy processing and enhance self-regulation but rarely does so.
More recently, Boud and Molloy (2012) advocated a new feedback mindset by contrasting two feedback models, one with teachers as the drivers of feedback and the other with learners generating and soliciting their own feedback. They argue that the conventional teacher model places false expectations on teachers in the higher education system and does not sustain learning beyond the classroom. In order for the learner model to take place, opportunities need to be created at the inception of curriculum design to help students develop the capabilities to operate as judges and owners of their learning.
Aims of the study
The research reported here is an attempt to explore the nature and characteristics of learners’ self-assessment and, more precisely, learners’ self-feedback. By designing a self-assessment task and collecting learners’ self-feedback in an English as a Foreign Language (EFL) context with a focus on developing speaking skills, it is hoped that empirical data can provide a realistic view of what first-year college students are able and not able to do in furnishing self-assessment and self-feedback. This under-standing may be helpful in formulating feedback theories that put learners at the centre. More specifically, by adopting the aforementioned Hattie and Timperley (2007) model of feedback, the following research questions guided the study:
(1) What did students find out about, as reflected in their self-assessment/ self-feedback, the discrepancies between their EFL oral performance and their learning goals?
(2) To what extent did learners’ self-assessment/self-feedback feed up (‘where am I going’), feedback (‘how am I going’) and feed forward (‘where to next’)? (3) In terms of the four feedback levels of task, process, self-regulation and self,
how did learners attend to them? Were there similarities or differences between theoretical assumptions and the actual observed learner self-assessment/self-feedback?
Methodology The context
This paper reports on the implementation of a self-assessment/self-feedback task and its results in a required freshman EFL course in Northern Taiwan. The university, with a total student population of about 15,000, two-thirds of which are undergradu-ates, requires first years (freshmen) to take the College English I and II courses for two consecutive terms, with two class hours allocated weekly for 18 weeks each term. Although individual teachers have the freedom to select their own materials and methods, the common course objectives are to enhance all four language skills of listening, speaking, reading and writing, as well as to foster lifelong learning capabilities and liberal education ideals. Participants in this study were learners enrolled in the College English II sections taught by the author.
Under the national curriculum, these students started receiving formal EFL education in the third year of elementary school, but many started earlier with pri-vate tutors and cram schools. Their EFL proficiency was approximately around B2 in the six-level standard (from basic users of A1, A2, independent users of B1, B2, to proficient users of C1, C2) as stipulated by Common European Framework of Reference for describing achievements of foreign language learners.
There were midterm and final examinations each semester. Each examination consisted of written and spoken parts assessing learner achievement after studying materials from the textbook – World Class 2 (Douglas and Morgan 2013). Each speaking test required learners to answer six open-ended questions for one minute each in a language lab, in which everyone wore a headset and spoke to a micro-phone. These oral questions were selected from a course question bank developed and accumulated as the semester progressed. For each unit taught, the author pre-pared 10 short questions covering major materials and class discussions. By answer-ing these questions, learners were expected to retell and summarise information, to reflect on relevant personal experiences and world knowledge, and to provide per-sonal opinions with justification on related issues, all in English. These questions were regularly practiced inside and outside of class, and were posted on the course Moodle platform for constant review and practice. Examples of the questions include:
• Summarise the article ‘What Happens When a Language Dies’ (Unit 1: Language and Life);
• Describe your spending habits, including how much you spend each month and on what you spend (Unit 2: Money Talks);
• What are the advantages and disadvantages of living in a big city (Unit 3: Bright Lights, Big Cities)? and
• What did you learn about yourself and your classmates from the ‘Life Satisfaction Survey’ conducted in class (Unit 4: Being Yourself)?
To prepare for the midterm examination in the first semester, students worked on 30 questions for the first three units. To prepare for the final examination, they had three more units and another 30 questions added to the question bank, totalling 60, with 80% of their grade allocated to the new ques-tions and 20% to the old ones. Students were advised not to write answers word for word and memorise them. Instead, they were asked to write keywords and outlines and practice orally from the brief notes they had written. Because of this, learners were allowed to bring notes of no more than 15 words per question for both midterm and final examinations.
The self-assessment/self-feedback task
The self-assessment task took place during the second semester. Each student was given his/her own six audio files from the final oral examination in the previous semester, and was required to complete a self-assessment/self-feedback assignment. It was hoped that they could benefit from this self-assessment/self-feedback and improve in the midterm and final oral examinations in the second semester. This self-assessment/self-feedback task asked students to do the following. The assign-ment guideline, as presented below, was accompanied by an example from the course teaching assistant under the instructor’s guidance, so as to illustrate what a completed assignment might look like.
(1) Listen and choose three from the six one-minute answers.
(2) Listen to the three self-selected audio files carefully and transcribe them as truthfully as possible.
(3) Based on the transcribed answers, analyse your speaking performance from the following four aspects: (a) content and organisation, (b) pronunciation, intonation andfluency, (c) word choice and meaning expression, and (d) sen-tence structure and grammar. Discuss what you have done well and what you need to improve in the above areas by citing specific examples from your transcript. The language of discussion could be in either English, the target language, or Chinese, your mother tongue.
(4) Discuss what you think you can do to improve your English oral pro fi-ciency.
Data analysis
Sixty-three completed assignments were submitted and used as data for the study. The researcher/instructor explained the purpose of study, obtained written consent from students, and adhered to research ethics of participant anonymity and exclusive academic use of data. To ensure the quality of self-assessment/self-feedback data at a basic level, two measures were applied in the first round of screening. First, the typed transcripts were checked against the corresponding audiofiles and were rated
on a scale from 1 to 5 for truthfulness, with 5 being the most truthful. One assignment did not have any transcript included and was removed from the data set. In the remaining 62 pieces, none was rated 1 or 2, i.e. with the transcript contents deviating from the recording for more than 50%. Truthful word-for-word transcripts of no more than two-thirds of recording were awarded 3; those above that level and combined with some additional marking of speech (false starts, repetitions, pauses or verbal fillers) were awarded 4; and those combined with scrupulous markings of speech were given 5. In the end, there were 7, 14 and 41 pieces rated as 3, 4 and 5, respectively.
The data set was then checked for overall compliance to task requirement. Based on the self-assessment/self-feedback task specification, task compliance was rated on a scale from 1 to 3, with 3 being the most compliant. The self-assessment/self-feed-back content that covered all four required aspects of analysis and proposed future actions was rated as 3; those missing one or two parts as 2; and those missing more than two parts as 1. Fifty-six pieces were rated as 3, three as 2, and four as 1. The analysis was carried out by the researcher first. A research assistant was instructed to do the same independently, followed up by discussions after a two-week interval to resolve differences. After removing those rated 3 and below on truthfulness of transcription and those rated 2 and below for task compliance, 50 assignments remained for content analysis.
Through initial readings of the data with no predetermined themes or categories in mind, the researcher decided to adopt Hattie and Timperley’s (2007) feedback framework for the analysis of learners’ self-assessment/self-feedback. Following the model, data were then read and coded, respectively, for: (a) discrepancies between goals and performance, (b) answers to feed up, feedback, and feed forward ques-tions, and (c) feedback at task, process, self-regulation and self levels. Learners’ self-feedback relating to performance criteria, as having been specified in task requirement, included content and organisation, pronunciation, intonation, fluency, word choice and meaning expression and sentence structure and grammar. Under each category, meanings illuminated in self-assessment/self-feedback were sought. Following suggestions for thematic data analysis (Braun and Clarke2006), recurring patterns and the underlying learner conceptions were identified. Data coding was conducted again by the author after a one-month interval, in which minor differences were resolved. Excerpts from data were then selected for the presentation of research findings. Wherever necessary, excerpts were translated into English and back-translation was used to ensure validity.
Results and discussions
All 50 assignments complied with the self-assessment/self-feedback task speci fica-tion, but the degree of detail and comprehensiveness varied to a great extent. Some students used very brief, vague, one-size-fits-all comments throughout the four aspects of self-assessment/self-feedback, such as‘my content was good’ and ‘I need to improve my pronunciation’. Other students demonstrated expert-like critical abil-ity, analysing each part in detail with terms from the criteria and rubrics they learned in the previous semester. Still others extended their self-assessment/self-feedback beyond the requirement of examining the audio-recorded performance and were able to present a more panoramic review of their EFL learning in general. One element seemed to distinguish the first from the latter two groups of learners, i.e. whether
specific transcribed examples were included in the self-assessment/self-feedback. Twenty-seven students drew on their transcript samples in the self-assessment/ self-feedback; they appeared to be more articulate and provided lots of interesting information. The remaining 23 failed to do so and their self-assessment/self-feedback comments were comparatively superficial and less insightful. The following discussion will follow the major components in Hattie and Timperley’s (2007) feedback model.
Identifying discrepancies
Learning goals, although not defined or discussed explicitly for the self-assess-ment/self-feedback task, seemed to be something learners took as given from their learning in the previous semester and did not need further clarification. Learners clearly pointed out their strengths most of the time. They also identi-fied areas for improvement. Based on existing assessment and feedback theories, these areas are the discrepancies between learning goals and the current perfor-mance. However, the identified discrepancies as shown in learner data revealed a much wider spectrum of their observed gaps. In addition to learning goals, learners gauged their actual performance against at least three more types of referents. These referents by which they compared their performance were: (a) what they thought they had performed, (b) what they had prepared for the per-formance and (c) what they were able to perform in a different setting for the same subject matter.
Excerpts 1 and 2 are examples of the first type, which was more prevalent than the other two types. These learners had some impression of their own performance, but the self-assessment/self-feedback task gave them a chance to realise that their actual performance somehow deviated from what they believed to be the case. This discovery was often accompanied with a sense of surprise. Although EFL speaking was not a new subject of learning and all participants had been learning the lan-guage for 10 or more years, for most of them, listening to their own voice in audio recording was a novel experience which enabled them to notice the discrepancy between what they believed they had performed and what they discovered through listening.
(1) My pronunciation was not accurate. When I listened, I was very much sur-prised by myself. Like in thefirst question I said the market of stock, it took me several times to finally understand what I was saying. In question 2, picky sounded like pity. And there are many more such cases.
(2) I finally got to know what my voice sounded like in others’ ears. I always thought I was clear and fluent. But in trying to complete this assignment I had to be super attentive, or I could not understand what I myself was say-ing. It must be even more difficult for others to understand me.
The second type of discrepancy was between the actual performance and the pre-pared scripts. One learner discovered that, although he had prepre-pared a lot by writing down in advance his ideal answers to each question, his actual recorded answers were very much discounted. This learner said he then realised that proposing answers in writing did not equal to being able to express the same ideas fully by speaking, and more oral practice had to be part of the preparation. It took him two
pages in the self-assessment/self-feedback assignment to put down his original written notes and compare them against his spoken transcripts. Although learners taking this course had always been encouraged to prepare bullet point notes, rather than word-for-word scripts, some stuck to their old habits.
In the third type of discrepancy, as demonstrated in excerpt 3, the learner com-pared her spoken vocabulary with her written one. To be more specific, she found that she had a larger vocabulary with written texts, i.e. in reading and writing, than what she was able to use in expressing herself orally. This kind of discrepancy, although different from the second type, had similarly invoked in learners awareness about the particularity of EFL speaking as a distinctive skill, and was likely to make learners more mindful in their future preparation for speaking assessments.
(3) In terms of word choice and meaning expression, I discovered that my spoken vocabulary is much smaller than my written vocabulary. I think it is because, while speaking, I could only say whatever occurred to me in real time. And the first-moment vocabularies are usually the very basic kind. These simple vocabularies often failed to express my thoughts fully. For example, in ques-tion 3, I wanted to say part-time jobs could provide me opportunities in developing interpersonal skills, but I only said experience social life.
In addition to discovering discrepancies, some learners went a step further to analyse these discrepancies and make sensible attributions, although it was not a required component of the self-assessment/self-feedback task, as exemplified in excerpt 4.
(4) I guess because I was afraid of speaking English and not quite used to it, and when I was speaking, I was at the same time thinking about grammar and what to say next, that made me sound hesitant all the time.
The data illustrated that the discrepancies identified by learners were multifaceted. While current theories inform us of such discrepancy identification as a critical starting point for learning, the discrepancy in most teachers’ conception is oftentimes limited to the gap between learner performance and the learning goals. In this particu-lar context, the types of discrepancies were observed by learners as comparisons of performance additionally with an impression, with preparation and with performance in another situation. These various kinds of discrepancies bear pedagogical signi fi-cance and may not always attract the attention of teachers or tutors, who have long been the predominant feedback providers. Moreover, we can expect that if the dis-crepancy was primed by the teacher, it may not have been a self-discovery and invoked the sense of surprise as expressed by many learners. According to Butler and Winne’s (1995) self-regulation model, this self-discovery and sense of surprise could have a strong impact on follow-up learning. Nevertheless, the fact that self-assessment/self-feedback from nearly half of the participants (23 or 46%) was relatively vague and ambiguous should not be overlooked.
Feeding up
Feeding up was closely related to identifying discrepancy. On the surface, there was little feeding up in learners’ self-assessment/self-feedback, but actually, feeding up
was implicit in the background against which learners gauged their performance analysis. When they said this was not good enough or that it had to be changed, there was a pretty clear standard referred to but not overtly spelled out. Without well-acknowledged goals, it may be quite a challenge for learners to do meaningful self-assessment and provide useful self-feedback.
Feeding forward
Feeding forward appeared at two levels. In the final part of the task requirement, learners were asked to discuss what they thought they could do to improve speaking proficiency after self-assessment/self-feedback. This part existed in all submitted assignments as a concluding remark, and learner responses were rich and varied. Learners proposed for themselves a great variety of cognitive, metacognitive, affec-tive, social and self-regulative strategies, and suggested actions to be taken. Some even came with behaviour frequencies prescribed, such as ‘reading out loud at least once a day’, as future self-regulation reminders. The wide ranging ideas from one single student are illustrated in excerpt 5.
(5) a. Increase the amount of my vocabulary and phrases.
b. Make sure I know how to use the word when I memorise a new one. c. Try using different kinds of connectors when I converse with others. d. After I am familiar with basic vocabulary, expression and grammar, I should fight against my fear of speaking English and express my opinions with confidence.
e. Read English articles and listen to English news when I have time. f. Make myself fall in love with English.
g. Find opportunities to exchange with foreign students.
h. Be calm and relax in a speaking test. Do not be stressed out to forget what I have prepared.
Other than bullet point listings, some learners provided more justifications and reasoning for the actions they intended to take, as shown in excerpt 6. These feed-forward responses were more comprehensive, with the inclusion of personal scrutiny on why certain actions should be taken, how to carry out those actions, what the anticipated difficulties may be or even an estimate of how likely one would succeed. They have also revealed the spectrum of learner knowledge on tactics and strategies in the subject discipline.
(6) I think thefirst thing is to increase my vocabulary size and resolve my gram-matical problems. For this, I can read more articles and novels, or watch movies and TV series, so as to be familiar with how native speakers talk and gradually internalise that ability, so I can speak grammatical sentences natu-rally and express myself more accurately. But I know this will take a long time! I feel recording is an effective method. Listening to myself will let me know if I pronounce words clearly or if I amfluent. I often had a lot of inter-jections and redundancies without consciously knowing it. Now I can warn myself through this method.
Another level of feeding forward appeared immediately after analysis of perfor-mance as a natural follow-up. For the question ‘what you need to improve’, some responded with meticulous corrections of mistakes found (as will be discussed in Feeding back), others with future learning strategies for the particular problems observed, as in excerpt 7. This type of more focused feeding forward, compared to the more general kind discussed above, was usually derived directly from the specific weaknesses found, and was therefore more precise.
(7) a. My answers were meagre and many were not directly related to the questions. They were loosely organised and I tended to speak nonsense randomly.
b. I have to prepare more,first by fully understanding the questions, and then by practicing my answers repeatedly so I could have a better control over what I say.
Feeding back
Self-assessment/self-feedback content on the more general strengths and weaknesses, or the more detailed analysis of what was done well and what had to be improved were all directly related to feeding back. Some learners observed very carefully and picked up all mistakes or weaknesses they could possiblyfind. A few even classified their mistakes into different categories, calculated the number and marked them with different colours, as shown in excerpt 8. Others went further to correct all the mistakes observed, although this was not required or even suggested in the task specification.
(8) I think my pronunciation and intonation is okay. I can hear myself clearly and understand what I was saying. But forfluency, I don’t think I was good. In my transcript, the parts marked blue were meaningless uhs and ums, which occurred eight times in one minute on average. And the parts that I marked green were my self-repeats, occurring about once or twice in each of my answer.
Deeper, more complex analysis was also present in the data. For example, in excerpt 9, the learner discovered a problem in her organisation of messages by jumping back and forth from one subtopic to another without appropriate transitions. She was able to elaborate her findings with notes in parentheses highlighting the subtopics she identified, and placed them in front of relevant transcribed sentences. The issue she raised was not discussed in this particular course but was often taught in EFL writing– one of coherence and transitions.
(9) As for my weakness, the sequence of my points was messy. Sometimes I spoke on a topic and soon passed to the next one without finishing up the previous topic properly. For example, in Question 5, (normal robots) It can do laundry and maybe take care of children and cook for people. (nanobots) And in the future, there may be nanobots to repair our cells. (back to normal robots) And maybe it can deliver packages for people so we don’t need mailmen.
Interactions between feedback at the task and at the process level
Some feedback messages were solely at the task level, others solely at the process level. A majority of them, however, did not belong to one type, but involved feed-back at both the task and the process levels. Feedfeed-back exclusively at the task level was related to the immediate performance– whether that particular answer was com-plete or correct, which could not be transferred to answers for other questions, and thus was limited in its implication for the learner. Feedback solely at the process level often seemed rather vague and lacked clear reference. For example in excerpt 10, the learner did not use any transcript in discussing his strengths and weaknesses. This kind of feedback sounded more like cliché and did not have the potential to take learners further.
(10) I used examples to support my answers. But I did not use many examples. And I often explain using unnecessary sentences.
On the contrary, feedback that appeared more effective usually combined not only specific examples at the task level but also generalised conceptions at the process level, which were in turn deduced from those concrete examples. Excerpts 8 and 9 were such examples. In both excerpts, the learners first gave a summary of their observations, which was at the process level. The general statements of ‘not fluent’ and‘sequence being messy’ were then followed up by precise details collected from the transcripts, which in turn validated the learners’ self-assessment/self-feedback as well as pointed to much clearer future actions.
Feedback at the self-regulation level
In a broader sense, the learners’ self-assessment/self-feedback task itself could be seen as training for self-regulation, and all learners’ responses could be considered either attempts or results of self-regulation. By way of following the task directions and completing the assignment through listening, transcribing and analysing in the four aspects of EFL speaking and proposing future actions, learners went over self-monitoring on previous performance and self-directing for future behaviour regula-tions. Self-regulation is present in all learners’ self-assessment/self-feedback responses, despite some being more successful than others in their potential self-regulation efficacy.
Feedback at the self level
A number of learners produced self-assessment/self-feedback at the self level and, contrary to expectation, this type of feedback appeared to be quite positive in terms of its efficacy and potential for future improvement. Some learners considered their own personal characteristics and dispositions in relation to the learning of EFL speaking in general, as shown in excerpt 11. This learner’s candid statements were a result of his observation and deep reflection, which revealed a healthy positive attitude in facing his unique problem. His prescrip-tions also seemed realistic and well grounded. Inherent in this kind of self-level feedback was clear perceptions of the self encompassing a reflection in the person’s learning habits and history.
(11) I am the kind of person who tends to get nervous easily and could not improvise in stressful situations. Because of this, I think I have to prepare much earlier and more fully for speaking tests like this, more than what is necessary. As for pronunciation and intonation, I think I did well, probably because I heeded to this part when memorising vocabulary in middle and high schools. But, owing to my croaking voice, I’d better pay special atten-tion to my intonaatten-tion.
Much of such plausible self-assessment/self-feedback was made possible by learners doing more than what was required in the self-assessment/self-feedback task. For example, in excerpt 12, this learner did not limit himself to the three required transcripts, and was able to identify his problems of rigidly adhering to a few templates for all the 60 questions.
(12) Before the test, I wrote down my answers to all questions. So the content was not a big problem. But just because of that, I tended to use very similar formats in answering all questions. It may not be apparent viewing only from the three selected audio files and their transcripts. But if you check my entire answer profile, you’ll find that I followed the same two or three structures all the way through, with only content words replaced.
Interestingly, the findings on learners’ self-level feedback seemed to contradict Hattie and Timperley’s (2007) argument that feedback at this level is too often unre-lated to performance on the task and counterproductive to learning. The majority of self-level feedback was illuminating. With the self-assessment/self-feedback opportu-nity given, many learners made a birds-eye view analysis, realistically taking into consideration their experiences and their congenital strengths and limitations. As exposed in their reasoning and attitude, this self-level feedback represented thought-ful and thorough reflections of the learner assessors. Moreover, these feedback mes-sages helped learners to understand themselves better and become more affectively ready for future learning challenges. A possible reason for this difference may have to do with learner age. While the learners described in Hattie and Timperley (2007) were mainly pupils, those in this study were mature young adults who could be more autonomous and shouldered more learning responsibilities.
In summary, learners’ self-assessment/self-feedback as shown in this study identified discrepancies of current performance against goals, as well as against learners’ impression, preparation and performance in other situations. Self-assessment/self-feedback messages encompassed aspects depicted in Hattie and Timperley’s feedback model (2007), including answers to feeding up, back and forward questions, and to the four levels of task, process, self-regulation and self. The detected features of self-feedback mostly supported what we know about feed-back. In particular, learners’ self-feedback appeared to be more powerful when there was interaction between the task and the process level, in that task-level details gave a solid basis for generalisation and transfer at the process level. However, although feedback at the self level is considered counterproductive in Hattie and Timperley’s model, the data here suggested that self-level self-feedback was illuminating and beneficial for learners.
Implications, limitations and conclusion
Before discussing further, let us imagine an alternative scenario of typical teacher feedback for the same learner performance. Most likely, the one or a few more tea-chers/tutors would spend a few days listening to all audio files and trying to inform each learner, in writing, audio or face to face of the strengths and areas for improve-ment. In the meantime, because teachers were taking care of the feedback, learners would move on with daily routines and less likely be engaged in what their teacher did, while the memory of how they performed in the 10 minutes or so of speaking gradually fades away. Sometime later, learners would passively receive teacher feedback, read or not read it, act or not act upon it, and continue learning the same way or with changes informed by teachers, until they undertake the next speaking test. With self-assessment and self-feedback, the data examined in this study were generated by learners and were unequivocally learner-centred. Many problems dis-cussed in the literature with teacher feedback did not exist in the observed learner self-feedback. There was certainly no misalignment of perceptions between teachers and learners (Orsmond and Merry2011). The self-feedback messages were less sus-ceptible to problems such as teachers’ delivery, tone, style or excessive amount that may overwhelm learners or damage their self-esteem (Wingate2010). Moreover, the self-feedback messages were comprehensible and not filtered out by learners as would be those coming from the teachers (Butler and Winne 1995). In order to complete self-assessment/self-feedback, learners first had to engage in careful post-performance review and reflection, thereby taking over learning responsibilities and turning themselves from passive recipients of feedback into the owners of their own learning. Based on their idiosyncratic background knowledge of the world and of the subject matter, learners discovered more about their learning through self-assess-ment and self-feedback. In short, similar to what Nicol, Thomson, and Breslin (2014) have found with learners conducting peer assessment, it should be better to give than to receive.
In addition to not having common teacher feedback shortcomings, the study reported here uncovered that learners’ self-feedback entailed the width and depth that could hardly be expected from any conscientious teacher. Most EFL teachers would not know a learner’s vocabulary size in different language skills and its implications for the learner, nor what a learner has prepared for a test and how his/her test perfor-mance deviates from it, let alone everyone’s past learning histories, personality traits and emotions. Learners, not teachers, are the ones who possess all this information, and could make sense of it through meaningful self-assessment/self-feedback. As can be seen, with some guidance and support, learners could be ideal feedback providers for the self because they are knowledgeable with chronological and cross-sectional background information on their own learning.
Besides, in self-assessment/self-feedback, time and effort for feeding back to the self would less likely be divided and diluted, as would be the case for teachers teaching large classes. As shown in this study, learners were no longer the usual one among a crowd of many students being served by a single teacher assessor. By way of self-assessment/self-feedback, they made it possible to attend to such details as counting error types and average frequencies, thus permitting more thoroughness in assessment for which most teachers cannot afford.
What learners’ self-assessment/self-feedback revealed to us bears further implications when viewed together with recent research developments. First, the
self-feedback provided important clues to the teacher on what help learners really need. Secondly, as have been discussed, Boud and Molloy (2012) advocate a stu-dent-seeking model of feedback shifted from the habitual teacher telling one. Thirdly, researchers (Beaumont, O’Doherty, and Shannon 2011; Bloxham and Campbell 2010; Yang and Carless 2013) have called for teacher–learner dialogues involving feedback on performance and subsequent reflections and adjustments to replace the kind of feedback featuring only one-way teacher monologues. It follows that, with their capacity to produce meaningful and abundant self-feedback, learners could and should initiate this dialogue.
Unlike in most circumstances, where teachers decide what they teach and pre-scribe what learners learn, if dialogues are initiated by learners’ self-feedback, such feedback could provide fertile ground for productive teaching and learning. False beliefs and misconceptions could be eradicated or clarified. Positive attitudes and practices could be intensified. Focused instructions could take learners a step closer to learning goals in a more systematic manner. Moreover, since self-feedback involved issues of affect, discipline, self-regulation and personal management, it pro-vided a good opportunity for teaching lifelong learning skills and for the teacher to share experiences as an expert. With the multitude of issues involved in assessing EFL speaking, we could reasonably expect that teacher feedback on the same learner performance could hardly come close to coinciding with the content in learners’ self-feedback. Therefore, with learners’ self-feedback as a legitimate basis for fol-low-up teaching, teachers do not have to risk shooting off the target and teaching what learners do not need or cannot take.
It is thus suggested that, other than providing feedback to individual learners, tea-chers’ time and effort could also be well spent in designing activities to guide learners in furnishing meaningful self-feedback. For different subject matter, attention and time could be devoted to what learners need to be equipped with in conducting worth-while self-assessment and generating self-feedback. Self-assessment/self-feedback tasks could be seen as training for self-regulation. Advance preparation could facili-tate learners in conducting quality self-assessment/self-feedback. For the self-assess-ment/self-feedback task in this study, it was positioned at the beginning of the second half of the school year, foregrounded by students practicing the skills for one seme-ster, being tested in a midterm and afinal examination, and familiarising learners with the criteria of proficient EFL speaking. Because students were guided to analyse their multiple speaking samples from the previous speaking test, their self-assessment/self-feedback was by nature not limited to single answers at task level; rather, the self-assessment/self-feedback became by and large stocktaking for EFL speaking at a midpoint of their learning. Self-assessment/self-feedback at this depth is so far not a common classroom practice in most educational settings, but it could be considered. As research on teacher feedback has been proven less effective, due to contextual and structural constraints (Bailey and Garner 2010; Price et al. 2010), and learner engagement has been pointed out as a possible solution (Nicol, Thomson, and Breslin
2014; Orsmond and Merry 2013), pedagogical considerations that facilitate self-assessment/self-feedback in different disciplines for learners at different age warrant further research.
Nevertheless, it has to be noted that the feedback loop was not yet completed in the present study. Although problems were identified, they were not likely to disap-pear simply because of the self-assessment/self-feedback. A lot of on-task and off-task monitoring as well as extensive practice had to follow and were mandatory.
This notion points to the limitations of the current study. Although the data analysis showed interesting depth in learners’ self-assessment/self-feedback, it is not clear if and (if yes) how effectively or quickly learners could indeed improve in areas they noticed. It is not clear, either, whether or how often this kind of learner self-assess-ment/self-feedback should be repeated regularly during a course. Would the surprise and novelty of discovery and its impact brought about by such learner self-assessment/self-feedback be likely to diminish overtime if it is repeated? What exactly is it that teachers can do to ensure follow-up learning and improvement?
For learners who did not seem to be as engaged in the self-assessment/self-feed-back task as their peers were, such as the 23 out of 50 who did not supply specific examples and give more insightful self-feedback, what could be offered to help them and what were the factors involved? Are there certain types of learners or learning that may benefit more from self-assessment/self-feedback more than others? These questions suggest future research directions on learners’ assessment and self-feedback. More studies, such as experiments comparing learning gains with and without self-assessment/self-feedback, and interviews with and observations of learners after they have been engaged with self-assessment/self-feedback, are needed to help us understand the mechanism of self-assessment/self-feedback further.
In conclusion, unlike previous studies concerned with the accuracy or effects of learner self-assessment, this study explored the content and potential of college students’ self-assessment/self-feedback on their EFL speaking performance. Data analysis indicated that, under guidance, feedback generated by learners answered feed up, feedback and feed forward questions, and covered levels of task, process, self-regulation and self. Results supported Hattie and Timperley’s (2007) model, in that feedback involving both details from the task level and generalisation at the pro-cess level was more productive than feedback exclusively at either level. However, contrary to theoretical presumptions of self-level feedback being unproductive, learners’ self-level feedback appeared to be quite beneficial. Learners’ self-feedback in general did not have many of the problems already known with teacher feedback, and its richness may provide good materials for more learner-focused teaching and learning. It was suggested that teachers, besides preparing feedback for individual learners, spend time designing tasks to facilitate learner self-assessment/self-feedback and use that self-assessment/self-feedback information as a basis for subsequent instruction. Finally, more research is needed to help us understand the efficacy, consequences and necessary follow-ups of learners’ self-assessment/self-feedback.
Disclosure statement
No potential conflict of interest was reported by the author.
Funding
This work was supported by the Ministry of Science and Technology, Taiwan, Republic of China [grant number 102-2410-H-004-070].
Notes on contributor
Shu-Chen Huang has been teaching English as a foreign language (EFL) in Taiwan at the tertiary level for nearly 20 years. Her research interests include EFL pedagogy, learning motivation and formative assessment. She has published in journals such as Language Testing, Language Assessment Quarterly and Journal of English for Academic Purposes.
References
Andrade, H. 2008.“Self-assessment through Rubrics.” Educational Leadership 65 (4): 60–63. Bailey, R., and M. Garner. 2010.“Is the Feedback in Higher Education Assessment Worth
the Paper It is Written on? Teachers’ Reflection on Their Practices.” Teaching in Higher Education 15 (2): 187–198.
Beaumont, C., M. O’Doherty, and L. Shannon. 2011. “Reconceptualising Assessment Feedback: A Key to Improving Student Learning?” Studies in Higher Education 36 (6): 671–687. Black, P., and D. Wiliam. 2009.“Developing the Theory of Formative Assessment.”
Educa-tional Assessment, Evaluation, and Accountability 21: 5–31.
Bloxham, S., and L. Campbell. 2010. “Generating Dialogue in Assessment Feedback: Exploring the Use of Interactive Cover Sheets.” Assessment & Evaluation in Higher Education 35: 291–300.
Boud, D. 1995. Enhancing Learning through Self Assessment. London: Kogan Press. Boud, D., and N. Falchikov. 1989. “Quantitative Studies of Student Self-assessment in
Higher Education: A Critical Analysis of Findings.” Higher Education 18: 529–549. Boud, D., and E. Molloy. 2012. “Rethinking Models of Feedback for Learning: The
Challenge of Design.” Assessment & Evaluation in Higher Education 38 (6): 698–712. Brantmeier, C. 2005. “Nonlinguistic Variables in Advanced L2 Reading: Learner’s
Self-assessment and Enjoyment.” Foreign Language Annals 38 (4): 493–503.
Braun, V., and V. Clarke. 2006. “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3 (2): 77–101.
Butler, Y. G., and J. Lee. 2006. “On-task Versus Off-task Self-assessments among Korean Elementary School Students Studying English.” The Modern Language Journal 90 (4): 506–518.
Butler, Y. G., and J. Lee. 2010.“The Effects of Self-assessment among Young Learners of English.” Language Testing 27 (1): 5–31.
Butler, D. L., and P. H. Winne. 1995.“Feedback and Self-regulated Learning: A Theoretical Synthesis.” Review of Educational Research 65 (3): 245–281.
Chen, Y.-M. 2008. “Learning to Self-assess Oral Performance in English: A Longitudinal Case Study.” Language Teaching Research 12 (2): 235–262.
Douglas, N., and J. R. Morgan. 2013. World Class 2. Boston, MA: National Geographic Learning.
Hattie, J., and H. Timperley. 2007. “The Power of Feedback.” Review of Educational Research 77: 81–112.
Lee, I. 2009.“Feedback Revolution: What Gets in the Way?” ELT Journal 65 (1): 1–12. Lew, M. D. N., W. A. M. Alwis, and H. G. Schmidt. 2010. “Accuracy of Students’
Self-assessment and Their Beliefs about Its Utility.” Assessment & Evaluation in Higher Education 35 (2): 135–156.
Matsuno, S. 2009.“Self-, Peer-, and Teacher-assessments in Japanese University EFL Writing Classrooms.” Language Testing 26 (1): 75–100.
McConlogue, T. 2012.“But is It Fair? Developing Students’ Understanding of Grading Com-plex Written Work through Peer Assessment.” Assessment & Evaluation in Higher Education 37 (1): 113–123.
Milne, C. 2009. “Assessing Self-evaluation in a Science Methods Course: Power, Agency, Authority and Learning.” Teaching and Teacher Education 25: 758–766.
Nicol, D., A. Thomson, and C. Breslin. 2014. “Rethinking Feedback Practices in Higher Education: A Peer Review Perspective.” Assessment & Evaluation in Higher Education 39 (1): 102–122.
Orsmond, P., and S. Merry. 2011. “Feedback Alignment: Effective and Ineffective Links between Tutors’ and Students’ Understanding of Coursework Feedback.” Assessment & Evaluation in Higher Education 36 (2): 125–136.
Orsmond, P., and S. Merry. 2013. “The Importance of Self-assessment in Students’ Use of Tutors’ Feedback: A Qualitative Study of High and Non-high Achieving Biology Under-graduates.” Assessment & Evaluation in Higher Education 38 (6): 737–753.
Orsmond, P., S. Merry, and A. Callaghan. 2004.“Implementation of a Formative Assessment Model Incorporating Peer and Self-assessment.” Innovations in Education and Teaching International 41 (3): 273–290.
Price, M., K. Handley, J. Millar, and B. O’Donovan. 2010. “Feedback: All That Effort, but What is the Effect?” Assessment & Evaluation in Higher Education 35 (3): 277–289. Rivers, W. P. 2001.“Autonomy at All Costs: An Ethnography of Metacognitive
Self-assess-ment and Self-manageSelf-assess-ment among Experienced Language Learners.” The Modern Language Journal 85 (2): 279–290.
de Saint Léger, D. 2009. “Self-assessment of Speaking Skills and Participation in a Foreign Language Class.” Foreign Language Annals 42 (1): 158–178.
Tan, K. H. K. 2004. “Does Student Self-assessment Empower or Discipline Students?” Assessment & Evaluation in Higher Education 29 (6): 651–662.
Wingate, U. 2010. “The Impact of Formative Feedback on the Development of Academic Writing.” Assessment & Evaluation in Higher Education 35 (5): 519–533.
Yang, M., and D. Carless. 2013.“The Feedback Triangle and the Enhancement of Dialogic Feedback Processes.” Teaching in Higher Education 18 (3): 285–297.
