設定目標以激勵口語品質的提升---目標明確度與參照對象的研究 (II)

(1)

設定目標以激勵口語品質的提升—目標明確度與參照對象

的研究(第 2 年)

研究成果報告(完整版)

計畫類別：個別型計畫編號： NSC 98-2410-H-004-133-MY2 執行期間： 99 年 08 月 01 日至 101 年 04 月 30 日執行單位：國立政治大學外文中心計畫主持人：黃淑真計畫參與人員：碩士班研究生-兼任助理人員：吳品嬅報告附件：出席國際會議研究心得報告及發表論文公開資訊：本計畫涉及專利或其他智慧財產權，2 年後可公開查詢

中華民國 101 年 07 月 20 日

(2)

一為近期發展的評量新概念—以形成性評量輔助教學

（Assessment for Learning），另一為過去常用於企業管理的目標設定理論（goal setting theory）。研究者首先回顧兩個領域的文獻，並整理出目前研究已知與待解問題，提出將兩者結合運用的理論基礎及運用在教學之創新觀念，規畫其在大專英語課室中操作的模式。原計畫預計將理論運用在口語訓練，唯計畫通過後再做文獻補強時，發現到口語練習之即時與不可逆的特性較不利於需要學習者有心規範自身行為的目標設定方式，故在執行一期後將原本設定在口語訓練的教學實驗中止，改以寫作與修改的訓練為標的，以期較能適切觀察到目標設定對學生學習行為的影響，發揮研究的效益。研究以筆者設計的形成性評量寫作修改教學教案為本，設計三種不同的實驗組別，以探討其對學習寫作修改的影響，三個組別分別是「評量為本之教案」、「評量為本之教案＋目標設定」及「評量為本之教案＋目標設定＋策略輔助」，以檢驗目標設定理論運用在寫作教學的效果。依過去研究發現，目標設定用在複雜的學習情境下，不見得如早期在工廠環境中使用目標設定來得有效，因為以目標設定激勵行為需有兩個條件，目標的設定僅是心理上的動機激發，必須再搭配行為者的能力，否則無法奏效，因此給予策略輔助以加強能力十分重要，且目標必須著重過程的學習而非量化的最終結果。若行為人欠缺必要的能力，目標的設定則可能導致焦慮而造成反效果，還可能比不設目標的情況更糟。三組大一學生分別在這三種情況下學習修改自己的英文寫作初稿，且反覆課程週期數次，以期觀察到較長期且將所學運用到不同寫作任務的學習發展。兩位有經驗的評分員分別依研究者修定的評分表獨立評價所有學生的作品，經由統計分析得出以下主要結論。首先，評量為本的教案的確有助學生將初稿的品質提升，每組學生每次修改都有顯著進步，但各組間單次的進步都沒有顯著差異，顯示目標設定與策略在單次的學習中都不能在評量為本的教學之上有外加的效應。進一步檢視跨時的多次練習效果，三組的各個練習初稿仍是沒有顯著差異，顯示三組學生在將前次所學運用到下次初稿的寫作時均沒有顯著的效果。但在檢視幾次不同的修改稿時，卻看到很有趣的現象，將所有的修改稿進行跨組比較，「評量為本之教案＋目標設定＋策略輔助」組顯著優於「評量為本之教案」組，而「評量為本之教案」組又顯著優於「評量為本之教案＋目標設定」組。這個結果相當程度地在寫作教學場域驗證了目標設定理論既有研究的主張，即給學生目標必須搭配明確教導方法，否則寧可不設目標，但以上的效果是

(3)

中文關鍵詞：目標設定、英文寫作、寫作修改、課室評量、形成性評量、教師回饋、教學法

英文摘要： Assessment for learning (AfL) emphasizes involving learners in the assessment process to promote learning. To facilitate this process, findings in goal-setting mechanism were designed into an EFL writing course. In a quasi-experimental design, college learners engaged in learning to revise their English writing and improving the quality. The

purpose of this study was to examine whether the addition of goal setting and strategy provision enhanced learning effectiveness of instruction based on AfL. The three experimental groups were AfL, AfL + Goal, and AfL + Goal + Strategy. All three groups went through similar draft-instruction-revision procedures for three times. All students were expected to acquire the ability to self-assess own current ability and move to reasonable desired

learning goals. The difference among the three groups was one in the manipulation of goals. The first group did not explicitly set goals. They only reflected on the strategies used and assess their drafts and revisions after revision was completed. The second group was asked to set goals for revision and plan the revision beforehand. The third group was further given a simple list of strategies as a reminder to facilitate their behaviors. All three groups

significantly improved the quality of writing in revision over time. They, however, did not differ in the magnitude of single improvement. The data was examined across three tasks. When compared on the quality of drafts, they did not differ. But when compared on revisions, it was found that the AfL + Goal + Strategy performed significantly better than the AfL Group, and the AfL Group performed

significantly better than the AfL + Goal Group. The results seemed to suggest that goals were effective with the facilitation of strategy. Goals alone may be more deleterious than not having goals. Such effect

(4)

英文關鍵詞： goal setting, EFL writing, writing revision,

classroom assessment, formative assessment, teacher feedback, pedagogy

(5)

Strategy Provision

Table of Contents

Introduction………1

A Theory of Formative Assessment………1

Goal Setting……….…5

Performance Outcome Goals and Learning Goals………9

Goal Setting in Academic Learning Situations……….11

Revision in the Writing Classroom………....13

Setting Goals for Revision……….15

The Study………16

Research Questions……….16

Participants………..17

Instructional Procedures………17

The Experimental Groups………..……20

Control Group: AfL Only……….20

Experimental Group 1: AfL + Goal Setting……….21

Experimental Group 2: AfL + Goal Setting + Strategy Provision…………21

Instruments and Measures………..………22

Quality of Drafts and Revisions……….……….22

Revision Worksheet……….22

Data Analysis and Results……….………23

Discussions and Implications………32

References………..35

(6)

II

Table 1. Inter-rater Reliability………..24 Table 2. Summary of ANOVA on First Draft Among Three Groups………..24 Table 3. Follow-up Multiple Comparisons Using Tukey Tests………...25 Table 4. Descriptive Statistics of All Drafts and Revisions by All Three

Groups……….26 Table 5. Descriptive Statistics of Draft-to-Revision Improvement of Three

Groups on Three Tasks………28 Table 6. Summary of the Test of Regression Coefficient Homogeneity…………..28 Table 7. Summary of ANCOVA on Improvement among Three Groups

in Three Tasks………..29 Table 8. ANOVA Summary on Three Drafts………30 Table 9. Multiple Comparison on Three Drafts Using Bonferroni Test………30 Table 10. Summary of the Test of Regression Coefficient Homogeneity

on All Revision Scores……….30 Table 11. Estimated regression coefficients for first draft (X) and

revisions (Y)……….31 Table 12. ANOVA Summary on Three Revisions………....31 Table 13. Multiple Comparison on Revisions by Groups Using

Bonferroni Tests………...31 Table 14. Multiple Comparison on Revisions by Writings Using

(7)

Figure 1. Framework of Assessment for Learning………3 Figure 2. Instructional Steps of One Draft-Instruction-Revision Unit in

the Control and Experimental Groups………..19 Figure 3. Line Graph of Writing Scores for Three Tasks from Three Groups……….27

(8)

1 Introduction

About a decade ago, McNamara (2001) called for a reconsideration of priorities in language assessment research. Among the competing demands of measurement validity, accountability management, and classroom teaching/learning, it is said that the latter one had long been sacrificed at the price of the other two. For the interests of learners and teachers, McNamara argued that the boundary between classroom-based assessment and pedagogy could and should be blurred, and assessment research should be more responsive to the needs of teachers and learners. Other researchers, around the same time, made similar contentions that the pedagogical function of assessment, although having been recognized, deserves more research attention (Rea-Dickins & Gardner, 2000). That call at the beginning of this millennium has since gradually influenced the conception of both teaching and assessment in the field of language education. Scholarly efforts have in recent years been directed to the integration of assessments, curricula, and pedagogy (Cumming, 2009; Lee, 2007), to the development of a conceptual basis (Leung, 2004), and to theorization of such assessment ideas (Davison & Leung, 2009), be it labeled as formative assessment, teacher-based assessment, classroom-based assessment, or assessment for learning.

A Theory of Formative Assessment

A recently developed theory of formative assessment helps us understand the concept more comprehensively (See Figure 1). Black and Wiliam (2009) formulate formative assessment in two dimensions consisting of five sequential pedagogical procedures. The two dimensions feature three agents in a classroom, i.e. the teacher, peers, and the learner him/herself as well as three key processes in learning and teaching, i.e. establishing where the learner is going, where the learner is right now,

(9)

and, in order to bridge the identified gap, how to get there. Framed under these two dimensions, the teacher first clarifies learning intentions and criteria for success, and then helps learners acquire this critical knowledge through their peers in the

classroom. In the second step, formative assessment comes in, with evidence of student understanding, or lack thereof, elicited by questioning or task performance, to help teachers determine where the learner is right now. With both learning intentions established and learner performance assessed, the teacher is able to diagnose the discrepancy between the two and provide valid feedback to move learners forward. Pedagogical interventions do not stop here because it is actually more important in steps four and five to make sure students obtain a similar understanding of the gap and are able to, through the scaffolding of the teacher and their peers, move toward their learning goals and become the owners of their own learning.

(10)

3 Where the learner is

going

Where the learner is right now

How to get there

Teacher 1. Clarifying learning intentions and criteria for success Understanding and sharing learning intentions and criteria for success Understanding learning intentions and criteria for success 2. Engineering effective classroom discussions and other learning tasks that elicit evidence of student understanding 3. Providing feedback that moves learners forward Peer

4. Activating students as instructional resources for one another

Learner

5. Activating students as the owners of their own learning

Figure 1. Framework of Assessment for Learning (Black & Wiliam, 2009; p.8)

This theory illustrates a pedagogical approach with assessment at the center stage. As the metaphor “inside the black box” suggested (Black, Harrison, Lee, Marshall, & Wiliam, 2004; Black & Wiliam, 1998), teacher assessment is often unknown to others. Both teachers and learners had been used to approaching their jobs of teaching and learning without attempting to reveal this black box. Although the mysterious mechanism of teacher assessment has always been functioning and influencing teaching and learning, it has been an area rarely explicated to learners and not clearly articulated by teachers. The theory and relevant publications in the past years

advocate a kind of instruction in which learners are more aware of their goals and their current status in comparison to the goals, and teachers endeavor to facilitate

(11)

learners in the process of acquiring this subtle and complicated knowledge of professional assessment. It is believed that once students are empowered with a capacity to understand assessment criteria and to assess themselves, their leaning would be equipped with a clearer sense of direction and their achievement could be raised. This is probably what makes formative assessment pedagogy stand out from more traditional instructional designs.

Learning, however, is not complete when learners are able to self-assess. In fact, it is when learning will take off. Once learners are equipped with the clearer sense of direction in mind, as depicted in steps one, two, and three of the Black and Wiliam framework, they are expected to go on with steps four and five to activate

instructional resources from peers and themselves for their own learning. Even with assessment criteria clarified and status quo evaluated, learners do not automatically move from where they are to where they want to go. What could a teacher do to facilitate this process? Discussions to date on these two steps are relatively scarce compared to the previous three. One reason may be that, at this point, it is assumed that the nature of instruction becomes not so different from those of other instructions that lack a formative assessment component. But is this really the case when, after all the efforts from teachers, peers, and learners themselves, learning goals are

deliberately established?

Instructions without a formative assessment element are like sailboats in the sea without a compass. They could not inform learners of what they are targeting, despite all the effort put forward for learning. They could not be genuinely responsive to student learning needs. And the lesson is not contingent on actual learner

(12)

5

take learners to their destiny. Assessment and pedagogy needs to be put together for collaboration in the classroom. As the boundary between these two blurs, they share similar features. As McNamara (2001) elaborates, one of these features is that both “lead[s] to decisions about intervention and the targeting of learning effort” (p. 344). At this point, a theory, goal-setting, which deals with self-regulation mechanism and goal-directed behaviors, seems to be relevant and will be introduced below. The reason why goal-setting seems fit in an instruction framed under formative assessment has to do with the premise of formative assessment in the setting of goals. Formative assessment, or assessment for learning, makes sure that learning goals are deliberately clarified and communicated to learners. As in the above framework, with the goals established in step one, current status evaluated by the teacher in step two, and feedback provided in step three, teachers expect to activate the student group in step four and individual students themselves in step five as instructional resources. To aid this challenging empowering procedure in steps four and five, goal-setting theories seem to offer some inspiration. A discussion of the theory and up-to-date empirical findings will follow.

Goal-setting

Goal-setting is an inductive theory developed along the accumulation of numerous empirical studies over the past half century. Representative scholars are Edwin A. Locke and Gary P. Latham. The theory concerns human behavior and motivation as well as how efforts could be optimally directed under appropriate goals. It has been mostly applied in workplace management to enhance organizational effectiveness and efficiency (Locke & Latham, 1990). For example, assembly line supervisors set goals for workers to enhance productivity or reduce costs, and sales managers set goals for salespeople to obtain certain market share. Other than the

(13)

workplace, goal-setting is also adopted in athletes’ sports training, patients’ health management and therapies, and many other human behaviors that require

goal-directed self-regulation.

Goal-setting mechanism has been explicated in the theory. It is postulated that when faced with a goal that is reachable, specific, and committed to, people will automatically utilize goal-relevant knowledge and skills to achieve the goal (Latham & Kinne, 1974; Schutz, 1994). When existing knowledge and skills are inadequate, people would search in past similar experiences solution strategies and consciously regulate their behaviors (Latham & Baldes, 1975). According to Locke and Latham (1990), there are four aspects in the functioning of goal-setting. First, goals provide a direction. Appropriate goals direct human’s attention and behaviors toward

goal-related activities. Under the regulation of goals, people selectively reduce their limited time and effort on activities that are unrelated to goals and focus on goal attainment. Second, goals motivate and reinforce efforts. Especially when goals are difficult, more time and efforts could be mobilized through goals. This has been supported by empirical evidence in both physical and mental activities (Bandura & Cervone, 1983). Third, goals influence the persistence of effort. Difficult goals usually make people sustain their effort longer. Fourth, goal-setting could indirectly lead people to discover and use goal-related knowledge and strategies. Therefore, once a goal is accepted by the individual, it affects personal conduct and cognition, and consequently enhances performance standards. The evaluation of goal attainment is simple in some work situations when an objective figure, such as the number of pieces of products completed or the percentage of market share increased, could be obtained. There are also other situations when the performance in comparison to goals set involves professional judgment. A less direct indicator – goal

(14)

7

commitment/adherence – has also been commonly used to refer to the degree of engagement as one way to evaluate the success of a particular goal-setting mechanism (Klein, Wesson, Hollenbeck, Wright, & DeShon, 2001).

Three factors have been identified as having the potential to influence the effect of goal-setting. They are individual commitment to goals, performance outcome and feedback information, and task complexity (Latham & Locke, 2006; Locke & Latham, 1990, 2002, 2006). Details are explained below.

First, a goal has to be acknowledged and committed to by an individual in order for the mechanism of effort regulation to function in the person. In addition, high self-efficacy of the person in the particular task will facilitate this commitment. Other ways of ensuring commitment include making goals known to the public and leaders clearly communicating a vision and support. Other than assigned by the authority, goals could be collaboratively or self-set. Research findings were mixed in this respect. Some said when difficulty level of goals was held constant, assigned goals and collaboratively-set goals did not show significant difference (e.g. Latham & Marshall, 1982; Latham & Steele, 1983). Others demonstrated that having employees participate in goal-setting decisions was superior to giving goals assigned by their supervisors (e.g. Erez, 1986; Erez, Earley, & Hulin, 1985; Erez & Kanfer, 1983). Latham, Erez, and Locke (1988) investigated these studies and concluded that both types of goals could have their due effect, as long as the purpose and rationale of goal-setting are fully and successfully communicated. That is to say, goal-setting effects could not be expected if assigned goals are coercive and not accepted by the individual. Moreover, giving employees ability training, success experiences, a role model, or expressing trust and confidence in them may enhance self-efficacy and

(15)

consequently goal achievement (Bandura, 1997; White & Locke, 2000).

Secondly, simply setting the goals at the beginning of a task with no input provided during the course of action may not lead to satisfactory goal attainment. Timely information on performance outcome and level of achievement could lead the individual to adjust follow-up behaviors. If interim goal attainment is below

expectation, people would usually exert more effort or refine strategies to catch up. Generally, feedback is necessary. But the optimal frequency of feedback depends on the nature of task. For example, giving truck drivers daily or weekly report on what they have completed may be informative. On the contrary, providing hourly weight records would be unnecessary for weight watchers. Too much information may sometimes interfere with goal-directed behaviors and cause extra anxiety (Locke & Latham, 1990). It is, however, not so straightforward in determining the optimal level of information feedback under different circumstances for different tasks. This aspect of effective goal-setting is particularly relevant to a formative assessment pedagogy, in which the concept of performance standard is infused in learners at the onset and they learn to self-assess own performance against the target standard. For one who has already learned about self-assessment and could provide him/herself feedback,

goal-setting may be expected to function differently than those without adequate self-assessment knowledge. But how this knowledge may influence goal-attainment behaviors and, if there is an influence, whether it is facilitating or debilitating is subject to further study.

Finally, task complexity is an important variable in goal-setting mechanism. Earlier studies on goal-setting were mostly conducted using simpler tasks, such as putting labels on pencils. The effects of goals in regulating these behaviors were

(16)

9

obvious. More recent studies extend to more complex tasks such as the acquisition of complicated skills like playing golf or the increase of market share to a certain level. Our understanding on goal-setting effects is thus becoming more sophisticated. Empirical studies providing evidence for the efficacy of goal-setting in complex tasks have accumulated over the years. A few properties of effective goals have been repeatedly tested and confirmed (Locke & Latham, 2002), including goals being difficult, proximal, and specific. First, given the ability to reach the goal, more difficult goals elicit more efforts and hence higher performance. Difficult goals are usually better than “do your best” goals, which lack clear reference to results and therefore could not regulate behaviors. Without a certain level of difficulty, goals could be reached without much effort and the original intention of goal-setting in motivating more effort is sacrificed. Second, proximal goals help in dividing an overall distal goal into more manageable smaller units and thus the goals become more achievable. Proximal goals have been demonstrated to be facilitative (Seijts & Latham, 2001). Third, specific goals have been shown to exert more effort than a vague “do your best” goal (Locke & Latham, 2006).

Performance Outcome Goals versus Learning Goals

Research has shown that goal-setting does not always lead to better performance than not having a goal. Kanfer and Ackerman (1989) indicated a condition when goals could be deleterious. Because goal-setting is not just a theory of motivation; it is a theory of both motivation and ability (Seijts & Latham, 2005; Seijts, Latham, Tasa, & Latham, 2004). In performing a difficult task for which one lacks the necessary ability, high motivation stimulated by goals alone is not enough. When the degree of task complexity is high relative to an individual’s capability, the discovery of new skills and strategies becomes part of goal-achieving efforts. The necessary skills are yet to

(17)

be automatized and require conscious self-monitoring and the use of limited cognitive capacity. Under these conditions, high performance outcome goals may not be

facilitative and, if not set appropriately, may cause individuals to scramble in what they do and feel anxious. Seijts and Latham (2005) make a distinction between performance outcome goals and learning goals. As they postulate, goals may be framed in different ways. Although the workplace manager’s ultimate concern may be the same performance outcome, a goal could either be framed “so that the focus is on performance, (e.g., decrease costs by 10 percent this quarter)”, or be framed “to focus attention on knowledge or skill acquisition (e.g., find ten ways of developing a relationship with end-users of our products)” (Seijts & Latham, 2005, p. 125). This distinction is analogous to another dichotomy, that of product (i.e. performance outcome) and process goals (i.e. learning). It is argued that under difficult tasks, a learning goal which directs attention to the discovery of strategies and mastery of skills and away from the performance outcome and stress is more ideal than a performance goal.

Nevertheless, findings from empirical studies are not so straightforward. Recent results favoring learning goals as opposed to outcome goals include a task in a complex business simulation for maximizing market share (Seijts, Latham, Tasa, & Latham, 2004) and one in a laboratory setting using a class-scheduling task (Seijts & Latham, 2001). In both studies, performance as well as goal commitment is higher under specific, difficult learning goals as opposed to no goal or performance goals. Contrary to the above findings, in a physical education setting where learners try to acquire a dart-throwing skill, no difference was found among process goal,

performance outcome goal, and a combination of both (Kolovelonis, Goudas, & Dermitzaki, 2011). What benefits in acquiring this physical skill, rather, is

(18)

11 self-recording.

Goal-setting in Academic Learning Situations

Goal-setting has been much less frequently applied in academic learning situations in comparison to work situations and results to date were mixed. First, effect of goal setting in school learning has been illustrated with learners as young as age five to six (Palmer & Wehmeyer, 2003). Quite a number of goal-setting studies in education are concerned about special education for learners with disabilities. For

example, Page-Voth and Graham (1999) helped 7th- and 8th-grade students with

writing and learning disabilities set goals for their essay writing. The goals designed to increase the number of arguments or counterarguments did help students write longer and better. But positive effects of goal-setting were absent in another study on learners with learning disabilities. Johnson, Graham, and Harris (1997) taught grade school students reading comprehension strategies. Their results indicated that reading strategy instruction produced meaningful, lasting and generalizable effects on students’ story comprehension skills. However, explicit instruction in goal setting and

self-instruction did not improve the comprehension performance of these students.

Going beyond the limits of special education, the following example may well illustrate how goal-setting for complex learning in tertiary-level education has been experimented. In a college psychology course, Fleming (2002) improved students’ exam performance by introducing goal-setting and study strategies. Her experimental group was given two kinds of forms – goal-setting and activity report. The

goal-setting forms contained three learning activities: reviewing notes, reading, and studying material. Listed under each category were seven descriptions of learning strategies, such as “reading chapter” or “test myself on what I read.” Students were

(19)

asked to note down the number of minutes they planned to spend on each of the three activities and circled the strategies they intended to use. The activity forms were parallel to the goal-setting forms in content, except that same descriptions were phrased in past tense and learners recorded the actual time spent on three activities. The relevant learning strategies were addressed in the content of Unit 2 in the course itself, so the instructor/researcher did not spend additional time discussing the strategies, nor did she grade the forms. Learners knew this and simply handed in the forms as a class routine. Although the control group learned the same strategies in the same way, they performed less satisfactorily on exams than their peers who were aided by the self-regulatory mechanism these two forms afforded. The kind of goals in this study caters to strategies learners need, helps them regulate behaviors, and

demonstrates positive effects.

But a goal-setting study by Shih and Alexander (2000) situated in Taiwan for fourth-grade pupils learning math fraction showed that goal-setting had no effect. What influenced learners was the referent of feedback. Those in a self-referenced feedback group performed better than those in a social-referenced group. The goals they set for learners were the number of problems solved correctly. This, by definition, is an outcome goal, rather than a learning goal.

In the context of foreign language learning, two recent studies found goal-setting to be less favorable. First, Huang (2008) compared two different types of process goals with a control group and examined their effects on foreign language learners’ oral production. Participants were put into three groups of conversation dyads: control (do your best), meaning-focused (produce more argument points), and form-focused (mind syntactic and grammatical aspects of oral output). Results indicate that the three

(20)

13

groups were not different in language accuracy, fluency and complexity. The only difference lies in number of turns of conversation. That is, the control group dyads produced significantly more words and conversational turns than the other two. It was suggested that in the absence of a goal, while learners were busy enough dealing with their communication tasks, they were less restrained cognitively and could focus more on their oral practice and produce more turns. The real time communication demand taxed learners’ cognitive capacity and extra goals may inhibit their effort. The nature of the oral conversation task and its result resonate with Kanfer and Ackerman’s (1989) resource allocation model. But a question remains. Whether these form-focused or meaning-focused goals are learning or performance goals is subject to discussion. The definition of learning versus performance goals may not be so straightforward. The second one is Kato (2009). Framed under the topic of learner autonomy and

self-regulation, this study on college learners of Japanese, requires students to record and review their learning activities on a weekly basis as either under goal-setting or self-assessment. Results indicated that both were evaluated as effective by more than half of the students. But student surveys revealed that learners prefer self-assessment to goal-setting. Moreover, beginners seem to benefit more from such interventions than do advanced learners.

Revision in the writing classroom

Many experienced writers consider revision an indispensible part of the entire writing process. Yet, unlike feedback on student writings being considered by many writing teachers as a must, revision is seldom explicitly taught. Despite the main purpose of feedbacks in helping learners improve their writing, many feedback messages are not heard or, if heard, not understood. Even when they are understood, students often do not know what to do with them. In a Singaporean English context,

(21)

Silver and Lee (2007) found that, although 80% of students responded to teacher feedback and revised their drafts, only 10% of these revisions were successful. They concluded that it is insufficient to simply provide feedback and expect students to revise. Butler and Britt (2011) found similar problems with their American college freshmen in argumentative writing. They designed written tutorials to inform learners of both a revision and an argumentation schema. Their findings suggested that

revision, especially global revision, could be improved by instruction. And they argued that such revision instruction was especially critical for learners who had been used to having their writings corrected for grammar and local errors only. Similar effects from revision lessons were also found in Sengupta (2000) for Hong Kong secondary students’ writing performance.

What are entailed in revision and the teaching of revision? Revisions are considered both a goal- and a problem-oriented process (Williams, 2004), or a goal-driven and problem-solving one (Butler & Britt, 2011). Williams distinguishes the three linear stages of revision as detection/evaluation/comparison,

diagnosis/identification, and operation/execution/correction. These are sometimes condensed in a two-stage model (e.g. Myhill & Jones, 2007) requiring two types of abilities. First, writers have to be able to identify areas for improvement. And secondly, they need access to various techniques to revise those areas for the better. Revision is not easily automatized – it requires conscious effort and self-monitoring. Revision strategies are usually classified into metacognitive, cognitive, and social strategies, or global revisions of meaning and structure and local ones of grammar and lexis. Writers may add, delete, substitute, or reconstruct existing texts. During the revision process, reference resources such as dictionaries or opinions from others are usually sought.

(22)

15

Setting goals for revision

Despite the fact that revision is often regarded as a goal-driven activity,

goal-setting is seldom utilized in the teaching of writing revision. However, one such study proved to be quite informative. Zimmerman and Kitsantas (1999) devised a well-defined revision task in which high school girls were asked to combine a given series of kernel sentences into a single non-redundant one. There were an outcome goal group focusing on minimizing the final number of words, a process goal group emphasizing a three-step method, and another group shifting sequentially from process to outcome goals. Results showed that the shifting-goal group outperformed the process-goal group, which in turn outperformed the outcome goal group. The authors concluded that it is the progressive mastery of hierarchical goals that made learners more independent and more self-motivated, and eventually performed better than the other students. Process goals at the beginning helped learners focus on their learning. Once they were more familiar with how to deal with their work, the outcome product goal of reducing the number of words had a positive effect.

Although goal-setting research has been well established in other fields and the effectiveness demonstrated, its application and influence in second/foreign language teaching and learning is largely unknown. Existing studies render mixed results. As reviewed above, Huang (2008) suggested the possibility of interference from

goal-setting on learners’ real-time oral production, and Kato (2009) demonstrated the superiority of self-assessment over goal-setting as self-regulatory tools among

learners of Japanese. More positive results are found in Zimmerman and Kitsantas (1999) using shifting goals for writing revision. But their very straightforward task of combining several sentences into one reflects only a small part of the whole spectrum

(23)

of challenges of writing revision in naturalistic settings. With the complexity of revision tasks faced by L2 writers, how may goals be framed to facilitate revision? Given some choice, what types of revision goals, such as content or language, would learners adhere to? In addition, taken into consideration the complexity of revision and the conscious effort required, will the provision of strategies complement the insufficiency of goals and help learners allocate their resources better? These questions guided the design of this study, as explained below.

The Study

The above literature review started with assessment for learning and its implication for instructional practices that empower students as the owner of their learning. It then continues with how goal-setting, a theory of both motivation and ability, may accommodate the entire process of assessment for learning. Research findings on goal setting and its application in academic learning situations have been synthesized and a number of questions have been raised regarding how goal setting could benefit the learning of writing revision in a foreign language.

Research Questions

The specific research questions are:

1. Does an assessment-for-learning pedagogy improve the quality of student writing from draft to revision?

2. Do the additions of goal setting or goal setting plus strategy provision make a difference in the quality of each student revision?

3. Given a series of draft-instruction-revision tasks, do student drafts improve over the three writing tasks?

(24)

17 over the three writing tasks?

Participants

One hundred and eleven college freshmen in a Taiwanese university participated in the study. They were enrolled in three different sections of the first semester of two of a four-credit required College English course taught by the same

instructor/researcher using the same syllabus and materials. In this integrated skill course, students met once each week for two fifty-minute sessions for eighteen weeks. Selected by two national entrance examinations, these students generally performed more satisfactorily than most of their peers academically. Specifically on their English test scores, out of a standardized scale of 15 with 15 being the highest and 0 the lowest, almost all students got scores of 15 or 14. The only two exceptions among these 111 got a 13 and a 12. Their high school English writing experience was mostly gauged around the composition part required in the entrance examination, which is usually a 120-word description of a series of four comic strips or a narrative of two paragraphs. The three intact sessions were randomly assigned into one of the three treatment groups, namely AfL only, AfL + goal-setting, and AfL + goal-setting + strategy list.

Instructional Procedures

The target writing genre in the instructional plan was argumentation in the form of a 300-word opinion essay similar to the last part of TOEIC writing. Students were expected, under the formative assessment pedagogy described in Huang (under second review), to understand the criteria and standards of good opinion essays, to assess their drafts against these criteria and standards, and to revise their drafts

(25)

and to be used in peer review and self-assessment is presented in Appendix A. Learners went through the same learning cycle for three rounds under three prompts on topics related to campus experience (see Appendix B). In weeks 3, 6, and 11, they wrote the draft and in weeks 4, 7, and 12 they worked on the revision. For each

writing cycle of two weeks, the three groups went through the same procedures except in the last period of fifty minutes when they worked on their revisions. Figure 2 illustrates the procedures.

All three groups started with drafting individually for 30 minutes without any pre-writing intervention from the instructor. After the draft was completed, the instructor guided them in reviewing and discussing a finished essay of the same topic written by a more experienced peer – their course TA. The instructional rubrics were used again in this discussion. Against the set criteria, the instructor evaluated the sample, showing learners what was done well and what could be improved and why. This discussed was designed as a model for the follow-up learner peer review in small groups. In groups of 3 or 4, learners rotated their drafts and provided evaluation to their peers using the same rubrics, giving both numerical figures and verbal comments. The authors then collected evaluation sheets from peers, clipped them together with the draft, reviewed them, and turned their work in before the end of the class.

(26)

19

1st session of 50 minutes 2nd session of 50 minutes

Week 1 Individual writing of the draft for 30

minutes

Teacher-led discussion of a TA

sample

Teacher modeling peer review using the TA

sample and instructional rubrics

Group peer reviews

At the end, learners collect reviews from peers and self-assess

Week 2

Teacher’s summary of assessment on drafts

and explanations of problems indentified

Teacher-led discussion on how to revise;

Group discussion on specific revisions

AfL Only Group

Individual revision for 30 minutes

Reflection on revision using worksheets (Appendix C1) AfL+ Groups Planning revision using worksheets 1. Appendix C2 (AfL + goal) or 2. Appendix C3 (AfL + goal + strategy)

Individual revision for 30 minutes Final check against worksheets

(27)

In the following week, the instructor returned the drafts to students and delivered a 50-minute lesson based on AfL principles. First, a summary of assessment on student drafts in bullet points was presented, including both areas done well and those in need of improvement. The focus was on the latter part, consisting of common problems identified, anonymous student excerpts illustrating the problems, and discussions on how the problems could be fixed. After the problems were pointed out to students, they were asked to discuss in small groups for a few minutes to come up with a collective solution. Later their revisions were presented to the class on the blackboard. The instructor then guided learners through these various revisions and discussed the plausibility of each. Different strategies for revisions were packed as mini lectures at the end to inform students of various approaches they could use. Up till this point, all three groups had the same writing experience. The only difference happened in the second period of the second week, when participants were asked to work on their revision. Revision worksheets for the three groups are presented in Appendix C.

The Experimental Groups

Control Group: AfL Only

This group was simply told to revise their draft. After thirty minutes when participants completed their revised version, they were given twenty minutes to self-assess the revised version using the same rubrics. It was at this point that they were given time to reflect on what they did for the revised version. As seen in Appendix C1, control group participants were asked to list in writing what they attempted and check those items they had completed.

(28)

21 Experimental Group 1: AfL + Goal Setting

This group was given a written prompt (Appendix C2) and fifteen minutes to work on it independently before they started to revise. The five criteria of the

instructional rubric, which had been discussed and used by learners in peer review and class discussion, were listed for them to circle the one or ones they wanted to focus on in revision. Students then wrote down on the blank table the strategies, or a to-do list, they had in mind for their revision. They were expected to draw on the same

instructional resources they had been exposed to as the other two groups. The

additional feature, compared to the AfL Only Group, was a goal-setting stage in which they were consciously primed to focus on one or more specific criteria they chose and list the strategies they planned to apply in revision. Once done with the revision, they were given five minutes to check their self-set lists and self-assess their revised work.

Experimental Group 2: AfL + Goal Setting + Strategy Provision

This group experienced the same goal-setting as the AfL + Goal-setting Group did using the same length of time. The additional feature was the provision of a strategy list right below the blank table (See Appendix C3). The list included general strategies for writing revision and may be more comprehensive and less specific compared to revision strategies discussed in class. It was believed that the list

provided an additional resource when learners set their learning goals for the revision work. Whether or not such resource served as a support or taxed their limited

cognitive capacity is one question to be answered in this study. Same as the AfL + Goal-setting Group, after completion of revision, students spent five minutes checking their self-set lists and conducted self-assessment.

(29)

Instruments and Measures

Quality of Drafts and Revisions

In order to evaluate the quality of student writing, two independent raters were paid to rate all learner drafts and revised versions. Blind to the experiment, they both rated on a holistic scale of 1 to 15, with 1 being of the poorest quality and 15 being the highest. One of them is an experienced EFL instructor and has been involved in the rating of English writing for national college entrance exams and the General English Proficiency Tests. The other is a research assistant from the English department of the same university.

Revision Worksheets

The three groups used different revision worksheets (Appendix C1, C2, and C3) that were designed to facilitate their revision. The AfL Only Group, after the revision instruction, was left on their own to work on the revision. Therefore, in the fourth class period of each writing unit, participants were given their drafts and told to revise. It was after these thirty minutes of revision that the worksheets were given to them. The first part of the worksheet enabled learners to record peer assessment on the draft as well as the learners’ self-assessment of both the draft and the revised work. The second part of the worksheet asked learners to reflect and record the strategies they used in revising the draft.

The other two groups were asked to do some planning before the revision started. For the AfL + Goal Group, the worksheet guided learners to set a goal for their

revision. They could choose among the five goals of holistic quality (H),

argumentation (A), organization (O), lexical use (L), and grammar (G). They could also decide to work on single or multiple goals. The setting of goals was meant to direct their attention to some selected specific aspects of the revision so they would

(30)

23

not feel lost in handling such a complicated task of revising a draft. The provision of choice rather than an arbitrary assignment of goals, according to past study findings, could make learners more committed to their own goals. Right below the area where they circled the goals, a table was provided to let learners write down strategies they intended to use in the coming up revision. Learners were given about fifteen minutes to plan their revision with the help of this worksheet. And later they started revising their drafts for a total of thirty minutes. At the end of the class period, about five minutes were allotted for them to check on the original strategy list for what was and was not done successfully. They then performed the same self-assessment the AfL Only Group did. Finally, they completed a five-item goal commitment questionnaire.

The AfL + Goal + Strategy Group went through the same procedures as the AfL + Goal Group did. The only difference was the provision of a strategy list at the time when they set goals and planned for revision. The list simply provided brief verbal descriptions of revision strategies discussed in the feedback lessons given to all three groups. According to studies comparing learning goals and performance goals, the provision of strategies, in addition to setting goals, is crucial when the tasks are complicated and involve learning.

Data Analysis and Results

Inter-rater reliability was first calculated between the two independent raters. As shown in Table 1, their ratings were all positively correlated at significant levels except for the second revision of the AfL + Goal Group. After discussions with statisticians, it was decided that the figures were generally acceptable and further rating may contaminate the results. However, cautions should be exercised when interpreting results regarding the second revision of the AfL + Goal Group.

(31)

Table 1. Inter-rater Reliability

Groups Task Statistics Draft Revision

AfL Only 1st Pearson Correlation Coefficient r 0.640 0.578

Significance (two-tailed) p (0.000) (0.000)

2nd Pearson Correlation Coefficient 0.622 0.506

Significance (two-tailed) (0.000) (0.002)

3rd Pearson Correlation Coefficient 0.860 0.834

AfL + Goal 1st Pearson Correlation Coefficient 0.576 0.784

AfL + Goal 1st Pearson Correlation Coefficient 0.789 0.882

+ Strategy Significance (two-tailed) (0.000) (0.000)

The second examination was on the first draft between the three groups. If the three groups differed significantly from each other, their draft scores would be used as covariates in further comparisons. Results of analysis of variance on the first draft were shown in Table 2 and Table 3.

Table 2. Summary of ANOVA on First Draft Among Three Groups

df F p η2_obs._power

Groups 2 9.21*** .000 .150 .974

(32)

25

Table 3. Follow-up Multiple Comparisons Using Tukey Tests

(I) Group (J) Group (I-J) p 95% CI

AfL Only AfL +

Goal -0.51 0.484 [-1.56, 0.54] AfL + Goal + Strategy 1.43** 0.006 [0.35, 2.50] AfL +

Goal AfL Only 0.51 0.484 [-0.54, 1.56]

AfL + Goal + Strategy 1.94*** 0.000 [0.83, 3.05] AfL + Goal + Strategy AfL Only -1.43** 0.006 [-2.50, -0.35] AfL + Goal -1.94*** 0.000 [-3.05, -0.83]

The above information indicated that the three groups, as measured by their first drafts, were not the same. The third group, AfL + Goal + Strategy, scored

significantly lower than the first group, AfL Only, by 1.43, and further lower than the second group, AfL + Goal + Strategy, by 1.94. The differences between the AfL + Goal + Strategy Group and the other two were statistically significant. This difference was taken into consideration in follow-up analyses.

Descriptive statistics, i.e, the average holistic scores between the two raters of each group and each piece of writing, are shown in Table 4. The same information was depicted in Figure 3 as three lines representing the changes of scores over time for each of the three groups. As illustrated in the line graph, all groups performed better in revisions than in the relevant drafts. AfL Only Group improved from the first draft to the first revision, and further from the second draft to the second revision; but

(33)

there was a sharp drop in the third draft before the score picked up in the third revision. For the AfL + Goal Group, they performed better than the other groups in the first task. Their second draft and revision were relatively the same as the scores they obtained in the first round. But the drop in the third round was more obvious than the AfL Only Group. The AfL + Goal + Strategy Group, however, started much lower than the other two groups, but their improvement over the three tasks was steady. Eventually in the third round, they outperformed the other groups and obtained the highest scores among all three.

Table 4. Descriptive Statistics of All Drafts and Revisions by All Three Groups

AfL Only (n=40) AfL + Goal (n=35) AfL + Goal + Strategy (n=33)

M SD 95% CI M SD 95% CI M SD 95% CI 1st Draft 8.16 1.74 [7.60, 8.72] 8.67 1.69 [8.09, 9.25] 6.73 2.29 [5.91, 7.56] Revision 9.36 1.76 [8.78, 9.95] 9.66 1.79 [9.01, 10.32] 8.54 2.09 [7.73, 9.34] 2nd_Draft 9.50 1.29 [9.07, 9.93] 8.73 1.44 [8.22, 9.24] 7.61 2.30 [6.78, 8.44] Revision 10.23 1.30 [9.78, 10.68] 9.60 1.08 [9.15, 10.05] 8.98 2.17 [8.19, 9.78] 3rd _Draft 7.81 2.25 [7.03, 8.59] 6.65 2.56 [5.71, 7.58] 8.43 1.43 [7.89, 8.98] Revision 8.28 2.01 [7.63, 8.93] 7.89 2.43 [6.84, 8.94] 9.56 1.34 [9.02, 10.10]

(34)

27

Figure 3. Line Graph of Writing Scores for Three Tasks from Three Groups

6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 初稿1 修改稿1 初稿2 修改稿2 初稿3 修改稿3 班級1 班級2 班級3

(35)

For research question one on whether learner revisions improved from drafts, each revision score was subtracted by its associated draft score. Descriptive statistics of the resulted figures are presented in Table 5 with means, standard deviations, and 95% confidence intervals. As shown, all the means are positive and none of the range of confidence intervals include zero, which indicated that the improvement on all writings for all groups was statistically significant.

Table 5. Descriptive Statistics of Draft-to-Revision Improvement of Three Groups on Three Tasks

1st Writing 2nd Writing 3rd Writing

Groups M SD 95% CI M SD 95% CI M SD 95% CI

AfL Only 1.22 0.83 [0.94, 1.49] 0.68 0.7 [0.43, 0.92] 0.62 0.71 [0.37, 0.86]

AfL + Goal 1.18 1.14 [0.76, 1.60] 1 0.82 [0.66, 1.34] 0.8 1.19 [0.29, 1.32]

AfL + Goal

+ Strategy 1.71 1.65 [1.08, 2.35] 1.37 1.83 [0.70, 2.04] 1.04 0.87 [0.69, 1.39]

The next question to ask is whether the improvements among the three groups differ in magnitude. The null hypothesis is that the three groups did not differ in terms of levels of improvement. Before comparing the three groups, test of the homogeneity of regression coefficients was conducted (results shown in Table 6). It was found out that the interactive effect between independent variable (Group) and covariate (scores of the first draft) was not significant (all p values are larger than 0.05), indicating that the regression coefficients were homogeneous, which renders justification for the follow-up ANCOVA test.

Table 6. Summary of the Test of Regression Coefficient Homogeneity

1st Writing 2nd Writing 3rd Writing

Source of Variance df F p df F p df F p

Scores of 1st Draft 2 2.18 0.119 2 2.03 0.137 2 0.12 0.890

Group 1 10.80** 0.001 1 0.19 0.664 1 3.97* 0.050

Scores of 1st Draft ×

Group 2 1.96 0.147 2 1.36 0.263 2 0.17 0.841

(36)

29

ANCOVA analyses were conducted on the improvement between drafts and revisions for all three groups in all three tasks. The results, as shown in Table 7, indicated that for all three tasks, the p values are above the significance level of 0.05. That is to say, looking at each revision separately, the three groups did not differ from each other in terms of the levels of improvement.

Table 7. Summary of ANCOVA on Improvement among Three Groups in Three Tasks

ANCOVA Summary on the 1st Writing

Source of Variance df F p η 2_obs._power Score of First Draft 1 14.34*** 0.000 0.135 0.963 Group 2 0.22 0.803 0.005 0.084 Within Group Errors 92 (1.29)

ANCOVA Summary on the 2nd Writing

Source of Variance df F p η 2_obs._power Score of First Draft 1 0.51 0.479 0.006 0.108 Group 2 1.62 0.203 0.037 0.334 Within Group Errors 85 (0.94)

ANCOVA Summary on the 3rd Writing

Source of Variance df F p η 2_obs._power Score of First Draft 1 3.79 0.055 0.046 0.485 Group 2 0.84 0.437 0.021 0.189 Within Group Errors 79 (0.81)

Research question three is concerned about learners’ performance on drafts and revisions over time, that is, comparisons among all three drafts and among all three revisions. First, repeated measure ANOVA was conducted on three drafts. As shown in Table 8, the difference among three groups was not significant. That is to say, none of the three groups improve more than others from draft to draft. Results of follow-up

(37)

multiple comparisons using Bonferroni tests were shown in Table 9. For the groups, scores on the second draft differed from the first and the third ones.

Table 8. ANOVA Summary on Three Drafts

Source of Variance df F p η2_obs._power

Group ₂ _2.64 _.077 _.057 _.512

Within Group Errors ₈₇ _(6.90)

Table 9. Multiple Comparison on Three Drafts Using Bonferroni Test

(I)Writing (J)Writing Difference of Means_(I‐J) p Difference 95% CI

1 2 ‐0.73** 0.001 [‐1.21, ‐0.24] 3 0.35 0.350 [‐0.19, 0.89] 2 1 0.73** 0.001 [0.24, 1.21] 3 1.08*** 0.000 [0.52, 1.63] 3 1 ‐0.35 0.350 [‐0.89, 0.19] 2 ‐1.08*** 0.000 [‐1.63, ‐0.52]

The second part of research question three is concerned about the difference among three revisions in three groups. Test of homogeneity, as shown on Table 10, indicated that the interactive effect between independent variable (Group) and covariate (score of first draft) was significant (p value smaller than 0.05). Therefore, the estimated regression coefficients were calculated and presented in Table 11 for use in follow-up analyses.

Table 10. Summary of the Test of Regression Coefficient Homogeneity on All Revision Scores

Source of Variance df F p

Group 2 6.56** .002

Score of 1st Draft 1 98.39*** .000

Group x Score of 1st Draft 2 4.64* .013

Within Group Errors 69 (2.72)

(38)

31

Table 11. Estimated regression coefficients for first draft (X) and revisions (Y)

Writings

Group 1st 2nd 3rd

AfL Only 0.901*** 0.372** 0.441*

AfL + Goal 0.843*** 0.243* 1.214***

AfL + Goal + Strategy 0.64*** 0.402* 0.109

Results of repeated measure ANOVA on three revisions among three groups, as shown in Table 12, suggested that the three groups differed in their three revision scores. Multiple comparisons in Table 13 showed that the AfL + Goal + Strategy Group, when all three rounds of writing were taken into consideration, performed better than the AfL Only Group in revisions, and the AfL Only Group in turn performed better than the AfL + Goal Group.

Table 12. ANOVA Summary on Three Revisions

Source of Variance df F p η2 obs. power

Group ₂ _155.65*** _.000 _.812 _1.000

Within Group

Errors 72 (1.07)

Table 13. Multiple Comparison on Revisions by Groups Using Bonferroni Tests

(I)Group (J)Group Mean Difference

(I-J) p Difference 95% CI AfL AfL + Goal 1.61*** 0.000 1.18 2.04 AfL + Goal + Strategy -1.69*** 0.000 -2.08 -1.29 AfL + Goal AfL -1.61*** 0.000 -2.04 -1.18 AfL + Goal + Strategy -3.30*** 0.000 -3.76 -2.84 AfL + Goal + Strategy AfL 1.69*** 0.000 1.29 2.08 AfL + Goal 3.30*** 0.000 2.84 3.76

(39)

Multiple comparisons were performed among the three rounds of writing, as shown in Table 14. When all three groups were assessed together, they performed best in the second revision, followed by the third and then the first.

Table 14. Multiple Comparison on Revisions by Writings Using Bonferroni Tests

(I)Writing (J)Writing Mean Difference_(I‐J) p 95% CI of the _Difference

1st ₂nd _‐4.56*** _0.000 _‐4.80 _‐4.32 3rd _‐2.01*** _0.000 _‐2.33 _‐1.69 2nd ₁st _4.56*** _0.000 _4.32 _4.80 3rd _2.55*** _0.000 _2.23 _2.87 3rd ₁st _2.01*** _0.000 _1.69 _2.33 2nd _‐2.55*** _0.000 _‐2.87 _‐2.23

Discussions and Implications

In sum, results of statistical analyses provided clear answers to the three research questions that guided this study. First of all, all three groups, instructed under the assessment-for-learning framework, regardless of having goals or being provided with a strategy list or not, improved from each draft to each associated revision, and the improvement was all significant. But the draft-to-revision improvement among three groups was indistinguishable. This means that single improvements from draft to revision, as long as the “assessment for learning” design of instruction was present, were not strengthened by either the addition of goal setting or strategy list.

Furthermore, when student performance was compared across three tasks, we found that the draft scores among the three groups were not significantly different. However, the differences among three revisions were. To be more specific, the AfL + Goal + Strategy Group performed better than the AfL Only Group, which in turn performed significantly better than the AfL + Goal Group.

The results bear significance in both theory and practice. First, the consistent significant improvement from drafts to revisions for all three groups in all three tasks suggested that the “assessment for learning” ideal as advocated in the literature worked well for EFL learners who were learning to improve their essay writing. We have learned from past L2 writing studies that learner revisions are not always better than their drafts (e.g. Lee, 2007; Silver & Lee, 2007). Oftentimes the revisions are not

(40)

33

better than the drafts, even when teachers provided individual feedbacks. However, in this study, although individual feedbacks were not given to students, the instructional design as shown in Figure 2 worked. Students, after being guided to understand the quality desired and to assess the quality of their own works, were given detailed explanations on pervasive patterns of problems identified by the instructor. They were then given a chance to revise selected experts and discuss the revisions. This type of instruction was meant to empower learners to apply the principles into their own learning, diagnosing own problems and finding solutions. This is an encouraging finding as teachers may have a chance to be liberated from the laborious commenting of individual work and instead to focus on teaching learners better ways of revision.

Secondly, we learned from the answer to the second research question that the addition of goal setting and the provision of a strategy list did not make a difference when each revision was examined. That is to say, learners performed equally in revising their drafts, either with goals or not and either with a strategy list or not. This result seemed to suggest that the three groups were the same. But we have to look at the question in more depth. Research question two, when examined together with questions three and four, revealed important messages. The last two questions allowed us to investigate this study across different learning tasks over time. Although there was no difference among three groups when draft-to-revision improvements were examined separately for three tasks, if the effect is present when all three writings were considered, it has important implications. Our answer from research question three was a negative one, suggesting that there was no group effect from one draft to another. But the answer to question four was a positive one. AfL + Goal + Strategy Group performed the best, followed by the AfL Only Group, and further by the AfL + Goal Group. It means that the addition of goal setting plus a strategy list was most helpful in facilitating learner revision. The addition of goal only, however, was the worst among the three groups, rendering inferior results than the AfL Only Group. This finding coincides with previous findings in theories and from other studies in different learning situations and will be discussed below.

Goal setting, as suggested by Kanfer and Ackerman (1989), Seijts and Latham (2005), and Seijts, Latham, Tasa, and Latham (2004), may bring potential damages.

(41)

When the task is complex and requires learning and when the performer lacks necessary strategy and skills, performance outcome goal may become a source of anxiety and lower the performance. It would be worse than not having a goal at all. That is why scholars advise the addition of strategies to augment goals. In this study, the findings confirmed past conclusions in the EFL writing revision situations. Although the difference among treatments was not found when revisions were examined separately, it was obvious when three tasks were considered aggregately. The impact was not robust enough to be seen in any single writing task or across three drafts. But when three rounds of revisions were considered together, the effect of learning from one task to another in the AfL + Goal + Strategy Group was observed. Learners benefited most by goals and strategies. And goals alone resulted in the least favorable result as it may have already aroused stress in learners for the complicated revision task they were required to do. It was better not to set goals if a strategy list is not provided.

This study was probably one of the very few that applied goal setting theories in foreign/second language learning contexts, in particular EFL writing revision. The combination of assessment for learning with goal setting has its theoretical

background and the results indicated that this is a viable direction for EFL teachers to consider using these strategies in instruction to empower students as self-regulated learners. Goal setting, when applied in learning situations, is much more complicated than in work settings, because learning goals could be defined in various ways and many learning tasks are comparatively more complex than work-related tasks. The results are encouraging and warrant more study in this direction.

(42)

35 References

Butler, J. A., & Britt, M. A. (2011). Investigating instruction for improving revision of argumentative essays. Written Communication, 28(1), 70-96.

Fleming, V. M. (2002). Improving students’ exam performance by introducing study strategies and goal setting. Teaching of Psychology, 29, 2, 115-119.

Johnson, L., Graham, S., & Harris, K. R. (1997). The effects of goal setting and self-instruction on learning a reading comprehension strategy: A study of

students with learning disabilities. Journal of Learning Disabilities, 30(1), 80-91. Klein, H. J., & Lee, S. (2006). The effects of personality on learning: The mediating

role of goal setting. Human Performance, 19, 1, 43-66.

Klein, H. J., Wesson, M. J., Hollenbeck, J. R., Wright, P. M., & DeShon, R. P. (2001). The assessment of goal commitment: A measurement model meta-analysis.

Organizational Behavior and Human Decision Processes, 85, 32-55.

Kobayashi, H., & Rinnert, C. (2001). Factors relating to EFL writers’ discourse level revision skills. International Journal of English Studies, 1, 2, 71-101.

Kolovelonis, A., Goudas, M., & Dermitzaki, I. (2011). The effect of different goals and self-recording on self-regulation of learning a motor skill in a physical education setting. Learning and Instruction, 21, 355-364.

Latham, G. P., & Locke, E. A. (2006). Enhancing the benefits and overcoming the pitfalls of goal setting. Organizational Dynamics, 35(4), 332-340.

Latham, G. P., & Locke, E. A. (2007). New developments in and directions for goal-setting research. European Psychologist, 12, 4, 290-300.

Lee, I. (2007). Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning. Assessing Writing, 12, 180-198.

Lee, I. (2011). Formative assessment in EFL writing: An exploratory case study.

Changing English, 18, 99-111.

Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall.

Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current

Directions in Psychological Science, 15, 5, 265-268.

Myhill, D., & Jones, S. (2007). More than just error correction – Students’

perspectives on their revision processes during writing. Written Communication,

(43)

Ong, J., & Zhang, L. J. (2010). Effects of task complexity on the fluency and lexical complexity in EFL students’ argumentative writing. Journal of Second Language

Writing, 19, 218-233.

Page-Voth, V., & Graham, S. (1999). Effects of goal setting and strategy use on the writing performance and self-efficacy of students with writing and learning problems. Journal of Educational Psychology, 91(2), 230-240.

Palmer, S. B., & Wehmeyer, M. L. (2003). Promoting self-determination in early elementary school – Teaching self-regulated problem-solving and goal-setting skills. Remedial and Special Education, 24(2), 115-126.

Seijts, G. H., & Latham, G. P. (2001). The effect of distal learning, outcome, and proximal goals on a moderately complex task. Journal of Organizational

Behavior, 22, 291-307.

Seijts, G. H., & Latham, G. P. (2005). Learning versus performance goals: When should each be used? Academy of Management Executive, 19, 1, 124-131. Sengupta, S. (2000). An investigation into the effects of revision strategy instruction

on L2 secondary school learners. System, 28, 97-113.

Shih, S.-S., & Alexander, J. M. (2000). Interacting effects of goal setting and self- or other-referenced feedback on children’s development of self-efficacy and cognitive skill within the Taiwanese classroom. Journal of Educational

Psychology, 92(3), 536-543.

Silva, T., & Brice, C. (2004). Research in teaching writing. Annual Review of Applied

Linguistics, 24, 70-106.

Silver, R., & Lee, S. (2007). What does it take to make a change? Teacher feedback and student revisions. English Teaching: Practice and Critique, 6(1), 25-49. Seijts, G. H., Latham, G. P., Tasa, K., & Latham, B. W. (2004). Goal setting and goal

orientation: An integration of two different yet related literatures. Academy of

Management Journal, 47, 2, 227-239.

Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College Composition and Communication, 31, 4, 378-388.

VandeWalle, D., Cron, W. L., & Slocum Jr., J. W. (2001). The role of goal orientation following performance feedback, Journal of Applied Psychology, 86, 4, 629-640. Williams, J. (2004). Tutoring and revision: Second language writers in the writing