3.2 Instruments
3.2.7 Self-evaluation Form on Portfolio and Questionnaire on the
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
At the end of the semester, individual portfolios were handed in, including: a) written drafts, b) peer review sheets, c) checklists, and d) content on blogs (revisions, self-reflection journals, brainstorming and comments). Other than the e-portfolio on the cyberspace (the class blog), the paper-based portfolios served as a hard copy of database for the evaluation of learners’ performance and for research purpose.
3.2.7 Self-evaluation Form on Portfolio and Questionnaire on the Writing Class In order to conduct the interviews, the four participants were asked to fill out the self-evaluation forms on portfolio, and questionnaires on the writing class (see
Appendix F). The self-evaluation forms on portfolio aimed to help participants to reflect on their writing process as well as how they felt about the two drafts on Topic 1 & 5 (pre-test and post-test drafts). Questionnaires on the writing class helped the researcher to conduct the subsequent interviews with these four participants.
3.2.8 Modified CEEC Scoring Criteria
The drafts on Topic 1 (pre-test) and those on Topic 5 (post-test) of these four participants were graded based on a modified CEEC (College Entrance Examination Center) scoring criteria (Chang, 2004), which fall into five categories: organization (5 points), content (5 points), vocabulary (4 points), grammar (4 points) and mechanics (2 points). The CEEC scoring criteria were modified to enlarge the score gap of each category as below: organization (25 points), content (25 points), vocabulary (20 points), grammar (20 points) and mechanics (10 points) (see Appendix G), for the purpose of helping the researcher to examine the amount of progress more precisely.
3.2.9 Yagelski’s Coding Schemes for Revision Analysis across Cases (See Appendix H)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
To code for frequency and types of revisions in these 48 revisions (4 participants wrote 3 revisions on 4 topics), Yagelski’s coding schemes, (1995) and Johnson’s indicators of content and organization (1994) were adopted. Given that content and organization were the limelight of this study, Johnson’s indicators of content and organization acted as the primary coding schemes in counting content and structural changes while Yagelski’s coding schemes, as secondary sources, provided standards for categorization. In Yagelski’s coding schemes, the types of revisions were
categorized into four changes: a) surface changes, b) stylistic changes, c) structural changes, and d) content changes. The first three referred to what Faigley and Witte (1981) called “meaning-preserving” changes, which showed writing conventions (surface changes), word choice (stylistic changes); and the organization of the text (structural changes) respectively. The fourth referred to the content that affected the meaning of the text. One thing worthy of note in Yagelski’s coding schemes is that grammar-related revision was categorized into the surface changes.
3.2.10 Johnson’s Indicators of Content and Organization
According to Johnson (1994), indicators of content and organization illustrated the existence of content and organization in an essay. Three types of indicators of content were therefore defined: 1) explanation: giving definition or further
explanation for the purpose of clarification, 2) details: providing more information, and 3) examples: illustrating the writer’s ideas more clearly. Likewise, there were three types of the indicators of organization: 1) topic sentences: expressing the main idea of a paragraph in one sentence, 2) supporting sentences: giving details, examples or explanation to support the topic sentence, and 3) transition words: connecting sentences closely and logically. Based on Johnson’s definition above, an addition of detail, example or explanation in the revision was seen as one content change.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Likewise, a new supporting sentence or topic sentence was also regarded as one structural change. Moreover, Johnson’s indicators of content and organization (1994) were adopted in the draft analysis to support the findings of progress in content and organization.
3.2.11 Semi-structured Paired Interviews (See Appendix I)
To clarify the unclear points in the gathered data, the researcher conducted one-hour paired interviews with these four participants at least four times. Paired interviews took the place of individual interviews for the participants’ preference.
According to Lohm (2008), paired interviews can reduce the discomfort in interviews, provide more time for thinking, and elicit more responses through interaction.
Questions were asked in the interviews to answer the four research questions or supplement the findings derived from revision analysis, draft analysis and
questionnaires. If a vague statement was made, a further one-to-one interview would be conducted to ask for more specific details.
3.3 Procedures
The present study was conducted in the process of six steps: 1) the draft on Topic 1 (pre-test) and the writing attitude questionnaire (pre-test), 2) writing cycles, 3) the draft on Topic 5 (post-test), 4) questionnaire on students’ writing attitude (post-test), 5) the self-evaluation form on portfolio and questionnaire on the writing class , and 6) the interviews. The research procedure was displayed as below:
‧
First, the drafts on Topic 1 were written and questionnaire on students’ writing attitude were filled out as pre-test (step 1). For the rest of semester, the writing cycles repeated as the training sessions of process writing (step 2). Then, drafts on Topic 5 were written as post-test (step 3). At the end of semester, questionnaires on students’
writing attitude were filled out again as post-test (step 4). In the 2nd period of the same week, self-evaluation form on portfolio and questionnaire on the writing class were filled out (step 5). Finally, semi-structured paired interviews were conducted (step 6).
To offer a better picture of the process writing model used in this study, writing cycles were introduced as below.
3.3.1 Writing Cycles
According to Applebee (1986), the process writing model should include five
‧
steps: pre-writing, drafting, revising, editing and publishing. However, these steps are not necessarily followed in a linear line. The writer can go back and forth till he/she is satisfied with the final essay; that is to say, some steps of process writing cycle are repetitive and recursive. Therefore, a writing cycle was devised to carry out what the process writing preaches, i.e., recursive writing. The procedure of writing cycle was listed as follows:
Stage 1 Giving a topic Brainstorming Written drafts
Stage 2
on the draft Peer editing sheets (peer review)
Writing cycle Writing cycle
Based on Perl’s (1980) findings, students write more with greater fluency and satisfaction when their writings relate to them personally. With this consideration in mind, five topics highly related to teenage life were chosen to write in the
semester-long course. Each writing cycle went through three stages and lasted at least three weeks (2 hours/week). At stage one, the written drafts were handed in. At stage two, revision 1 (R1) was requested to be posted on the blog after referring to the teacher’s feedback on the draft and the peer feedback on the peer editing sheet (peer
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
review). At stage three, revision 2 (R2) was published on the blog after finishing the checklists or referring to comments on the blog. Though two revisions were the minimal requirement for each writing cycle, the researcher encouraged learners to write as many revisions as they wished. Then, revision 3 (R3) was also posted on the blog after referring to the update comments on the blog from the teacher or peers. One thing worthy of note was that the topic “My personal profile” (Topic 1) was chosen to fit the pre-test that should exclude any variables involved, including brainstorming activity, to name just one.
To encourage students’ active error correction, this researcher replaced the traditional red-pen error correction with error marking in UCI correction symbols (See Appendix J) while offering comments on the content and organization on the margins of drafts. Such kind of practice should help students not only improve their writing product (academic or professional success) but develop the writing process (making meaning).
With the marked errors and the teacher’s feedback on the margins of the drafts, the peer review was conducted through peer editing sheets (See Appendix K). By documenting the teacher’s and peer’s feedback and reviewing the writing process in a self-reflection journal, R1 was posted on the blog. During the weekend, students first read each other’s revisions and then leave revision-related comments on the blog.
To facilitate the practice of the writing cycle in the study, this researcher gave a brief introduction to process writing in class. Also, to fit the unique feature of a reverse chronological order of blog, some format rules on the publication of essays on the blog were also established. What’s more, the structure of essay (e.g., topic
sentences, the main idea, and supporting sentences) was also presented in class as part of training session of the peer review.
To further ensure that R1 followed the process writing model focusing on the
‧
improvement in content and organization, each participant was asked to finish the Checklist on Process Writing & Essay (see Appendix L), which served as a guidance to R2 writing. Besides the two required revisions for each writing cycle, participants were also encouraged to repeat the recursive process until they came up with a satisfactory final essay.
3.3.2 Timetable for Teaching Activities and Data Collection
As a summary of this study, a timetable serves as an excellent framework of what the teacher did (teaching activities) and what data he/she collected (data collection). Teaching activities refers to how the researcher assisted students in completing a series of tasks while data collection relates to what data the researcher collected. The breakdown for the both was presented in the timetable below.
Date Teaching Activities Data Collection Sep. 1. Introduction to syllabus & class
blog
2. Questionnaire survey on students’ writing attitude (pre-test)
3. Writing drafts on Topic 1 (My personal profile)
1. Drafts on Topic 1 (pre-test) 2. Questionnaire sheets on students’
writing attitude (pre-test) 3. The teacher’s log
Oct. 1. Introduction to process writing and the structure of essay 2. Training sessions on peer
reviews
1. The teacher’s log
2. Students’ self-reflection journals 3. Multiple revisions on Topic 1 &
Topic 2
‧
2. Blog Management: A review on the previous writing cycles
1. The teacher’s log
2. Students’ self-reflection journals 3. Multiple revisions on Topic 3
Dec. 1. Practice of Writing cycle 4 (What’s in my name)
2. Writing the draft on Topic 5 (My best friend)
3. Questionnaire survey on students’ writing attitude (post-test)
1. The teacher’s log
2. Students’ self-reflection journals 3. Multiple revisions
4. Drafts on Topic 5 (post-test) 5. Questionnaire sheets on students’
writing attitude (post-test)
Jan. 1. Collection of portfolio
2. Filling out self-evaluation form on portfolio & questionnaires on the writing class
1. Individual portfolios
2. Self-evaluation forms on portfolio
& questionnaire sheets on the writing class
Feb.
End of semester 1. Data transcribed from audiotaped semi-structured paired interviews
3.4 Data Analysis
To answer the six research questions, this researcher collected data including multiple revisions (48 revisions), the two different drafts as pre-test and post test (Draft on Topic 1 and Draft on Topic 5), the same set of questionnaires on students’
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
writing attitude (pre-test and post-test questionnaires), the teacher’s logs and semi-structured paired interviews.
In response to Research Question One, the 48 revisions of these four participants were collected and coded by Yagelski’s coding schemes (1995) to count how many changes the four participants made in the four categories: surface changes, stylistic changes, structural changes and content changes. Since Yagelski’s coding schemes only offered rough standards for structural and content changes, Johnson’s indicators of content and organization (1994) were adopted as the guideline to counting content and structural changes. After these revisions were analyzed, several semi-structured paired interviews were asked for verification and clarification.
As to Research Question Two, the two drafts (pre-test and post-test) were graded by two raters with modified CEEC scoring criteria. First, score differences between the pre-test and post-test drafts were calculated to show the gained points. The gained points, indicating progress in the five categories (content, organization, vocabulary, grammar and mechanics), were also transformed into percentages of progress. Besides, the total numbers of words in two drafts (length of essay) were also counted to see if fluency was achieved. Second, the percentages of progress made by each participant were compared with the total numbers of changes he/she made in revisions to see if there was any connection between the progress in the two drafts and the practice of multiple revisions. Third, Johnson’s indicators of content and organization (1994) were applied in the draft analysis to support the above findings of progress in content and organization. Finally, the interviews helped this researcher to verify the findings or clarify the unclear points.
Answering Research Questions Three and Four, questions were asked in the interviews to find out how blog functioned in each step of process writing model or what challenges students encountered and what teacher support they could need.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
As to Research Question Five, the same set of questionnaires on students’
writing attitude (36 question items designed in five-point Likert scale) were collected to calculate the score differences between pre-test and post-test questionnaires. Then, this researcher explained the questionnaire results to see if the four participants experienced any attitude change.
In answering Research Question Six, data from the teacher’s logs and interviews will be investigated to see if some other findings were derived from the blog-mediated process writing class in term of the context of classroom.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER FOUR RESULTS
In this chapter, the results of the study are presented in the six research questions respectively. First, the 48 revisions are analyzed to find out what types of changes these four participants made in the multiple revisions. Second, the two drafts are graded and analyzed to explore their individual progress in content and organization.
Third, data from the interviews explain how the blog helped in each step (pre-writing, drafting, revising/editing and publishing) of process writing and what challenges and teacher support were. Fourth, the questionnaire results together with the interview data pinpointed if these students experienced any writing attitude changes after taking this writing class. Last, based on mostly on the teacher’s logs and the interview data, this researchers elaborates findings derived from the context of classroom
4.1 Research Question One: What types of changes students made in the multiple revisions in the blog-mediated process writing class and what information does the revision analysis provide?
To answer Research Question One, the researcher adopted Yagelski’ coding schemes (1995), standards for categorization, and Johnson’s indicators of content and organization (1994), the guideline to counting structural and content changes, to code the revisions of the four participants in four categories – surface change (mechanics and grammar), stylistic changes (vocabulary), structural changes (organization) and content changes (content). One thing worth notice is that grammar related revision was categorized into surface changes in Yagelski’s coding schemes.
The number of changes per topic for each participant is listed from Table 4.1a–
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Table 4.1d. In these tables, R1, R2 and R3 stand for revision 1, revision 2 and revision 3 respectively. This researcher will discuss the individual difference when making revisions.
Table 4.1a Numbers of Changes per Topic Written by Connie
Topic 1 Topic 2 Topic 3 Topic 4
R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3
Surface 3 0 1 3 3 0 2 0 1 8 1 0
Stylistic 2 1 4 8 2 0 1 2 1 4 4 0
Structural 4 0 0 3 1 0 4 3 0 5 0 1
Content 4 1 0 4 5 1 5 4 0 5 2 1
Total 13 2 5 18 11 1 12 9 2 22 7 2
The total numbers of changes in Connie’s revision showed a tendency to decrease. That is, Connie made the most changes in R1, fewer in R2 (with the exception of R2 on Topic 1) and then the fewest in R3. In making R1, she made changes in all of four categories. In R2, the total number of changes dropped sharply and the attention to the four categories was less obvious. In R3, the total number of changes drastically decreased because she only made minimal changes in one or two categories.
Table 4.1b Numbers of Changes per Topic Written by Tina
Topic 1 Topic 2 Topic 3 Topic 4
R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3
Surface 2 2 0 8 0 1 8 0 2 11 2 0
Stylistic 3 0 0 3 0 3 2 0 0 3 1 1
Structural 3 1 3 0 2 0 1 2 2 0 1 1
Content 3 1 3 0 2 0 4 3 4 1 1 1
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Total 11 4 6 11 4 4 15 5 8 15 5 3
In Tina’s case, the total number of changes didn’t necessarily follow a tendency to decrease. There were two times when the total numbers of changes decreased in R2, but rose slightly in R3 (Topic 1 and 3). The preference for paying attention to errors in the four categories in R1 only happened twice (Topic 1 & 3). Like Connie, Tina made the most changes in R1. However, the tendency to decrease wasn’t always found from R1 to R2 or from R2 to R3.
Table 4.1c Numbers of Changes per Topic Written by Ice
Topic 1 Topic 2 Topic 3 Topic 4
R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3
Surface 4 0 1 6 1 0 8 1 0 6 2 0
Stylistic 3 0 2 6 1 0 2 4 1 4 1 0
Structural 0 5 1 0 1 1 0 3 2 0 1 2
Content 0 5 1 0 1 1 0 4 2 0 1 1
Total 7 10 5 12 4 2 10 12 5 10 5 3
As shown in Table 4.1.c, the total number of changes in Ice’s revisions didn’t tend to decline. Unlike Connie’s and Tina’s R1, Ice’s didn’t always make the most changes in R1. On the contrary, the total numbers of changes in R2 outnumbered those in R1 for two times (Topic 2 & 3). While making revision on these four topics, Ice never made structural and content changes until R2.
Table 4.1d Numbers of Changes per Topic Written by Sherry
Topic 1 Topic 2 Topic 3 Topic 4
R1 R2 R3 R1 R2 R3 R1 R2 R3 R1 R2 R3
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Surface 12 1 1 4 6 0 6 2 0 7 4 0
Stylistic 1 2 2 4 6 1 4 0 2 2 0 2
Structural 0 0 1 0 1 1 0 1 1 0 1 1
Content 1 4 1 0 1 1 0 2 1 3 1 1
Total 14 7 5 8 14 3 10 5 4 12 6 4
Sherry’s pattern was more like Ice’s than Connie’s because her R1 didn’t always have the most changes (Topic 2). The total number of changes in her revisions didn’t show a tendency to decline, either. The total number of changes in her R2
outnumbered that in R1 once (Topic 2). Except that she made several content changes in R1 twice (Topic 1 & 4), she followed Ice’s strategy that she didn’t make structural and content change until R2.
Concerning these multiple revisions, two finding are worthy of attention: the tendency to decrease in the total number of changes and the personal preference when making revisions. It was found that the total number of changes decreased as the number of revisions increased. The tendency to decrease was the most obvious in Connie’s revisions. It indicated that she made the most changes in R1, fewer in R2 and the fewest in R3. However, the tendency to decrease didn’t always exist in the other three participants. For example, Tina only have tendency to decrease in Topic 4, Ice had it in Topic 2 and 4 and Sherry had it in Topic 1, 3 and 4.
Another finding is the personal preference in making revisions and it decided what type of errors they would correct. For example, Connie tended to fix the errors in the four categories in R1 while Ice didn’t fix content and structural errors until R2.
Tina’s preference for making changes of all of four categories was more like Connie’s while Sherry’s preference for making structural and content changes in R2 was more like Ice’s, though their preference was comparatively weak. In other words, there were some exceptions in Sherry’s and Tina’s revisions. Personal preference in making
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
revisions also divided these four participants into two groups. One group (Connie and Tina) established a pattern, paying equal attention to errors in the four categories in R1 while the other group (Ice and Sherry) established another opposite pattern where they left structural and content errors untouched in R1 and later fixed them in R2.
The numbers and percentages of changes by type are displayed from Table 4.2a to Table 4.2d. The four participants’ revisions will be discussed respectively in the four categories: surface change, stylistic changes, structural changes and content changes. The data in Table 4.2a to Table 4.2d were explained in terms of three things:
1) tendency to decrease, 2) personal preference in making revisions, and 3) the total number of changes by type.
Table 4.2a Numbers/Percentages of Changes by Type for Each Revision by Connie
Revision 1 Revision 2 Revision 3 Total
Surface 16 (24.6%) 4 (13.8%) 2 (20%) 22
Stylistic 15 (23.1%) 9 (31) 5 (50%) 29
Structural 16 (24.6%) 4 (13.8%) 1 (10%) 21
Content 18 (27.7%) 12 (41.4%) 2 (20%) 32
Total 65 29 10 104
In Connie’s revisions, the tendency to decrease appeared in the four categories.
That is, surface changes, stylistic changes, structural changes and content changes appeared most in R1, fewer in R2 and the fewest in R3. When doing R1, her attention is equally distributed to errors in the four categories. In R2, the primary focus fell on content and the secondary focus on style. In R3, stylistic changes were paid more attention than the other three types. Moreover, the tendency to decrease in Connie’s revision is comparatively sharper than the other three participants.
Personal preference in making revisions could be also seen from the total
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
numbers of changes in these categories, which were listed according to the order of high-to-low: content changes (32), stylistic changes (29), surface changes (22) and structural changes (21). It seemed that Connie preferred making content changes. Her second favorite was stylistic changes while the least favorites were surface and
numbers of changes in these categories, which were listed according to the order of high-to-low: content changes (32), stylistic changes (29), surface changes (22) and structural changes (21). It seemed that Connie preferred making content changes. Her second favorite was stylistic changes while the least favorites were surface and