The present study attempted to investigate college students’ comments on others’
writing drafts in online peer revision sessions, the functions of social cues, and their perceptions of online peer revision. In this chapter, the research method was described in detail, including the description of the course, online system, participants, data collection, and data analysis.
Description of the Course
The present study was undertaken in a regular 18-week English writing course at a public university in Northern Taiwan. The class met once a week for two hours. According to the language policy of this university, each student was required to take at least 6 credits of foreign language courses. Four of the six credits were Freshman English (4 credits) and another 2 credits of language course were from either an advanced English course or a language course other than English. Hence, this course, one of the elective advanced courses, was offered for non-English major students who had already completed Freshman English courses. According to the course syllabus, the purpose of the course was to help students become familiar with English composition structures and basic writing skills.
This course adopted a process writing approach, in which the students were required to work through three stages in the writing process: prewriting, drafting, and revising. In the prewriting stage, the students planned and brainstormed some ideas. Then, they composed their first drafts. Finally, they revised their drafts based on the feedback given by their peers and the instructor and completed their final drafts.
During the data collection semester, the students were required to write three assignments. Each writing task involved a writing cycle. Each writing cycle consisted of
pre-writing tasks, first draft, peer revision in face-to-face and online modes, second draft based on the revision, teacher’s comments, a final draft, and a portfolio. Figure 3.1 shows the procedure of a writing cycle. As Figure 3.1 demonstrates, the first stage was the pre-writing task in which the students planned and brainstormed some ideas before composing their first drafts. The instructor taught the students common writing problems which were composed of minor and major writing problems. Minor problems included grammatical problems such as tenses, verb forms, word choice; major problems included sentence structure such as simple and compound sentences, paragraph structure and essay structure, and content. The instructor also taught the students how to compose a paragraph such as brainstorming, listing, outlining, drafting, revising and editing. After the
Pre-writing tasks (in class)
Writing first draft (after class)
Face-to-face peer revision (in class)
Online peer revision via MSN (after class)
Online peer revision in Blackboard (after class)
Writing the second draft (after class)
Teacher comments on the second draft (after class)
Writing the final draft and Portfolio (in the end of the semester)
Figure 3.1 The writing cycles of the course
pre-writing task, students wrote their first drafts. A face-to-face peer revision was
undertaken in the following class in which students paired up to read and give comments to the draft of each others’ composition based on the peer revision checklist given by the instructor (see Appendix D). After class, the students were required to do online peer revision via MSN and Blackboard. Then, the students were required to write their second drafts based on the comments given by their peers from the face-to-face and online peer revision. The second drafts were graded by the teacher. In response to the teacher’s comments, students were required to revise their second drafts and write the final drafts.
The final product was collected in individuals’ portfolios at the end of the semester.
Online System
The online systems used in this present study were MSN1 and Blackboard2. First, MSN, an acronym of Microsoft Network, was one of the popular instant messaging (IM) programs offered by Microsoft® and was especially designed with the functions of text and voice conversation, web-cameras, and transferring files. MSN presented users with a window interface entailing two major frames—conversation window and message typing area, as shown in Figure 3.2. The conversation window displayed the writer and the readers’ ongoing communicative messages types earlier in their individual lower frame.
Their messages in the message typing area could not be seen by the other peers until they were sent. Moreover, MSN could record conversation discourse and the attempts of transferring files, which could be retrieved from the computers; it is instrumental for subsequent data analysis.
1 MSN is a free software utility which can be used to chat with others over the internet. MSN is a collection of Internet services provided by Microsoft. MSN Messenger can be acquired with great ease from the Microsoft Network MSN Webpage. For further information, please refer to its webpage
http://www.msn.com/.
2 Blackboard is a class delivery system designed to enhance teaching detailed information. Please refer to its webpage http://www.blackboard.com.
Nickname of the user
Figure 3.2 MSN system
Figure 3.3 Blackboard Academic Suite Message
Typing area Conversation
window
Blackboard, a widely-adopted class delivery platform offered by the university, was used as a venue to post the students’ comments to their peers, as shown in Figure 3.3.
Blackboard Learning System was accessed from the internet anytime and anywhere and thus students could retrieve all of their course materials, including a course syllabus assignments, lecture notes, slides, internet hyperlinks, and audio/visual aides, and submit their assignments, their writing drafts, and MSN logs. In addition to these functions, Blackboard was also used for students to do online peer revision, where they could read peers’ assignments and offer comments.
Participants
Thirty students enrolled in this writing course in the spring semester of 2008. At the beginning of the semester, 22 students, 11 females and 11 males, volunteered to take part in the study and signed a consent form (see Appendix A). Their ages ranged from 20 to 23 years. They studied English for an average of 9.5 years. They were all non-English majors from different academic backgrounds. Then, twelve of the 22 students, 7 females and 5 males, volunteered to receive a follow-up interview at the end of the semester. Table 3.1 shows the participants’ demographic information. They were given the code of X and Y. X refers to males and Y refers to females.
In the middle of the semester, a background questionnaire was employed to explore students’ background and demographic information with respect to their experiences of English learning, online communication, writing, and their use of technology in the writing course (see in Appendix B). All of them had English writing experiences in high school and cram school. Eight of them, X6, X7, X9, X10, X11, Y8, Y10, Y11, took English writing courses in college. Most of them had no peer revision experiences except one student, X5, who experienced face-to-face peer revision in a previous writing course. Five of the twenty-two participants, X2, X5, X9, Y5, and Y11, experienced drafting, discussing,
Table 3.1
Demographic Information of the Participants
Participants Age Gender Academic status Major
*X1 21 Male sophomore Communications Engineering
*X2 21 Male sophomore Civil Engineering
X3 21 Male sophomore Biological Science and Technology
*X4 20 Male sophomore Information and Finance Management
*X5 20 Male sophomore Management Science
X6 22 Male junior Mechanic Engineering
*X7 22 Male junior Biological Science and Technology
X8 22 Male junior Industrial Engineering and Management X9 22 Male junior Biological Science and Technology X10 23 Male senior Information and Finance Management
X11 23 Male senior Management Science
Y1 21 Female sophomore Mechanic Engineering
*Y2 21 Female sophomore Civil Engineering
*Y3 21 Female sophomore Civil Engineering
Y4 20 Female sophomore Computer Science
*Y5 20 Female sophomore Biological Science and Technology
*Y6 20 Female sophomore Information and Finance Management Y7 22 Female junior Information and Finance Management
*Y8 22 Female junior Management Science
*Y9 22 Female junior Industrial Engineering and Management Y10 23 Female senior Electrical and Control Engineering
*Y11 23 Female senior Industrial Engineering and Management
* refers to those who were interviewed at the end of the semester.
and revising their writing products in their previous writing courses. As for using MSN for online communication, every student had the experience of chatting with others in Chinese or English through synchronous systems such as MSN, Yahoo Messenger, or
asynchronous systems such as Bulletin Board System (BBS). They communicated online mainly for the purposes of chatting, discussing, and asking for help with their friends and classmates.
Data Collection
The data of the present study were collected from multiple sources, including a questionnaire, MSN logs, writing drafts, an interview, and course documents, such as a course syllabus, class slides, and handouts. Table 3.2 shows data resources and collection time. At the beginning of the semester, 22 questionnaires were administered in class. At the end of the third writing cycle, 58 MSN logs and 46 students’ writing drafts were collected. Before the students’ final exam, 12 interviews were conducted. Course documents were collected throughout the data collection semester.
Table 3.2
Data Recourses and Collection Time
Data resources Collection time
A Questionnaire Beginning of the semester
MSN Logs
Students’ Writing Drafts At the end of third writing cycle
An Interview Before final exam
Course documents Throughout the data collection semester
Questionnaire
Twenty-two Background Questionnaires written in Chinese (see Appendix B) were administrated as a survey for the participants’ background information. The Background questionnaire included 7 questions about the students’ demographic information with respect to their, gender, age, major, and years of English learning (Q 1), their experiences of online communication (Q 2 and 3), the participants’ past experiences in writing (Q 4 through Q 6), and experiences in the use of technology in the writing course (Q 7).
MSN Logs and the Students’ Writing Drafts
After each peer revision session, the students were required to save their interaction
of online peer revision and uploaded their MSN logs to Blackboard. Since the students were allowed to use both English and Chinese in the online peer revision sessions, MSN logs written in English and Chinese were collected. Fifty-eight participants’ MSN logs were collected after each writing cycle in order to examine the whole picture of how students offered peer comments and the functions of the social cues in their MSN communication. Moreover, 46 students’ writing drafts were collected to verify the interaction content of their MSN logs.
Interview
A semi-structured interview, with 17 open-ended questions, was conducted in Chinese and audio-taped after the participants completed the three online peer revision sessions at the end of the semester (see Appendix C). Twelve out of 22 participants volunteered to participate in the interview. The interview for each participant lasted from 20 to 30 minutes. The interview served three functions. First, it explored the reasons why the students offered certain peer comments in the online peer revision based on the interview questions (Q 1 to 3). Second, it gained information of students’ perceptions of peer revision in the online synchronous mode based on the interview questions (Q 4 to 10).
Third, it investigated the functions of the social cues during the online peer revision sessions based on a preliminary analysis of the students’ MSN logs and the interview questions (Q 11 to 17). To realize the functions of the social cues, I pointed to social cues and symbolic icons in the participants’ MSN logs and asked the them to describe and elucidate their intentions and reasons of the social cues and symbolic icons.
Course Syllabus, Class Slides, and Handouts
A course syllabus, class slides, and handouts were collected from Blackboard. The course syllabus provided information on objectives, content, and schedule of the course.
Class slides and handouts demonstrated how the course proceeded and how the instructor adopted a variety of methods to teach L2 writing, such as face-to-face peer revision, class discussions, and instructions on writing for the students’ assignments.
Data Analysis
The data analysis consisted of two parts—analysis of MSN logs and interview transcripts.
Analysis of MSN Logs
To address RQ 1 “what comments emerge from synchronous online peer revision in terms of different nature, areas, and types,” the MSN logs were analyzed to examine different kinds of peer comments from the online peer revision. In this study, content analysis was adopted to analyze the MSN logs. Krippendorff (2004) defined content analysis as “a research technique for making replicable and valid inferences from texts to
Table 3.3
Types of Episodes (cited from de Guerrero & Villamil, 1994, p. 486)
ON-TASK EPISODE: An utterance or a group of utterances semantically related in topic or purpose to one discrete troublesource or a series of connected troublesources (as in the case if several errors within one sentence). An on-task episode may be interrupted and continued later in the course of the interaction.
ABOUT-TASK EPISODE: A segment of conversation in which the participants talk about task procedures, for example, interpreting task instructions, rather than about specific
troublesources.
OFF-TASK EPISODE: A unit of discourse in which the participants are not engaged in revising a troublesource and are talking about issues or aspects of their lives unrelated to the content of the composition.
the context of their use” (p. 18). To conduct a content analysis, MSN logs were broken down into analytical episodes, which were defined as exchanges between a writer and his peer reviewer which were related to a specific topic. That is, an episode could be
negotiation of idea arrangement, correction of the use of word choices, or simply chitchat on something off the writing essays (de Guerrero & Villamil, 1994, p. 486). Thus, MSN logs of each online peer session were coded into three different categories: on-task,
about-task, and off-task episodes, as shown in Table 3.3. Excerpt 3.1 shows an example of on-task, about-task, and off-task episodes. As demonstrated in Excerpt 3.1, Y1 pointed out an error in X3’s topic sentence (line 2 through 4) and simplified X3’s supporting sentence (line 6 through 10), which were coded as on-task episodes because the messages were related to peer revision per se. Moreover, the students started their online peer revision at the very beginning by discussion of transferring files (line 1), which was coded as an about-task episode because it was not directly related to peer revision, but about the task procedure (i.e. transferring of files). Finally, they expressed their appreciation for the partners’ help and conversation (line 35 through 40), which were coded as an off-task episode, which was unrelated to peer revision per se.
Excerpt 3.1 (Y1 is the reviewer and X3 is the writer.)
1. X3: “哈囉。傳給我你的檔案。那我們開始囉！” Off-task episode only way to meet with someone who lives far away from us is to travel far away there. It takes a lot of time and energy.)這是你寫的。
.
1. X3: (Hello. Pass me your file. Then, we start.) Off-task episode
2.
3.
4.
Y1: (I go first. Your topic sentence is clear; however, your last word ‘aspect’ should add a ‘s’ because you offer more than one example, right?)
On-task episode
Y1: (Then, your supporting sentence is as follows. ‘In the past, the only way to meet with someone who lives far away from us is to travel far away there. It takes a lot of time and energy.’ The above is written by you.)
35. Y1: (Others are clear and I think you write pretty well.)
About-task episode 36. X3: (You are so good. I admire you so much. Thank
you.)
37. Y1: (That’s what I have to do. Thanks.)
38. X3: (Don’t mention it. I should be the person who has to thank you.)
39. Y1: (Haha, hang in there. See you.) 40. X3: (Hmm.)
Note: The symbol of vertical “…” stands for message omission.
The on-task episodes were further analyzed based on the coding scheme adopted from
Liu and Sadler (2003). The coding scheme included comments in terms of nature (revision-oriented and non-revision-oriented), areas (global and local), and types
(evaluation, clarification, suggestion, and alteration). Liu and Sadler (2003) defined each coding category as follows.
Global areas refer to the feedback concerning idea development, audience and purpose, and organization of writing, whereas local areas refer to feedback with regard to copy-editing, such as wording, grammar, and punctuation. Evaluation denotes comments on either good or bad features of writing; clarification signifies probing for explanations and justifications; suggestion stands for pointing out the directions for changes; alteration refers to providing specific changes (p. 202).
In terms of nature, Except 3.2 shows an example of revision-oriented and non-revision-oriented comments. As shown in Excerpt 4.2, Y5 offered a
non-revision-oriented comment (line 1 through 9), which did not intend Y6 to revise her original sentence. Y5 also offered a number of revision-oriented comments to Y6 (line 10 through 25), which suggested Y6 revising her original sentences.
Excerpt 4.2 (Y5 is the reviewer and Y6 is the writer.) 1.
what he said.’，好像怪怪的。” Revision-oriented
comment 12. Y6: “喔？”
13. Y5: “I (can) still remember what he said.會不會
14. 比較好？”
Y5: (Look at the sentence ‘There is one thing most unforgettable about my grandmother.’)
following the ‘most.’ Is there any mistake?) 5.
6.
Y5: (There should be a definite article in the superlative sentence. The sentence is fine, but it seems to be weird if you connect it.)
7. Y6: (Hence, it will be better if I follow your
Y5: (Look at the sentence ‘Until today, I still remember that what he said.’ It is weird.)
Revision-oriented still remember what he said’?)
15. Y6: (I think both are fine.)
16. Y5: (The main point is the ‘that what.’ You shouldn’t use both of them. )
17. Y6: (Hmm. You are right. I should not use both of them.)
18.
19.
Y5: (Besides, if you ‘until,’ it seems to become ‘I remember all the time, but I cannot remember
20. it today.’ But I’m not sure.) 21.
22.
23.
Y6: (Hmm. Maybe it will be better if I use ‘present perfect tense.’ There will be one problem if I use ‘present tense.’ Present perfect tense will be okay.)
24. Y5: (Hmm.) 25. Y6: (Thank you.)
In terms of areas, Excerpt 3.3 shows an example of global and local comments. As revealed in Excerpt 3.3, X3 offered Y7 comments such as subject-verb agreement (line 1 through 8) and the meanings of phrases (line 9 through 11), which were local comments.
In addition, X3 also offered a comment concerning content and organization (line 20 through 24), which was a global comment.
Excerpt 3.3 (X3 is the reviewer and Y7 is the writer.) 1.
24. Y7: “好的，我想一下。”
4. X3: (You should read it more carefully.) 5. Y7: (Oh, I see it.)
6.
7.
X3: (You should change the verb into ‘gains’ because the verb is third-person and singular. Hence, the verb should be added a ‘s.)
8. Y3: (Hmm!) 9.
10.
X3: (Besides, I do not know the meanings of these two phrases ‘came across’ and ‘have ups and
downs.’) Local comment
11. Y7: (They refer to ‘meeting someone accidentally’ and
‘life full of good and bad things.’) .
X3: (Then, the last question is your content. If you only focus on your life in high school, I am not sure whether it is suitable for the topic.
Moreover, you talk about the procedure of your knowing each other, which occupies half of your content; however, his influence on you only occupies a half of your article. That’s it.)
Global comment
24. Y7: (OK. I’ll think about it.)
In terms of types, Excerpt 3.4 shows an example of the four types of
comments—evaluation, clarification, suggestion, and alteration. As illustrated in Excerpt 3.4, X2 employed a great number of comments to evaluate Y8’s examples (line 1 through 7), two alteration comments regarding sentence structure (line 8 through 12) and the use of the auxiliary (line 15 through 17), and one suggestion comment regarding the use of the conjunction (line 18 through 19). Also, X2 used one clarification comment to explain some
unclear sentences in this Excerpt (line 13 through 14).
Excerpt 4.4 (X2 is the reviewer and Y8 is the writer.) 1.
2.
3.
X2: “topic還算明確。第一個例子舉的不太順的感覺。
First, we can get news rapidly. For example, we always watch TV for news.”
Second, with the progress of Internet, we can watch
TV program online.” Alteration
11.
X2: “The other advantage of watching TV program online is that you don’t ‘have to’ worry about.建議加這個語氣
比較強。” Alteration
X2: (Your topic is clear, but your first example in the sentence ‘First, we can get news rapidly. For example, we always watch TV for news.’ is not smooth.)
Evaluation
4. Y8: (Hm…) 5.
6.
X2: (I think that when you offer the example of ‘we always
X2: (I think that when you offer the example of ‘we always