• 沒有找到結果。

The design of the Pre-test-Post-test Control Group Design (Campell &amp

N/A
N/A
Protected

Academic year: 2021

Share "The design of the Pre-test-Post-test Control Group Design (Campell &amp"

Copied!
17
0
0

加載中.... (立即查看全文)

全文

(1)

Chapter Three

Method

In this chapter, I will discuss the research design, the participants, the setting of the study, the procedures used to collect data, and the instruments and procedures used to analyze the data.

Research Design

The overall plan of this study was to compare the writing proficiency and

organizational abilities of 80 intermediate-level EFL student writers. The design of the Pre-test-Post-test Control Group Design (Campell & Stanely, 1963) was employed in this study. The experimental group received the instruction in organizational exercises and the control group received the instruction in grammatical exercises.

Participants

The participants of the study were 80 female students from two senior classes in Hsin-chuang Senior High School, a high school located in the suburb of Taipei City. The experimental group consisted of one class of 43 students and the control group consisted of the other class of 37 students.

These 80 students are all Chinese-speaking Taiwanese and none of them has ever lived more than one month or traveled more than one month in English−speaking

countries. Though all 80 students chose to focus on natural science in high school, the

(2)

numbers of hours they had for English classes were different. The control group had to take biology course while the experimental group needn’t; therefore, the experimental group had six hours of general English per week, but the control group had only five hours.

All of 80 students showed strong motivation for learning English because they were required to take the English writing test in the Joint College Entrance Examination (JCEE). During the fall semester of the academic year 2001 (Sep.1, 2001-Jan. 31, 2002), the students took the writing course conducted by the researcher. The class met every other week and each meeting lasted for two hours. None of the students had taken any English writing course. It was the first time for them to be formally introduced to an English writing curriculum.

The Setting

The current study was conducted in Hsin-chuang Senior High School, a high school located in the suburb of Taipei City. The research site was chosen because the researcher has been teaching English in this school for five years and is very familiar with students’

level of English proficiency.

The Design

The purpose of this study was to compare the writing proficiency and organizing abilities of 80 intermediate-level EFL student writers at Hsin-chuang Senior High School.

(3)

The project began in September, 2001 and ended in January, 2002.

This study used the Pre-test-Post-test Control Group Design (Campell & Stanley, 1963). The overall design took the following form:

Pre-test Treatment Post-test

Experimental O X O

Control O O

The design indicated above suggested the experimental group took a pre-test and a post-test (represented by “O”). In between, they underwent the instruction in organizing strategies (signaled by “X”). The control group took the pre-test and the post-test, but they did not go through the treatment -- receiving the instruction of organizing strategies.

Variables

The independent variables and dependent variables of this within−subjects research

design are listed as follows:

Independent Variables

The independent variables included all the pre-writing strategies, namely,

structure-recognizing, brainstorming, clustering, and outlining. The students in the experimental group were taught with pre-writing strategies (structure−recognizing, brainstorming, idea−grouping) and were reinforced with extra organizational exercises

(recognizing and writing topic sentences, supporting sentences, and concluding sentences,

(4)

and writing outlines).

The control group students were taught with only three pre-writing strategies -- structure−recognizing, brainstorming, and idea−grouping. They did not do the

organizational exercises as did the experimental group, nor were they required to write

outlines before they began to write. Instead, they were requested to do grammatical exercises including sentence−combining and sentence−making.

Dependent Variables

The dependent variables included overall writing quality, organization, content, grammar, vocabulary, and mechanics.

Overall writing quality

Overall writing quality refers to students’ overall writing proficiency, which is operationalized by the total scores resulted from adding up the score students received for each of the five categories of writing; that is, organization, content, grammar, vocabulary, and mechanics. The measurement of a students’ overall writing quality is obtained by averaging scores given by two raters in the category of total score.

Organization

Organization refers to students’ abilities in dividing ideas into groups and putting them in a logical order. A well- organized composition should include the smooth connection of ideas and the coherence of sentences in the writing. The measurement of

(5)

organization is obtained by averaging scores given by two raters in the category of organization.

Content richness

Content richness refers to students’ abilities in writing compositions with clear main ideas and substantive supporting details. A good composition with rich content should have a clear thesis and relevant supporting and concluding sentences. The measurement of content richness is obtained by averaging scores given by two raters in the category of content.

Grammatical accuracy

Grammatical accuracy refers to students’ abilities in writing grammatically correct compositions. Sentence constructions and grammatical components, such as tense and agreement will be the focus in grading students’ compositions. The measurement of grammatical fluency is obtained by averaging scores given by two raters in the category of grammar.

Vocabulary appropriateness

Vocabulary appropriateness refers to students’ abilities in writing compositions with precise and appropriate diction. Word choice and word form will be the focus in grading students’ compositions. The measurement of vocabulary appropriateness is obtained by averaging scores given by two raters in the category of vocabulary.

(6)

Mechanics accuracy

Mechanics accuracy refers to students’ abilities in writing compositions with little or no mechanics mistakes, such as the mistakes of spelling, punctuation, indentation and capitalization. The measurement of mechanics accuracy is obtained by averaging scores given by two raters in the category of mechanics.

To sum up, the research design of this study was to compare the writing proficiency and organizational abilities of 80 EFL learners in senior high school after they received the instruction in organizing strategies.

In the following section, the methods and procedures of teaching the organizing strategies will be discussed in detail.

Curriculum

In this section, I will first describe the teaching methods and the classroom procedures, and then I will present the syllabus for the English composition classes.

Teaching Methods and Classroom Procedures

The focus of this writing course is to teach students the organizing strategies. Both the experimental group and the control group went through the first three

steps—“recognition of the structures of paragraphs,” “brainstorming on a given topic,”

and “grouping ideas.” The fourth step for the experimental group was “organizational exercises,” but for the control group was sentence-combining/making exercises.

(7)

Step 1. Recognition of the Structures of Paragraphs

At the beginning of each meeting, all the students were provided with one or several model passages to read. The teacher (researcher) then led the whole class to read and analyze the given passage by introducing the topic sentences (the introduction paragraph), the developing sentences (the body), and the concluding sentences (the conclusion paragraph). This procedure aims at familiarizing students with the basic

structures of an English paragraph. The sources of the passages included those written by students themselves and other well−organized ones provided by the teacher. For example,

the following was one of the samples:

(a) The Internet is an excellent tool for students because it places information within easy reach. (b) The Internet is simple for anyone with minimal computer skills to use, so researching reports is easy. (c) In addition, the Internet enables students to remain in close contact with their professors and each other via e-mail. E-mail is ideal because it is immediate and does not require constant monitoring; students just check an e-mail account to see if they have any messages. (d) Perhaps the best thing about the Internet is that allows all students equal access to information. (e) That is why it is an extremely valuable tool.

Students would be first led to read the passage and then try to figure out what the topic sentence (a), the supporting sentences (b) (c) (d), and the concluding sentence (e) were. The transitional words used in this passage like “in addition,” the best thing is,” and

“that is why” would also be discussed to make students realize the importance of clear ranking in English paragraphs.

(8)

Step 2. Brainstorming on a Given Topic

After the teacher gave the class a topic to write, she first led the class to brainstorm.

The teacher wrote down students' ideas randomly on the blackboard, and then the ideas were grouped in clusters. This process consisted of noting ideas visually as nodes of a network and, in some cases, marking the links between nodes as relations between ideas.

For example, for the topic “Reading Magazines,” the teacher would first ask the students how to approach such a topic and then lead them to brainstorm in three major aspects:

why people read magazines and where and when they do this activity. Ideas like “reading magazines is useful, interesting, and instructive,” “in the library or in the bookstore,” or

“in the free time or on the weekends” would pop up from the students, and then the teacher wrote down all the ideas on the blackboard no matter how weird they might be.

Some linking between the related ideas would be circled in clusters based on the three aspects and those unrelated ideas would be crossed out.

Step 3. Grouping Ideas

The teacher then demonstrated how to group related ideas together to form a paragraph. The teacher instructed how a composition could possibly be organized and developed. The whole class were involved in discussing what was needed to be written or included in each paragraph. For example, for the topic “Reading Magazines,” the teacher would ask the students what the first paragraph should include (the use of reading

(9)

magazines) and what the second paragraph should include (where and when to read magazines). Then the teacher demonstrated how to write the topic sentences for the two paragraphs. The first paragraph could begin with “Reading magazines is very useful to us in our life,” and the second paragraph could begin with “I enjoy reading magazines in the library.” After that, the teacher would ask students to write down the possible supporting sentences from the clusters written on the blackboard. The related ideas should be put in the same paragraph. Some times the finished paragraphs would be written on the

blackboard if there was enough time, but some times they would just be read out loud by the students or by the teacher.

Both the experimental group and the control group followed the first three procedures. For the next two steps, however, the experimental group and the control group had to meet different requirements. For the fourth step, the experimental group received training in organizational exercises, but the control group received instruction in grammatical exercises.

Step 4. Organizational Exercises v.s. Sentence-Combining/Making Exercises

The focus of instruction for the experimental group was on organizational exercises.

After showing students what a coherent paragraph was like and how to form a coherent paragraph, the teacher asked the experimental group to do related organizational

exercises based on the teaching syllabus (see Table 1). The control group, instead, were

(10)

required to do grammatical exercises -- sentence-combining and sentence-making exercises (see Table 2).

Step 5. The Written Task

In the second period of each meeting (two periods for each meeting), a new topic different from the one discussed in the first period was provided for the students. Since the experimental group were now supposed to be very familiar with the format of an ideal paragraph, they were required to write down an outline using the format taught in the previous period before they began to write the newly assigned composition. On the other hand, the control group were simply required to write the composition without having to produce an outline. Both groups were allowed the same amount of time (40 minutes) to finish their compositions, and delayed assignments were not acceptable.

Class Assignments and Syllabus

Before the instruction in organization began, the participants were asked to take a pre-test. The topic was “My Hobby.” After the instruction began, the participants (both groups) were required to write six other compositions. Finally, they were asked to take a post-test. The syllabus of the writing classes are described in Table 1 (the experimental group) and in Table 2 (the control group).

For this study, each participant was asked to write eight compositions, including the pre-test and the post-test. The topics can be seen in Table 1, and Table 2. The first

(11)

topic ”My Hobby,” serving as the pre-test in this study, was finished before the instruction began. After the instruction began, the participants were required to write another six English compositions during the semester, whose topics can be seen in the Tables, and finally at the end of the semester, the participants were asked to write a final composition “What I Plan to Do Next Summer,” which served as the post-test.

Since the purpose of the study was to compare students’ writing performances before and after the instruction, only the compositions collected from the pre-test and the post-test were used as the data for the study.

(12)

Table 1

Syllabus for the English Composition Class (The Experimental Group)

Date Content

Sept. 11

Pre−test (Title: My Hobby)

Introduce what organization is & identify irrelevant sentences (see Appendix B)

Sept. 25

Review & discussion of the pre−test

Identify & write topic sentences (see Appendix B)

Written exercise (Title: Why We Should Learn English Well) Oct. 16

Review & discussion of the previous exercise

Identify supporting and concluding sentences (see Appendix B) Written exercise (Title: An Unforgettable Experience)

Oct. 30

Review & discussion of the previous exercise Organize ideas by time & space (see Appendix B) Written exercise (Title: On My way to School) Nov. 20

Review & discussion of the previous exercise Organize ideas by rank order (see Appendix B)

Written exercise (Title: Health Is More Important Than Wealth) Dec. 4

Review & discussion of the previous exercise

Describe people, things, and places (see Appendix B) Written exercise (Title: My Graduation Trip)

Dec. 18

Review & discussion of the previous exercise Compare and contrast (see Appendix B)

Written exercise (Title: Environmental Protection) Jan. 8

Post−test (Title: What I Plan to Do Next Summer)

(13)

Table 2

Syllabus for the English Composition Class (The Control Group)

Date Content

Sept. 11

Pre−test (Title: My Hobby)

Introduce what organization is & identify topic sentences (see Appendix B)

Sept. 25

Review & discussion of the pre−test

Sentence-combining exercises (see Appendix C)

Written exercise (Title: Why We Should Learn English Well) Oct. 16

Review & discussion of the previous exercise Sentence-combining exercises

Written exercise (Title: An Unforgettable Experience) Oct. 30

Review & discussion of the previous exercise Sentence-combining exercises

Written exercise (Title: On My way to School) Nov. 20

Review & discussion of the previous exercise Sentence-combining exercises

Written exercise (Title: Health Is More Important Than Wealth) Dec. 4

Review & discussion of the previous exercise Sentence-making exercises (see Appendix C) Written exercise (Title: My Graduation Trip) Dec. 18

Review & discussion of the previous exercise Sentence-making exercises

Written exercise (Title: Environmental Protection) Jan. 8

Post−test (Title: What I Plan to Do Next Summer)

(14)

Data Sources

The data sources included students’ written assignments collected from the pre-test and the post-test. For the pre-test, a total of 80 English compositions were collected, but only 78 compositions were collected in the post-test because two participants in the control group were absent on that day.

Measurement Procedures

This section discusses the instrument used to score the participants’ compositions,

and the scoring procedure is also described.

The Instrument

The purpose of this study was to examine whether the participants made

significant progresses in the organization component and their overall writing abilities.

“The JCEE Rating Scale” is used as the instrument in this study. This analytic rubric originally aimed to measure students’ writing products in the English writing test in JCEE, and the following five components are measured: content (5 points), organization (5 points), grammar (4 points), vocabulary (4 points), and mechanics (2 points). The perfect score of each composition is 20 points. This rubric was chosen as the instrument in this study because all the participants were supposed to go through the test of JCEE and the earlier they were used to the grading form, the better.

(15)

Raters and the Training Session

Two raters were involved to evaluate the participants’ compositions. The researcher served as the first rater, and the other rater was a teacher in Hsin-chuang Senior High School and had been teaching English for two years. Both raters were competent writing teachers and were experienced in scoring English compositions.

Before the two raters started to score the papers, a training session was held in order to ensure reliability between the raters. The raters studied the grading instructions and the samples (see Appendix D) which were graded by “The JCEE Rating Scale” carefully and discussed the questions they had. For example, in order to avoid the unreasonable result of having a high total score with a low content score, both raters agreed that among the five individual components of “The JCEE Rating Scale,” the component of content should be put on the top of the priority; that is, the score of the content should be given before the rater proceeded to score the other components and if the score of the content was lower than 2 points, then scores given for the other components should not exceed 2 points.

Scoring Procedures

To start with, both the pre-test and post-test papers were collected. Then two copies of each paper were reproduced, and then, the raters started the rating procedure. They assigned one score to each component, and the sum of these five scores would be the

(16)

total score for each composition.

The average score of each composition was obtained by adding up the two scores given by the two raters and then dividing the sum by two. The mean score of each group was obtained by adding up the average scores of each group and, then, by dividing the sum by the total number of students in each group.

Testing on Reliability

Before the average score of each composition was calculated, reliability between the two trained raters was calculated. The analysis of Pearson correlation was used to examine whether there was reliability between the two raters and the r-value for each component is given in Table 3.

Table 3

Reliability between the Two Raters

Correlation Content Organization Grammar Vocabulary Mechanics Total

r-value .860 .902 .880 .779 .656 .811

The inter−rater reliability on content, organization, vocabulary, grammar, mechanics, and

overall writing quality is 0.860, 0.902, 0.880, 0.779, 0.656, and 0.811 respectively. Since the r-values for the components of content, organization, vocabulary, and overall writing quality were all more than 0.800, reasonably high correlation coefficients, the raters were

(17)

reliable in grading the compositions.

Data Analysis Procedures

The scores of the compositions in the pre-test and the post-test were then analyzed by t-test with SPSS 8.0 for Windows. According to the purpose, hypotheses, and design of this present study, paired-samples t-tests and independent samples t-tests were used to test the two hypotheses in this study. Level of significance in this study wasα= .05.

Summary

In Chapter 3, I have described the recruiting of the participants, the data collection

procedures, the teaching methods, and the classroom procedures. I have also described the instruments used in this study and the data analysis procedures. The results of the analysis are presented in the following chapters.

參考文獻

相關文件

The aim of the competition is to offer students a platform to express creatively through writing poetry in English. It also provides schools with a channel to

incorporating creative and academic writing elements and strategies into the English Language Curriculum to deepen the learning and teaching of writing and enhance students’

Making use of the Learning Progression Framework (LPF) for Reading in the design of post- reading activities to help students develop reading skills and strategies that support their

• School-based curriculum is enriched to allow for value addedness in the reading and writing performance of the students. • Students have a positive attitude and are interested and

Part 2 To provide suggestions on improving the design of the writing tasks based on the learning outcomes articulated in the LPF to enhance writing skills and foster

Task: Writing an article to the school newspaper arguing either for or against the proposal which requires students to undertake 50 hours of community service, in addition to

This flipbook is designed to display a collection of students’ works selected from the winning entries of the SOW Picture Book Design Competition organised in the 2020/21 school

By correcting for the speed of individual test takers, it is possible to reveal systematic differences between the items in a test, which were modeled by item discrimination and