pre-test post-test tests

(1)

Chapter Four Findings and Discussions

In this chapter, I will first present the findings and results of the pre-test, and then the post-test. I will then discuss the results of the findings in the second part of this chapter.

Findings

This section discusses the results of the pre-test.

Results of the Pre−test

The results of the pre−test are presented in Table 4. The mean score of the

experimental group was 10.36 and that of the control group was 10.56. The standard error mean score of the experimental group was 0.4561 and that of the control group was

0.3903. Both groups were shown to be statistically equal in their overall writing proficiencies because t−value (.328) did not reach the significant level of .05, so there

was no significant difference between the two groups in their overall writing proficiencies.

(2)

Table 4

Comparison of Scores for Overall Writing Quality in the Pre−test

Group N Mean SD Std. Error

Mean

T−value

Experimental Group

43 10.36 2.99 .4561 −.328^n.s.

Control Group

37 10.56 2.37 .3903

Note. n.s. = not significant at 95% level of probability Results of the Post-test

Before testing the two hypotheses, I will first present the results of the analysis for

the experimental group and the control group respectively in Table 5 and Table 6. The comparison of post−test mean scores on the five components is presented in Table 7.

The Experimental Group

Table 5 reports scores for five components and total scores in the pre-test and in the post-test.

(3)

Table 5

Pre−Post Score Change for the Experimental Group N=43

Factors Pre Post Change T−value

Content 2.61 3.03 +0.42 2.849^＊＊

Organization 2.28 3.05 +0.77 4.141^＊＊

Grammar 2.16 2.30 +0.14 1.133^n.s.

Vocabulary 1.92 2.20 +0.28 2.506^＊＊

Mechanics 1.38 1.29 −0.09 1.250^n.s.

Total 10.36 11.88 +1.52 2.820^＊＊

Note^.^＊＊p＜.05

n.s. = not significant at 95% level of probability

Pre= Pre-test; Post=Post-test; Change= the increase or decrease of scores between the pre-test and the post-test. The symbol “+” indicates an increase. The symbol “-” indicates a decrease.

According to Table 5, the increase of mean score change for organization was most remarkable (+0.77), followed by content (+0.42), vocabulary (+0.28), and grammar (+0.14). The increase of scores in the overall writing quality was 1.52. The increase of scores in organization, overall writing ability, content, and vocabulary were significant ( t= 4.141, 2.280, 2.849, 2.506, respectively, p<.05). Though the mean score on grammar of the post-test was higher than that of the post-test, such an increase was non-significant (t=1.133, p< .05). Only in the component of mechanics was the mean score of the

post-test lower than that of the pre-test, but the decrease was non-significant (t= 1.250,

(4)

p< .05).

The Control Group

Table 6 reports scores for five components and the total scores in the pre-test and in

the post-test.

Table 6

Pre−post Score Change for the Control Group N=37

Factors Pre Post Change T−value

Content 2.86 2.84 −0.02 0.135^n.s.

Organization 2.43 2.62 +0.19 1.327^n.s.

Grammar 2.20 2.24 +0.04 0.463^n.s.

Vocabulary 1.86 1.99 +0.14 1.691^n.s.

Mechanics 1.24 1.17 −0.07 0.867^n.s.

Total 10.59 10.87 +0.28 0.616^n.s.

Note.^＊＊p＜.05

According to Table 6, the increase of mean score change for organization was the most remarkable one (+0.19), followed by vocabulary (+0.14), and grammar (+0.04). The increase in the overall writing quality was 0.28. The increases of scores in organization, overall writing ability, grammar, and vocabulary were not statistically significant

(t=1.327, 0.616, 0.463, 1.691, respectively, p< .05). In the components of content (with a

(5)

decrease of 0.02) and mechanics (with a decrease of 0.07), instead of gaining scores, the mean scores of the post-test were even lower than those of the pre-test, but the decrease was non-significant (t= 0.135 for content, and t= 0.867 for mechanics, p< .05).

Both Groups

Table 7 reports the comparison of post-test mean scores between the experimental

group and the control group.

Table 7

Comparison of Mean Scores for the Post-test N=43

Experimental Group

N=35

Control Group

Factors Mean SD Mean SD Mean Difference T−value

Content 3.03 0.83 2.84 0.52 +0.19 1.249^n.s.

Organization 3.05 0.86 2.62 0.53 +0.43 2.704^＊＊

Grammar 2.30 0.68 2.24 0.40 +0.06 0.484^n.s.

Vocabulary 2.20 0.52 1.99 0.26 +0.21 2.308^＊＊

Mechanics 1.29 0.37 1.17 0.32 +0.12 1.505^n.s.

Total 11.88 2.81 10.87 1.65 +1.01 1.978^n.s.

Note.^＊＊p＜ .05

According to Table 7, the most remarkable mean difference between the two groups was the component of organization (+0.43), followed by vocabulary (+0.21), content (+0.19),

(6)

mechanics (+0.12), and grammar (+0.06). The mean difference in the overall writing quality was +1.01. The organizational abilities and the vocabulary richness of the

experimental group were significantly better than the control group (t= 2.704, and 2.308), but there was no significant difference between the two groups in the components of content, grammar, mechanics, and overall writing quality (t= 1.249, 0.484, 1.505, and 1.978, respectively, p< .05).

The first hypothesis tested in the present study is whether the participants write more organized compositions after they received the instruction of organizing strategies in the pre-writing phase. The mean scores on organization of each test were shown in Figure 1.

For the experimental group, the mean score for organization of the pre-test was 2.28 and that of the post-test was 3.05. The mean difference was +0.77. For the control group, the

mean score for organization of the pre-test was 2.43 and that of the post-test was 2.62.

The mean difference was +0.19.The post−test mean scores for the two groups were compared by t−test for two independent samples assuming unequal variances. According to Table 7, in the post−test, the organizational abilities of the experimental group were

significantly (at the 95% level) better than those of the control group ( t= 2.704, p＜.05).

The result of this comparison supported the first hypothesis; that is, EFL senior high school students who received the instruction of organizing strategies in the pre-writing phase performed better than those who received the instruction of grammatical exercises

(7)

in the organization of their compositions.

Figure 1. Mean Scores for Organization

2.28

3.05 2.43 2.62

0 1 2 3 4

pre-test post-test tests

scores

experimental group

control group

The second hypothesis tested in this study is whether the participants perform better in their overall writing proficiency after the instruction. The mean scores on overall writing quality of each test were shown in Figure 2. For the experimental group, the mean score for overall writing quality of the pre-test was 10.36 and that of the post-test was 11.88. The mean difference was +1.52. For the control group, the mean score for

organization of the pre-test was 10.59 and that of the post-test was 10.87. The mean difference was +0.28. The post−test mean scores for the two groups were compared by t−test for two independent samples assuming unequal variances. The result of this comparison, as can be seen in Table 7, indicated that in the post−test, the overall writing

(8)

quality of the experimental group was not significantly (at the 95% level) better than that of the control group ( t= 1.978, p＜.05), so the second hypothesis is rejected.

Figure 2. Mean Scores for Overall Writing Quality

10.36

11.88

10.59 10.87

9.5 10 10.5 11 11.5 12

pre-test post-test tests

scores

experimental group

control group

To sum up, the findings of this study support the first hypothesis, but reject the second hypothesis. In the next section, discussions about these findings will be addressed in detail.

Discussions

The Experimental Group

Compositions written by student S in the pre-test and the post-test were presented next for the discussions of the effects of the writing instruction on organization and overall writing proficiency.

(9)

Table 8

My Hobby by Student S Written in the Pre-test

I have a lot of hobbies. For example, I like to climb mountains, sing songs, and watch TV, etc. But of all my hobbies, I like reading most. Since I was a little girl, I had already got interested in books. I always took a children book to my mother and asked her to read it for me in the afternoon. My mother likes me to read books. She always says,” The children who read books won’t be bad children.” So she bought many books for me. Therefore, reading becomes the best activity I like to do after school.

Based on the JCEE Rating Scale, the student got an average of total score 10, and the average of each of the components was listed as follows: content 2.5, organization 2, vocabulary 2, grammar 2, and mechanics 1.5. The student first mentioned her wide interests and then narrowed down to one of them: reading. Though she stated clearly the reason why she had such a hobby, it was a pity that she didn’t go on writing why it still attracted her now, when and where she usually did this activity and what kinds of books she enjoyed reading most.

(10)

Table 9

What I Plan to Do Next Summer by Student S Written in the Post-test

Next summer will be the best vacation in my life. Because I’ll graduate from the senior high school and become a university student. Before I go to the university, I have several plans that I want to do.

First of them is to lose weight. I prepare to play basketball in the morning and go swimming in the evening every day. And I have to eat less unneeded and more healthy food than now. Second, I would like to take a trip to Japan with my best friends. We can visit the local scenery, know the local culture, and go shopping. We’ll buy many beautiful clothes, skirts, handbags, sunglasses, and so on.

After all those actions, I am sure that I can have a colorful beginning of my university life.

Based on the JCEE Rating Scale, the student got an average of total score 15, and the average of each of the components was listed as follows: content 4, organization 4, vocabulary 2.5, grammar 3, and mechanics 1.5. The student made clear in the first

paragraph that she did have some plans for the next summer. Then she went to go through the plans step by step and finally she made a short conclusion. Her ideas were stated clearly and quite organized, and the content was rich enough. The student made a significant improvement in her overall writing proficiency and organizational ability.

(11)

As reported in page 42, the score of overall writing quality for the experiment group in the pre-test is 10.36 and that in the post-test is 11.88. The mean change is 1.52. The results of the analysis indicated that the experimental group made a significant progress in overall writing quality (t= 2.820, P< .05). Such a significant improvement in writing quality was accounted for by the significant increase in three of the five components, with the gain in organization most remarkable (t= 4.141^＊＊), followed by the gain in content (t=2.849^＊＊) and in vocabulary (t=2.506^＊＊). In the component of grammar, the gain of the pre-post change scores was least remarkable and therefore it did not reach the significant level (t=1.133^n.s.). Only in the feature of mechanics did the experimental group get a minus gain and there is no growth at all.

The remarkable progress in overall writing quality can be attributed to the crucial role of organization. The instruction and practice of organizing strategies such as preparing an outlining before beginning to write successfully enhanced the participants’

performances in writing more organized compositions. When the student learners made significant progress in organizing ideas, they also had better control over what to express and how to present. Besides, other pre-writing strategies like brainstorming and

clustering might also have contributed to the richness of content and vocabulary precision because through these two activities, the participants might know better how to foster the association of ideas and develop the linking between the related ideas, which in turn, will

(12)

enrich the content and vocabulary of their compositions. These two reasons might explain why students, in the post-test, make significant progresses in writing compositions with richer content and more appropriate vocabulary.

The above results supported the overload hypothesis research done by Kellogg (1990). This study also supported Kellogg’s findings (1990) -- preparing a written outline during the pre-writing phase improved the quality of the participants’ compositions. The consistent results of this study suggested that the instruction of the organizing strategies in the pre-writing phase had a positive effect on the students.

The Control Group

Compositions written by the student C in the pre-test and the post-test were presented next for the discussion of the effects of the writing instruction on the control group.

(13)

Table 10

My Hobby Written by Student C in the Pre-test

I have many hobbies, but my favorite one is playing basketball. When I was a junior high school student, I have already fallen in love with it. Playing basketball is not only exciting but is a very healthy entertainment. I can do it to kill time if I am bored, even learn team work with others by playing basketball. At times in order to develop my basketball techniques, I competite with other people uninterruptedly. However, I can’t live without it.

Based on the JCEE Rating Scale, the student got an average of total score 10.5 and the average of each of the components was listed as follows: content 3, organization 2.5, vocabulary 2, grammar 2.5,and mechanics 1. This composition began with a clear topic sentence and went through the details of the reasons why she liked this activity, though some of the sentences were not connected very smoothly. The final sentence stopped abruptly with an inappropriate transitional word “however.”

(14)

Table 11

What I Plan to Do Next Summer by Student C Written in the Post-test

After the most important exam, I want to sleep for a week and then I will fly to America. America is my favorite country. I want to go traveling there very much. There are a lot of things I like to do and many places I want to go. Eating American food, speaking English, playing in the Disney wonder land and shopping there. Not knowing why, but I just like America. And at the end of the traveling, I will go to the California to visit my aunt and my uncle. I will stay there for weeks and enjoy the American life. After two months, I will be back to Taiwan. After that I am going to be a collegian. No matter which college I will enter, I will use the time well. Before that I have to study hard to enter my favorite college. As you sow, so shall you reap.

Based on the JCEE Rating Scale, the student got an average of the total score 13, and the average of each of the components was listed as follows: content 3, organization 3.5, vocabulary 2, grammar 3.5,and mechanics 1. The student first made it clear that she would like to go to America in the next summer and then went on to state the reasons why she would like to go there and what she planned to do during her stay there. The whole composition was written in only one paragraph, which was somehow monotonous and not vividly enough.

According to Table 6, the control group did not make a significant progress in

(15)

overall writing quality nor in all of the five components. Despite the increase of scores in the components of organization, grammar, and vocabulary, the pre-post score change in these components failed to reach the significant 95% level of probability.

There might be three reasons why the control group did not make a significant progress in overall writing quality. First, though the control group were instructed with the same pre-writing strategies as the experimental group were, the control group were not required to practice applying these strategies to their compositions; that is, they were taught to recognize the text structure, but not required to do the outlining exercises. Only oral explanation could not have made a lasting effect on the student learners. Second, the grammatical exercises practiced by the control group were mostly sentence-combining and sentence-making. This type of exercises covered only specific and limited

grammatical patterns and they are only partially related to producing a well-organized composition. Therefore, it is not easy to see a very significant effect in improving students' overall writing proficiency only by emphasizing the drills of certain

grammatical patterns. Third, the curriculum of the control group included only five hours of general English, which was one hour less than the experimental group. So this might be one possible reason why the control group did not make a significant increase in overall writing quality because their exposure to English was less than that of their counterpart group.

(16)

Comparison of the Post-test for Both Groups

A comparison of pre-post mean scores between the two groups demonstrated that the experimental group made significantly greater progress in organization than did the control group because the former were reinforced with extra organizational exercises in addition to the instruction of the organizing strategies. The experimental group also improved significantly in the component of vocabulary, which might be due to the fact that the ability to write more organized compositions after the instruction could

subsequently spur the use of more precise diction.

Although both groups showed progress in the component of content, there was no significant difference between the two groups in their performances in content. The lack of difference can be attributed to the fact that both groups were instructed with the same organizing strategies, which trained students how to brainstorm, and group ideas, so both the experimental and the control groups learned how to enrich the content of their

compositions.

Apart from that, both groups also showed progress in the component of grammar, but there was no significant difference between the two groups. Although the control group did extra sentence-combining and sentence-making exercises, they did not make significant progresses in grammatical accuracy, since the progress of grammar normally takes a long time, it is not surprising that there was no significant difference between

(17)

these two groups in their grammatical accuracy.

Mechanics is the only component in which the control group showed more progress

than the experimental group because both groups gained minus scores and the control group was deducted fewer scores than the experimental group, but still this difference between the two groups was non-significant. The reason for this result might be that the feature of mechanics was neglected in the instruction of both groups, but since the control group paid more attention to the sentence patterns, they might be more aware of these conventions used in the compositions.

In terms of overall writing quality, students in both groups wrote better

compositions in the post-test. Such a progress was significant in the experimental group but not significant in the control group. This seems to suggest that organizing strategies together with grammatical exercises were beneficial to the students, but organizing strategies together with further organizational practices had a greater effect, which was to be justified by the significant improvement in the components of content, organization, grammar, and vocabulary of the experimental group. On the other hand, though the control group did make some gain in the components of organization, grammar, and vocabulary, the improvement did not reach the significant level (at the 95% level).

Summary

Results of assessment of overall writing quality favor the experimental group. After

(18)

four months’ instruction of organizing strategies in the pre-writing phase, students in the experimental group experienced significant growth (p=.05) in all components except mechanics; while students in the control group, though making some improvement in the

components of organization, grammar, and vocabulary, showed no significant increase (p=.05) in all of the five components. The comparison of mean pre-post scores on the five factors indicated that the experimental group made significant more improvement than the control group in content and organization (p=.05).

The above results suggested that the curriculum incorporating grammatical exercises was beneficial to EFL senior-high-school students, though its effect was not significant.

The curriculum incorporating organizational practices, on the other hand, was proved superior in that students who followed it made significant progresses in content and organization. As shown in the JCEE Rating Scale, content and organization together constitute half of the total scores. The weight these two components carry suggest that they play a major role in a piece of writing, and there is no doubt that they should be emphasized for all levels of EFL student writers.