The two classes were supposedly equal in their academic capacities because they

Chapter Three Methodology

This chapter will present a detailed research design of this study, which is composed of four parts. The first part is a brief description of the subjects chosen for this study. The second and the third parts introduce the method of selecting and designing instruments, and the procedures of instruction and data collection. The fourth part presents the methods of analyzing the data.

3.1 Subjects

The subjects in this study were two first-year make-up classes (Class A and Class B) at I-lan Senior High School in I-lan County. The two classes were composed of sixty-eight repeaters in total: Class A consisted of thirty-three students, and Class B thirty-five. Their average age was 16 and they were all male students. Besides, the repeaters of these two classes had similar English learning background: all of them studied English formally as a foreign language for at least three years. Additionally, based on their raw scores at the Joint High School Entrance Examination, most of the participants had the basic reading ability (See Table 3.1). However, compared with the other first-year students at this school, these students were lower academic achievers because they were among those who were flunked at the end of the first semester and did not pass the make-up exam at grades under 40 points.

The two classes were supposedly equal in their academic capacities because they

were randomly distributed to the two classes. For further confirmation, there was no

significant difference in the overall reading comprehension proficiency in the pretest

between these two classes, as shown by the statistical one-way ANOVA (t=-1.585,

p<.05) (Table 4.1). However, it was found that the experimental group fell behind the control group in global comprehension proficiency (t=-2.421*, p<.05) (Table 4.1).

Besides, the results of motivational questionnaire and perceptual learning style preference questionnaire showed that there were no differences in motivation and learning style preference.

T ABLE 3.1 R AW S CORES AT THE J OINT H IGH S CHOOL E NTRANCE E XAMINATION

Score 50-54 45-49 40-44 35-39 30-34 25-29 20-24 10-14

Class A 2 7 6 7 6 2 3 0

Class B 1 5 5 15 4 4 0 1

Full score=60; Class A (N=33), Class B (N=35)

Thus, Class A was randomly assigned as the experimental group, while Class B the control group. Since this study planned to explore the effects of cooperative learning and teacher-oriented instruction through literature-based reading, each subject was to be assigned seven short stories for reading during about one month of experiment period. In addition, for the application of cooperative learning, according to each student’s English make-up exam grades, the experimental group was further divided into six heterogeneous groups. There were five or six persons in each group, and the English average grade in each group was similar. And this grouping remained unchanged in the process. Meanwhile, the subjects were not informed of the purpose of the reading activities applied to them, in order to avoid the irrelevant interference of the individual’s factors, such as the mental guesswork or expectation.

3.2 Instruments

To answer the research questions, eight instruments were adopted in total. They

were (1) pretest and posttest of reading comprehension proficiency, (2) texts of seven simplified American writers’ short stories, (3) seven tests of reading comprehension achievement of the seven short stories, (4) motivation questionnaire, (5) perceptual learning style preference questionnaire, (6) metacognitive awareness questionnaire, (7) questionnaire on participants’ responses to cooperative learning, and (8) questionnaire on participants’ attitudes toward literature reading. The function and design of each instrument will be explained in detail in this section.

3.2.1 Pretest and posttest of reading comprehension proficiency

A pretest of reading comprehension proficiency designed by the researcher was administered before the instruction. Furthermore, after the reading curriculum, in order to know whether there was any difference in the subjects’ reading comprehension proficiency before, right after, and two months after instruction, the researcher also practiced a posttest designed by the researcher herself. In other words, the same posttest was given to all the subjects in both groups twice, the first time right after the study as the posttest 1, and the second time two months after the study as the posttest 2.

As for the pretest (Appendix A-1) and the posttest (Appendix A-2), eight passages of similar length, difficulty index, genre and subject, were adopted from Six-Way Paragraphs, Introductory Level, edited by Pauk W. (2003) and published by

McGraw-Hill Companies. Each of the two tests contained four passages, and each of

the four passages was followed by ten multiple-choice reading comprehension

questions to evaluate participants’ reading comprehension. That is, there were forty

questions in total for each test. One point was given to one correct answer; thus, each

test was scored with a possible maximum total score of forty. Finally, the scores of the

pretest, the posttest 1 and the posttest 2 were compared to investigate both the

short-term effect and the long-term effect of the cooperative learning procedures and teacher-oriented instruction through literature-based reading on reading comprehension. The criteria for selecting the passages for the pretest and posttest and the layout of each passage were explained below.

First, the two reading proficiency tests were in the same range of the readability level, genre and subject (Table 3.2). The readability level of the eight passages fell from level 3.6 to 3.9 according to the computer software of Flesch-Kincaid Difficulty Index. Additionally, there was an average distribution of genre and subject in both the pretest and the posttest.

T ABLE 3.2 T HE LENGTH AND DIFFICULTY LEVEL OF THE PASSAGES USED FOR PRETES T AND POSTTEST

Pre-test passages

Title Word Number Difficulty Index Subject Getting a Good Night’s Sleep 257 3.6 Instructions

The African Elephant 300 3.9 Information

Looking for Diamonds 304 3.9 Story

Carver’s Early Years 335 3.8 Biography

Post-test passages

Title Word Number Difficulty Index Subject The Seven Cs of a Good Letter 262 3.6 Instructions

Elephants Alive And Dead 287 3.9 Information

The Pied Piper of Hamelin 328 3.9 Story

Carver’s Legacy 282 3.8 Biography

Second, in all the eight passages, a word list was offered below each paragraph (Appendix A-1 and A-2). However, only those new words that were essential for reading comprehension but could not be inferred from the context were given on the basis of the theory of Krashen’s i+1 principle in language acquisition. That is, “we acquire by understanding input containing structures that are a bit beyond our current competence.”

Each passage was followed by ten multiple-choices to measure repeaters’ reading comprehension proficiency (Appendix A-1 and A-2). The design of these comprehension questions was introduced as follows: First, all the new words provided in the word list were not provided here again. Second, there were ten multiple-choice reading comprehension questions for each passage, and global comprehension and local comprehension questions were averagely distributed over every passage. They were five local comprehension questions and five global comprehension questions.

Each question had four choices with one correct answer and three distracters. In the first five questions, local comprehension questions were included to test students’

reading techniques for supporting details. Eventually, each correct answer was given one point. As for the next five questions, five types of global comprehension questions were arranged: pronoun reference, vocabulary in context, deep meaning, subject matter, and conclusion respectively. Likewise, each correct answer was also given one point. In total, the forty multiple-choice questions in each test were scored with a possible maximum total score of forty.

After the researcher completed the selection of articles and the design of

comprehension questions, for further confirmation, four students who were at the

same level with the subjects were requested to read through the passages and take the

test. And they were asked to circle any new word, which would keep them from

getting the main idea but was not on the list. The outcomes presented corresponded to

the researcher’s principle: every question could be figured out by at least two of the four students.

3.2.2 Texts of seven simplified short stories

The reading materials adopted for the literature-based reading activity, for both the experimental group and the control group, were seven American writers’

simplified short stories (Appendix B) from American Writers and Their Works, edited by Marvin M. K. Li (2000) and published by Pyramid Press. It was meant for senior high school students to do outside reading.

The researcher’s criteria for selecting the short stories were illustrated as follows:

First, they were classical masterpieces, which had been widely recognized among the greatest short stories in the 19

or 20

century. They were short stories worth reading and could be considered excellent extracurricular reading materials for teenagers.

Second, the seven short stories covered a wide range of themes (Table 3.3), which could be discussed in class. Third, each short story was a self-contained selection and its readability was carefully controlled. According to the computer software of Flesch-Kincaid Grade level formula, the readability level ranged from level 3.0 to 4.6.

The word length of the short stories ranged from 429 to 729 words (Table 3.3). Fourth, all the seven short stories were written by different authors so that students would appreciate short stories of different styles. Finally, all of the students had not read the seven short stories prior to the experiment. For the above reasons, the researcher selected them as the materials for the reading activity.

Moreover, there were two concerns about the layout and the arrangement of the

reading materials adopted in the instruction process. First, since the Jigsaw activity

was implemented in class in the experimental group, each short story was divided into

six parts on the handout in advance. Accordingly, it was easier for every small group

to allocate the responsibility to its members. Basically, every member had an equal share of the responsibility; namely, 8 to 12 lines. Second, a word list was provided below every part. Among the new words, which never appeared in the vocabulary lists of the subjects’ previous seven textbooks, those words which were vital for complete understanding but could not be inferred from the context in each story were provided.

Third, the order of the seven short stories was arranged form the easiest level (3.0) to the most difficult one (4.6) according to the Flesch-Kincaid Difficulty Index (Table 3.3). Finally, each of the seven short stories was not distributed to the subjects until one week before instruction.

T ABLE 3.3 T HE LENGTH AND DIFFICULTY LEVEL OF THE SEVEN SHORT STORIES

Title Author Word Number Difficulty Index

The Last Leaf O. Henry 699 3.0

The Cask of Amontillado Edgar Allan Poe 527 3.0 The Secret Life of Walter

Mitty

James Thurber 429 3.9

Rip Van Winkle Washington Irving 594 3.9

Bartleby Herman Melville 729 4.2

The Lottery Shirley Jackson 667 4.4

Feathertop Hawthorne 698 4.6

3.2.3 Seven tests of reading comprehension achievement of the seven short stories

In order to evaluate the subjects’ reading comprehension achievement, the

researcher conducted the test of each short story right after the one-hour treatment of the story. Each test was composed of fifteen multiple-choice reading comprehension questions, including ten local comprehension questions and five global comprehension questions (Appendix C). The design was explained as follows: First, the text of the short story was given on the test paper because the aim of the test was to explore the subjects’ comprehension, but not their retention of the text. Second, the original word list was not provided again. Third, the first ten questions were local comprehension questions, in which subjects’ reading techniques for supporting details were tested. The next five questions were global comprehension questions: one on vocabulary in context, one on pronoun reference, one on deep meaning, one on subject matter, and one on conclusion (Appendix C). Fourth, each question had four choices with one correct answer and three distracters. Each correct answer was given one point, so the fifteen multiple-choice questions were scored with a possible maximum total score of fifteen. The scores from these seven tests were compared to examine the effects of cooperative learning and teacher-oriented instruction on reading comprehension achievement.

3.2.4 Motivation questionnaire

Language learners’ motivation has been identified as a crucial factor in affecting language learning (Reiss, 1983; Li, 1993; Xu, 1993; Zhou, 1993; Zhuang & Yang, 1996). In order to understand whether there was a difference in motivation between the experimental and control groups before the study, the researcher developed a questionnaire containing 25 items. The 25 items were developed into a Liker-type questionnaire in Chinese, with five answers to circle in each statement (Appendix D).

Most of the questions were given from the positive point of view, and such questions

would score 1 point, 2 points, 3 points, 4 points, 5 points corresponding to the

answers of “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.”

However, there were some questions asked from the negative point of view and questions like this would score 5, 4, 3, 2, 1 corresponding to the answers of “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” And before the study, a pilot study was administered to four students to make sure that there were no ambiguous words or confusing statements that might affect the content validity.

3.2.5 Perceptual learning style preference questionnaire

Language learners’ perceptual learning style preferences have been identified as a crucial factor in affecting language learning (Chen, 1998). In order to understand the learning style preferences of the students for the purpose of heterogeneous grouping in both groups, the researcher gave questionnaires (Appendix E) adopted from Reid (1984) and translated into Chinese to both groups of students before the study. The original English version was translated into Chinese by the researcher and then was given to four first-grade students to check if there were any confusing words or expressions that might affect their understanding of the questionnaires. The results collected from this questionnaire were used as part of the proof that there was no significant difference in the distribution of different learning styles between two groups.

3.2.6 Metacognitive awareness questionnaire

In this study, a questionnaire was used to examine participants’ metacognitive

awareness in reading English short stories. And it further aimed to see if there was any

difference between cooperative learning and traditional teaching instruction. The

questionnaire used in this study was adapted from Carrell (1989), Schmitt (1990), and

Taillefer and Pugh (1998).

The questionnaire in the study aimed to assess how the instruction reconstructed repeaters’ perception of reading (Appendix F). It consisted of two aspects. The first aspect investigated repeaters’ perception of the aim of reading and the effective reading strategies of a good reader (Question 1 and 2). The second aspect concerned repeaters’ perception of their own efficient reading strategies, reading problems, word attack skills, sentence processing strategies, and types of difficult reading comprehension questions (Question 3 to 7). Totally, it consisted of seven topic-focused questions, followed by six to nine statements and one open-ended statement. The subjects were required to circle the suitable statement or to write down their own opinion to the open-ended statement.

3.2.7 Questionnaire on participants’ responses to cooperative learning Since cooperative learning was a learner-centered activity, it was necessary to conduct a response questionnaire. This response questionnaire was designed by the researcher to investigate the subjects’ attitudes toward the cooperative learning procedure and its activities (Appendix G). It was composed of three parts. The first part was four statements to tap the participants’ perception as to whether the adopted cooperative learning activities were interesting and effective. The second part was sixteen statements to investigate the effects of cooperative learning on all aspects in learning. All the answers in the first and second parts were conducted in a five-point scale ranging from “strongly agree,” “agree,” “neutral,” “disagree,” to “strongly disagree.” The third part was two open-ended questions concerning the participants’

opinions on their willingness to be again instructed in a cooperative learning

classroom in the future along with their suggestions about its application and their

opinions on its advantages and disadvantages.

3.2.8 Questionnaire on participants’ attitudes toward literature-based reading

The questionnaire was designed by the researcher to investigate the subjects’

reactions to the effects of literature reading. It was mainly made up of two parts (Appendix H). The first part, seventeen statements, was mainly concerned with the effects of literature-based reading. All the answers here were conducted in a five-point scale ranging from “strongly agree,” “agree,” “neutral,” “disagree,” to “strongly disagree.” The second part, two open-ended questions, was asked to recount this experience of literature-based reading and to name the favorite and most difficult stories they perceived.

3.3 Procedures of instruction and data collection

The researcher herself carried out the instruction both in the experimental group and in the control group. The subjects were not informed of the aim of the study for the purpose of preventing individual interference or bias. The detailed steps of procedures of this study were described as follows in three stages: before the study, while the study, and after the study.

3.3.1 Before the implementation of the study

Considering the appropriateness of the research of this study, a pilot study was

conducted before the implementation of this study. Four first-year boy students at the

subjects’ level were recruited from those who had been flunked but had passed the

make-up exam at 40 or 41 points to try out the pretest and posttest in this study, for

they had very similar performance in achievement. Accordingly, the researcher made

necessary modifications. Besides, these four students were also asked to check if there

was any problem with wording in the questionnaire.

In order to get rid of the extra variables, the researcher made a stricter selection of the subjects, namely, by taking into account some other variables which possibly influence the subjects’ performance and thus the results of tests. These variables include the influences of the subjects’ preferable learning styles, and motivations toward English learning.

T ABLE 3.4 T HE TIME SCHEDULE FOR SHORT STORY READING

Date Schedule Test Form

Feb. 14 (Fri.) Motivational questionnaire

Learning style preference questionnaire

Feb. 22 (Sat.) Pretest 40 multiple-choices

Feb. 23 (Sun.) Story 1 15 multiple-choices

Mar. 1 (Sat.) Story 2 15 multiple-choices

Mar. 2 (Sun.) Story 3 15 multiple-choices

Mar. 8 (Sat.) Story 4 15 multiple-choices

Mar. 9 (Sun.) Story 5 15 multiple-choices

Mar. 15 (Sat.) Story 6 15 multiple-choices

Mar. 16 (Sun.) Story 7 15 multiple-choices

Posttest 1 40 multiple-choices

Mar. 22(Sat.)

Metacognitive awareness questionnaire, Cooperative learning questionnaire, Literature-based reading questionnaire

May 26 (Mon.) Posttest 2 40 multiple-choices

On the other hand, both groups were informed of the time schedule of the course,

the test time, and the test form (Table 3.4). They were also told that the results of all the achievement and proficiency tests would be counted into their grades in English to encourage them to do their best on the tests. Finally, they were asked to fill out the motivation questionnaire and the perceptual learning style preference questionnaire.

Both the experimental group’s and the control group’s motivational questionnaire and learning style preference questionnaire were recorded, calculated, and analyzed by statistical one-way ANOVA to make sure that there was no significant difference between two groups.

Then, in the next class, the reading comprehension proficiency of two groups was also measured through the pretest designed by the researcher, and again analyzed by the statistical t-test. The two classes of subjects took the pretest individually in one and half hour in their English classes. The above statistical data from the motivational questionnaire, the learning style preference questionnaire, and the pretest indicated that there were no significant differences between these two groups in their English motivation, in learning styles, and even in their overall reading comprehension proficiency. There was only a significant difference in global comprehension proficiency.

After that, the researcher randomly assigned Class A as the experimental group and Class B as the control group. Besides, the experimental group was divided into six “heterogeneous” small groups by the researcher according to their make-up exam grades: five or six repeaters in each group, and a group leader was elected respectively.

Each group consisted of students whose performance levels ranged from low to average to high and the average performance level of all the groups was about equal.

Next, the seats of the same group were organized closely together to facilitate group

discussion. During the process of this study, each subject in both groups was

distributed the same reading materials—each of the seven short stories—one week

before the instruction, and was requested to preview it in one week. Every short story was divided into six parts on the handout and meanwhile, every member in the experimental group was assigned his share in advance and required to master his own part.

3.3.2 While the implementation of the study

Seven 90-minute sessions of literature-based reading spaced over four weeks were administered. The teacher conducted the activity twice a week on the weekend.

In each session, right after cooperative learning and teacher-oriented instruction procedures, a reading comprehension achievement test of each short story was administered to study the effects of cooperative learning and traditional teacher-oriented instruction on reading comprehension achievement. The whole class period lasted one and half hour, which could be divided into three phases.

In the first phase—the first 60 minutes, the reading activity began in the

experimental group and control group. In the process, the control group was mainly

instructed with a traditional teacher-oriented method by explicating the new words,

sentence patterns and grammar in Chinese. In contrast, in the experimental group, one

of the two chosen cooperative learning activities were carried out—Jigsaw—to

monitor students’ self-regulated reading. Every member in every small group was

responsible for different parts of the short story, about eight or twelve lines, which had

been assigned to them one week before class. All the subjects were supposed to study

their share during the week. In class, the Jigsaw activity was divided into two parts. In

the beginning, group members gathered in the expert group and discussed with their

counterparts in other groups, who took responsibility for the same parts of the reading

materials with them, for half an hour. Afterwards, they respectively went back to their

original teams to share what they had discussed with one another and worked on a

brief written report for another half an hour. The researcher remained in the room during each session but did not interrupt the procedure unless the subjects needed help.

But when students asked about the definitions of any words, the researcher only gave some clues to help them instead of offering the answer directly. All the members were encouraged to engage in the team discussion, express personal opinions, cooperate with the teammates, and compete with other teams.

In the second phase—the next fifteen minutes, an achievement test of reading comprehension was administered in class to both the experimental and control groups right after the one-hour session. The test was composed of fifteen multiple-choice items, which involved the content they had read. The first ten questions were local questions, which students could find answers in the short story. The last five questions were global questions, which in turn included questions of pronoun reference, vocabulary in context, deep meaning, main subject, and conclusion.

In the third phase—the last fifteen minutes, students exchanged papers and scored the papers. One will get one point for one correct answer. At this time, the other cooperative learning activity, Student Teams-Achievement Division (STAD), was carried out in the experimental group. The scores every small group earned would be calculated and publicized on the class bulletin board and the first two teams with the most points would be awarded lollipops and cookies as the prize. All the students were also informed that the group with the most points would be given five extra points on their final results.

3.3.3 After the implementation of the study

Immediately after the cooperative learning and teacher-oriented instruction

procedures, the experimental and control groups were given the posttest 1 of reading

comprehension proficiency, which was similar to the pretest in length, difficulty index,

genre, and subject. And the outcomes were also scored and analyzed by the statistical t-test to see whether cooperative learning and traditional teacher-oriented instruction through literature reading really affected the subjects’ reading comprehension proficiency. Furthermore, the subjects were asked to fill in a questionnaire designed by the researcher to exhibit their metacognitive awareness, and their responses to and attitudes towards cooperative learning activities and literature-based reading. Again, two months after the posttest 1, the same posttest, the posttest 2, was given to the subjects to examine the long-term effect of different instruction strategies.

In short, the procedures for data collection are as follows: (1) motivational questionnaire and perceptual learning style preference questionnaire, (2) pretest, (3) reading instruction—cooperative learning and teacher-oriented procedures, (4) short story reading comprehension tests, (5) posttest 1, (6) metacognitive awareness questionnaire, (7) participants’ response questionnaire on cooperative learning and literature-based reading, and (8) posttest 2.

3.4 Data analysis

Quantitatively, a statistical program, the computer software of SPSS for Window t-test was utilized to analyze the collected data to answer the six research questions.

For the measurement of reading comprehension achievement and proficiency, the scores collected from the seven reading comprehension achievement tests and three reading comprehension proficiency tests were computed using SPSS to compare the inter- and intra- group differences. The inter-group comparisons were analyzed through the independent-samples t-test and the intra-group comparisons through the paired-samples t-test. The results of the t-tests were used for the analysis of reading comprehension achievement and proficiency.

As for the analysis of the motivational questionnaire, perceptual learning style

preference questionnaire, metacognitive awareness questionnaire, and literature-based reading response questionnaire, each participant’s responses to the statements were scored and compared for inter-group analysis through the independent-samples t-test.

Besides, data collected from the participants’ cooperative learning questionnaire

were computed to explore the participants’ responses to the cooperative learning

procedures. Finally, the participants’ reactions to open-ended questions were coded

and categorized by the researcher.