24
CHAPTER THREE
RESEARCH METHODOLOGY
This chapter presents the methodology of this study. It consists of two sections.
3.1 is a brief description of the subjects and the placement test used to choose the subjects. 3.2 introduces the research design, including the multiple-choice reading comprehension tests and the interview. The instruments and procedures of both methods are presented respectively in this section.
3.1 Subjects
With three reading modes—silent reading, self-reading aloud, and teacher’s reading aloud—under investigation, three groups of the second graders in Taipei Municipal Bai-ling Senior High School take part in the study. The students of the three groups are from three different classes, which are taught by the same English teacher. A placement test is conducted to choose 25 students with equal English reading proficiency level from each class. With equal proficiency level, the students’
performance in the three reading modes is thus comparable. Therefore, the students who get high scores or low scores in the placement test are excluded. Only the students with intermediate scores are involved in the study. The material for the placement test is the reading section of GEPT intermediate level, which is developed by the Language Training and Testing Center in Taiwan. It is one and the only test specially designed to evaluate Taiwanese’ English Proficiency. With the total scores of 120, the means of the three groups are 48, 48.12, and 48.24 respectively. The highest score in each group is 60 while the lowest is 30. No significant difference is found
among the performance of the three groups (*F.95 (2, 72) = .005< 3.15 P= .995
> .05). The means of the three groups and the results of the F test are presented in Table 3.1 and Table 3.2.
Table 3.1 The Three Groups’ Mean Scores in the Placement Test
Group Mean SD
A 48.00 9.53
B 48.12 9.12
C 48.24 9.33
Table 3.2 One-way ANOVA of the Three Groups’ Scores in the Placement Test
SV SS df MS F Sig.
Between Groups .720 2 .360 .005 .995
Within Groups 4975.200 72 69.100
Total 4975.920 74
*F.95 (2, 72) =3.15 *p> .05
3.2 Research Design
Both quantitative and qualitative methods are applied in this study. Quantitatively, multiple-choice reading comprehension tests are used to reveal the students’
performances under the effect of different reading modes. Statistical analysis is followed to compare the students’ performances. Qualitatively, interview is employed to gain insight into the influence of the reading modes from the students’ perspective.
The research design for the multiple-choice reading comprehension tests is presented in section 3.2.1, including the instruments and the procedures of data collecting.
Section 3.2.2 is the design for the interview, including the questions as well as the procedures of the interview.
3.2.1 The Multiple-Choice Reading Comprehension Tests
Various kinds of tests have been developed to measure learners’ reading ability, and multiple-choice reading comprehension test is one of the major test types, which can usually be found in large-scale tests, such as the GEPT and TOEFL. This kind of test is frequently used because it exhibits almost complete grader reliability and it avoids testing the difficult skills of writing as in short-answer questions. Both advantages are necessary for this study, for it aims to compare the students’
performance in the three different reading modes. Moreover, this study involves investigation on two different question types. Since several question types of multiple-choice reading comprehension tests have been acknowledged by reading researchers (e.g. Foote, 1976; Johnson, 1994), the test type are thus applied in this study. The reading passages and comprehension questions of the tests are specified in section 3.2.1.1. The procedures of the tests are presented in section 3.2.1.2.
3.2.1.1 The Reading Passages and Comprehension Questions
A multiple-choice reading comprehension test generally presents a passage followed by some questions, and both perform crucial functions in the test. The passages and the questions applied in this study are specified in the following sections.
3.2.1.1.1 The Reading Passages
This study applies 27 reading passages, 13 of which are narration and the rest 14 are exposition. The two text types are used to see whether text type plays a role in the students’ comprehension under the influence of different reading modes. The passages are arranged from short and easy ones to long and difficult ones to get the students
gradually involved in as well as provide them with suitable challenge to promote motivation. Most of the articles are adapted from exercises for reading comprehension provided by Far East or Lungteng English Reader, and a few are extracted from the reading sections of 1994-2002 English Subject Ability Tests and 1990-1999 JCEE English Tests. All the reading passages are presented in Appendix.
The reading passages have been chosen by the researcher, who has taught English for ten years—the last five years are in senior high school. Three of the researchers’ colleagues, who have taught English for 15 to 20 years, serve as consultants in case the researcher has any uncertainty. The passages are chosen according to the criteria provided by reading researchers or test designers (e.g.
Hughes, 1989; Urquhart & Weir, 1998) and the students’ background knowledge and proficiency level have been taken into account. The topics of the chosen passages (see Table 3.3 and 3.4) are close to the students’ life experience. Moreover, some
modification is made while words that are difficult for the students are found in the texts. Certain words are replaced by an easier synonym to fit the students’ proficiency level. For example, the word “huge” is in place of “immense” and “remove” is
substituted for “eliminate.” Those words that can’t be replaced are underlined with their Chinese translation presented in the margin.
3.2.1.1.2 The Comprehension Questions
In order to examine the effect of the reading modes on the students answering different questions, two question types—main idea questions and detail
questions—are adapted in this study. There are 22 main idea questions in total,—11 in exposition and 11 in narration; 60 detail questions—half in exposition and half in narration. The number of the questions in both the narrative and expository texts has
to be equal in order to compare the students’ performances of the two different text types. Three subtypes of detail questions are applied in this study. They are
specific-answer questions, cause-and-effect questions, and true-false questions. There are 10 questions for each subtype in both the narrative and expository texts.
However, some subjects are absent in the experimental process and their
performance for some passages are lacking. The results from those passages thus can’t be included for the statistical analysis. Therefore, within the 27 passages applied in the experiment, only 21 passages—12 expository and 9 narrative texts—are used for statistical analysis. The original 30 detail questions for each text types are reduced to 24. The previously 10 questions for each subtype of detail questions are reduced to 8.
The main idea questions for the two text types are also reduced to 10 in expository and 9 in narrative. The overall comprehension questions left for expository are 34 and 33 for narrative, i.e. 67 questions in total. Table 3.3 and 3.4 demonstrate the 21
reading passages left for statistical analysis and the distribution of the questions in the two text types.
Table 3.3 Distribution of Questions in Exposition Detail questions Question
types Number/ Topic
Main idea
Questions Cause-effect Specific answer True and false
E1 Chicken town 1 1 1
E2 Seahorses 1 1 1
E3 Model for recycling 1 1
E4 Future cars 1 1
E5 Body heat 1 1 1
E6 Primitive men 1 1 1
E7 E-mail 1 1 1
E8 Voyage of the world 1 1 2 1
E9 Hacking 1 2
E10 Cultural difference 2
E11 Tip 2
E12 Dead mail 1 1 1
Total of subtypes 8 8 8
Total of question types 10 24
Table 3.4 Distribution of Questions in Narration Detail questions Question
types Number/ Topic
Main idea
Questions Cause-effect Specific answer True and false
N1 A secret 1 1
N2 A surprise gift 1 1 1
N3 Picnic & sheep 1 2 1
N4 Impressing dinner 2 3
N5 Pocket money 1 3 1
N6 Time with family 2 1 1
N7 Blind saver 1 1 2
N8 Fire saver 1 1
N9 Climbing stairs 1 1 2
Total of subtypes 8 8 8
Total of question types 9 24
In order to achieve equal distribution of question types required in this study, some of the questions and their options are modified or created by the researcher while others are kept intact. For example, the main idea questions in the passages E3, E5 and N2 and the specific-answer question in E2 are written by the researcher. An originally word-meaning question in E9 is changed into a specific-answer question.
Some inappropriate questions and their options are also modified to make them closer to the criteria provided by experienced test designers (e.g. Fillmore and Kay, 1983;
Heaton, 1990). The following is an example of modification, which is extracted from N6.
Q: The main point of the passage is _________.
(A) success can be won only at the cost of enjoyment of life
(B) no pain, on gains
(C) appreciation of life shouldn’t be neglected in the course of pursuing success (D) success is not worth pursuing
The options of the above question are obvious diverse too greatly in length, which violates Heaton’s (1990) suggestion for option designing: writing options of equal length. The modification version is:
(A) success can be won only at the cost of enjoyment of life (B) one had better not to look back at what he had lost in life (C) only when one loses something does he know the value of it (D) life shouldn’t be neglected only because of pursuing success
Other modification is made when the correct option is obscure or the distractors are not clearly wrong. Besides, when one of the options is too obviously different from the others, all the options will be modified to make them look more similar.
Modification is then conducted to make all the options look similar, such as making the options all begin with the same word. Three of the researcher’s colleagues are invited to examine the correctness and appropriateness of the written or modified questions and options. The inappropriate ones are revised until the colleagues think they are ok.
3.2.1.2 The Procedure of the Reading Comprehension Tests
The students of the three groups are required to read the chosen passages in different modes: Group A read silently, Group B read with self-reading aloud and Group C with teacher’s reading aloud. Each reading passage and its comprehension questions are given to the students separately to prevent the students from reading back to the text silently while answering the questions. One narrative and one
expository text are conducted as practice to get the students familiar with the reading procedures.
One article with its reading comprehension questions is applied in the regular class period every day. The narrative and expository passages are given to the subjects in turn. The whole procedure for the 27 articles lasts for five weeks and two days. The students in Group B are asked to read the passages aloud two times. The teacher (also the researcher) sometimes joins in the reading to help pronounce some difficult words or speed up the reading pace. For the students in Group C, the teacher reads aloud each passage twice. The students, following the teacher’s reading, also read the text silently. The teacher reads the passage by phrases and clauses and sometimes pauses to have the students pronounce the next word to make sure the students have followed the reading successfully. Group A, the silent-reading group is given the same reading time as Group C. Most of the students in Group A report that they also read the passages silently two times or so in the given time. In the process of the experiments, the students of the self-reading-aloud group generally need longer time than those of the other two groups to read the texts twice. The time difference is about ten to fifteen minutes. But the time given to the three groups for doing the comprehension questions is the same.
3.2.2 The Interview
The interview is conducted to obtain students’ opinions about the reading modes.
The interview reports can also be used to check the reliability of the test results as well as to reveal something hidden in the reading comprehension tests. The interview questions are designed by the researcher. Each group is interviewed with some common questions and some specifically designed ones for the group. The interview
questions and the procedures of the interview are presented in the following sections.
3.2.2.1 The Interview Questions
Sixteen questions are prepared for the interview. Two of them are for all the 30 volunteers in the three groups. Four of them are specifically for the ten volunteers in the silent reading group; five for the self-reading aloud group and seven for teacher’s reading aloud group. The self-reading-aloud group and the-teacher’s-reading-aloud group are asked to compare their oral reading modes with silent reading, which is the method they usually employ in reading English.
The interview questions for all the volunteers in the three groups are:
1. Generally speaking, which do you think is easier to read, reading a story (e.g. An Unlucky Day) or reading a text of explanation (e.g. Seahorses)? Why?
2. Which do you think is easier, knowing what the text is generally about or knowing certain details in the text? Why?
The interview questions for the volunteers in the silent reading group are:
1. In the given time, how many times could you read the text?
2. Did you read back and forth the text when you find some points that you don’t understand?
3. Did you read word by word or skip words that aren’t important?
4. When encountering an unfamiliar word, did you try to pronounce it? Do you think reading the word aloud help you get the meaning of the word?
The interview questions for the volunteers in self-reading aloud group are:
1. Do you think your reading the text aloud helps you understand the text better and helps answer the questions as well ? Why (not)?
2. What do you think of the reading pace? Was it too fast or too slow?
3. Compared with silent reading, which do you prefer? Why?
For those who answer “Yes”:
4. If your reading the text aloud helps you comprehend the text better, which does it help more, the stories or the texts with explanation?
5. If your reading the text aloud helps you answer comprehension questions, which does it help more, the main idea questions or the detail questions?
For those who answer “NO”:
4. Which text is more difficult to read when you read the text aloud, the stories or the texts with explanation?
5. Which question is more difficult to answer when you read the text aloud, the main idea questions or the detail questions?
The interview questions for the volunteers in teacher’s reading aloud group are:
1. Do you think the teacher’s reading the text aloud helps you understand the text better and helps answer the questions as well? Why (not)?
2. What do you think of the teacher’s reading pace? Was it too fast or too slow?
3. The teacher reads by phrases and clauses instead of word by word. Do you think this reading mode helps your comprehension?
4. Have you ever failed to catch up with the teacher’s reading pace? If yes, what did you do then?
5. Compared with silent reading, which do you prefer? Why?
For those who answer “Yes”:
6. If the teacher’s reading the text aloud helps you comprehend the text better, which does it help more, the stories or the texts with explanation?
7. If the teacher’s reading the text aloud helps you answer comprehension questions, which does it help more, the main idea questions or the detail questions?
For those who answer “NO”:
6. Which text is more difficult to read when the teacher reads the text aloud, the stories or the texts with explanation?
7. Which question is more difficult to answer when the teacher reads the text aloud, the main idea questions or the detail questions?
3.2.2.2 The Procedure of the Interview
Ten out of the 25 students in each of the three groups volunteer to be interviewed.
These students all consent to be taped during the interview, and they are interviewed after reading all the 27 reading passages. One to three volunteers in the same group are interviewed at one time. The students form group by themselves. The interview is conducted in Chinese in the language lab of the school. Examples of narration,
exposition, main idea questions, and detail questions are provided before the interview.
The time for the interview is about 15 minutes, no more than half an hour. The interview starts from early June, 2003 and ends at the end of June. The content of the interview is recorded and transcribed.
