Pilot Study
The pilot study was conducted to see whether the material and procedure for the treatment were appropriate for the participants. Participants for the pilot study were 32 eighth graders of junior high school from the same grade of students as the participants in the main study. The material, procedure of the treatment as well as the results of the pilot study is presented in the following two sub-sections. The pilot study showed that the teaching material is suitable in difficulty and the content is novel for the prospective participants. Therefore, based on the pilot, texts with equivalent readabilities were applied in the main study.
Assessing Reading Material in the Pilot Study
The material adopted in the pilot study is an English narrative story from a collection of English legends called The Wise Man of Gotham posted on the website named The Gold Scales (http://oaks.nvg.org/pega4.html) set up by MPhil, T. K (1995).
The title of the story is A Missing Man Found. The number of words is 406. The readability level is 3.7 Flesch-Kincaid Grade Level. The sentence structure was within the scope of commonly used patterns for junior high school students, based on an examination by the researcher and her colleagues. Other detailed information about the material is provided under the Material and Procedure section.
The main purpose of the pilot study was to see whether the teaching procedure went smoothly, to find out the suitableness of the materials, and to test whether the
time spent on treatment was suitable. It was done in one class period. A class of 32 eighth graders participated in this phase of pilot study on August 14 and 15, 2014 for forty-five minutes in each class period. Before the instructional procedure began, the teacher firstly asked the students to do silent reading for five minutes and then decided on the difficulty level as well as whether they had read the story before.
(1) Did you read the story before? □ Yes. □ No.
(2) Do you think this reading is easy or difficult? □ Easy. □ Difficult.
(3) How much do you understand about this story?
□ I understand a great amount of this story.
□ I understand about half of this story.
□ I can’t quite understand this story.
The results of their feedback follow.
(1) The researcher found no students had read the story before the pilot study.
(2) 17 of these students reported this story to be easy, while 15 found it difficult.
(3) 8 out of the 32 students expressed they could understand a great amount of the story. Another 15 said that they could understand about half of the story. The rest 9 students expressed they had difficulty understanding the story. Although students perceived the texts differently, a great proportion of them generally understood the text. Therefore, the story with similar structure and readabilities in the same collection will be used as one of the teaching materials in the main study. Two other narrative stories with similar structure will serve as test materials as well.
Implementing Treatment
After the survey was done, a two-class period teaching procedure followed. It
was implemented segment by segment, with ten segments in total. In the first class period, six segments were discussed, while in the second period, the rest four segments were discussed, followed by a comprehension check sheet. The detailed teaching process is as followed.
At the beginning of the first class period, the teacher introduced the background of the story, and started to read aloud the story to the students. When encountering points which may cause confusion, the teacher stopped and asked initiating queries such as “What does the author tell us here?” and “How do things look for the twelve men now?” followed by follow-up queries such as “Why do you think so?” and
“What does the author mean here?” These queries will be presented orally (See Appendix H). Six segments were introduced in this class period. Seven minutes was given for pair discussion and whole-class discussion of a single segment. In the second class period, the teacher led the students to review the part which had been instructed in the first class period. After the review, the teacher introduced the rest of the four segments and led the students to discuss the plot. When the ten segments were all introduced, the teacher summarized the whole discussion.
When the teaching and discussion process was done in the second class period, the students were asked to fill in a worksheet with nine multiple choice questions and one open-ended question in 6 minutes (See appendix I). The first three questions were about facts of the story. The next three questions were about interpretation of the story.
The last three questions tapped readers’ opinion, judgment on one of the characters (Sandora et al., 1999), and moral lesson of the story. The open-ended question “What can we learn from the story?” is used to evaluate the degree of interpretation after each treatment session. After students were given ten minutes to answer the questions, they were invited to share their answers to the open-ended question with the whole class. The sharing process took 4 minutes, which provided an opportunity for students
to see how others perceive the story.
The pilot on teaching procedure as well as the prepared segmentation and queries showed that the target story was appropriate for the target participants. Students were able to respond to the questions and participate in discussions with extended Q & A.
Moreover, two class periods were adequate to cover one story, followed by a 10 minute comprehension test.
Main Study
Participants
61 eighth graders in Chong-lun Junior High School in Taichung were recruited in this study. 29 of them were included as an experimental group taking QtA lessons, while the other 32 in control group.
Selection and Development of Teaching Material
According to Beck et al. (1997), Questioning the Author can be applied to teaching of both expository and narrative texts. In the current study, narrative texts are chosen for several reasons.
First of all, junior high school students are more familiar to narrative texts, for their textbooks have provided lots of materials written in such a genre. Second, reading narrative stories bring pleasant reading experience to readers, so that students may be more willing to engage in the discussion. Third, the fable "The Fox and the Crow" was applied by Beck et al. (1997) as a narrative example for planning queries.
Since the researcher is applying Questioning the Author as a model in the current study, the same genre of materials are adopted for the purpose of replication.
Teaching Materials. Five narrative stories from the collection of The Wise Man of Gotham posted by Mphil, T. K. (1995) on the website named The Gold Scales (http://oaks.nvg.org/pega4.html) were the materials for QtA and Control instruction.
The Wise Men of Gotham is a collection of funny and inspiring stories that may arouse students’ interest to read and enjoy. The teacher introduced the residents of Gotham as well as the origin of these stories in the first class period of the treatment, for the collection of the stories share the same background knowledge, and all of the protagonists in the stories were called “the men of Gorham, indirectly referring to the residents of Gotham.
Besides, one story from the chosen five stories was taught in two class periods.
These five stories share similar difficulty level with the students’ textbooks, which have been chosen by the English faculty in our school and regarded as suitable for the potential participants.
Table 1 summarizes the information about these five chosen texts. As shown in the table, the readability levels of these texts range from 3.5 to 4.8, within the range of those in reading passages in current textbook, 3.5 to 5.2 .
Table 1.Information about the Treatment Material
Title of the story Word count Readability level
A Missing Man Found 391 3.5
Bridge Thinking 406 3.7
The Salt Fish and the Eel 467 4.0
The Kettle that Would Not Walk 432 4.2
The Cheese That Ran Away 370 4.8
Segmentation and Queries. Each treatment was segmented into ten parts, which had been decided by me in advance. My decision-making on how to segment the text was through an examination of the text. When I planned the lesson, I analyzed the text and stopped at points that may cause students’ potential comprehension difficulty or those which may arouse students’ different interpretations. Through the discussions in each segment, I expected that my students’ critical thinking ability can be sharpened, and also benefit from their peers.
For each of the five treatment stories, queries were developed for use during the treatment. The queries were to be presented after each reading segment of the story including a set of Initiating and Follow-up and Narrative queries (Beck et al., 1997).
There were ten Initiating Queries in each story (one for each segment), with 1-3 Follow-up Queries after each Initiating Query or Narrative Query in a segment (See Appendix D). The application of Initiating Queries is to encourage students to respond to the text. For example, “What does the author tell us here?” After students’
responses, the teacher raised queries based on students’ responses. These queries are called "Follow-up Queries," which can be used to mark the important part, so as to bring back students’ attention and to elaborate students’ response. Follow-up Queries such as “So, what does the author mean by “getting better positions to cast their lines”?
or “Does the author say that directly?” were well applied in the QtA class. As for Examples for Narratives Queries, examples can be “How do things look for the stranger now?” They help students concern about characters and the plot that the author intends to design. The types of queries focus on main idea, character development or confusing parts of the story, so that they aid understanding of the text and help readers construct their own meaning. Each query took 5 minutes to discuss.
Questions for Comprehension Check Sheet after the Reading Process.
Following the format used and designed by Sandora et al (1999), nine multiple-choice questions plus one open-ended question for each treatment passage were given to the students in order to encourage students to get involved in the lesson and to participate in class discussion (Appendix I). The first three questions were about facts of the story, the next three questions were about interpretation of the story, and the last three questions were readers’ opinion, judgment on one of the characters (Sandora et al., 1999), and the moral of the story. The open-ended question “What do you learn from the story?” was used to encourage students’ to reflect on the passage and to find the potential morals of the story. Comprehension Check sheets along with the reading passages are provided in the appendices section (See Appendix H, I). They were modified after the examination by, and discussed with two instructors teaching the same level of students. Data from the 5-session comprehension check were not used in the data analysis of this study.
Test Materials for Pre / Posttest
Reading Passages. Two narrative stories from ABC Bedtime Stories published by Hebron Soft Limited in 2013 (Nienhuis & Adams, 2013) were chosen as the pretest and posttest materials respectively. These two stories are The Eagle and the Farmer and The Old Woman of the Doctor (See Appendix B). They were chosen for several reasons. First, since the two chosen texts are both narrative stories, the genre is the same as the one applied in the treatment. Second, the readability levels of these two texts, according to Flesch-Kincaid Grade Level Index in computer software, are both 4.1 (Table 2), which is acceptable to the participants, for the readability levels of
Note: Reprint with the permission fromHebron Soft Limited.
the readings in their textbook range from 3.5 to 5.2 with approximately 100-170 words. Third, a survey on a pilot class showed that no one had read or heard about the story before.
Table 2.Data on Passages Used in Pretest and Posttest
Title of the story Word count Readability level
Pretest The Eagle and the Farmer 205 4.1
Posttest The Old Woman and the Doctor 205 4.1
The Recall and Response Sheet (Appendix C). Two Recall and Response sheets served as the pretest and the posttest material. The teacher encouraged students to write down as many details as possible after they read the passage for pretest and posttest. In addition, the teacher asked the students to write down their reflections on the story, trying to relate it to themselves or finding out the moral of the story. The Recall part and the response writing were analyzed separately, with the former one being the data for analyzing factual pausal units and inferences, and the later one being the data for responsive thought units. The coding criteria for factual recall were established according to Bernhardt’s coding system (1991), in which two native speakers of English decided the pausal units for the raters— me and my colleague. As for the other two types of thought units, inferential and responsive, were graded by the two raters as well. With the scoring of two raters, the inter-rater reliability was therefore established.
Motivations for Reading Questionnaire (MRQ) (Appendix E, F & G). The MRQ is applied to examine the potential change about students' reading motivation after treatment. The version applied in this present study was cited from Mori’s MRQ for Japanese L2 learners (Mori, 2002). Mori’s MRQ was designed by adjusting
Wigfield and Guthrie’s Motivations for Reading Questionnaire so that it suits L2 language learners (See Appendix
E). Besides, the original MRQ used a four-point Likert Scale, and Mori adopted a seven-point Likert Scale. This study also adopts Mori’s version of scale use.
However, the MRQ designed by Mori was for college students, while the version I need is for junior high school students, so I edited the version to cater to my study, reducing the item number from 30 to 24 in total. The following list of items were the deleted because of the reasons provided after each of them.
# 6. Even if reading were not a required subject, I would take a reading class anyway. (Deleted because junior high schools do not allow students to take classes at will.)
# 9. I am taking a reading class merely because it is a required subject. (Deleted because junior high schools do not allow students to take classes at will.)
# 10. I would like to get a job that uses what I studied in English reading class.
(Deleted because there is no specific class for English reading in junior high school.)
# 13. I liked reading classes at junior and senior high schools. (Deleted because the participants in this study are currently junior high school students.)
# 17. English reading is my weak subject. (Deleted because there is no specific class for English reading in junior high school.)
# 21. My grades for English reading classes at junior and senior high schools were not very good. (Deleted because the participants in this study are currently junior high school students.)
In addition, to make sure that my students could understand the items on the questionnaire, I translated it into Chinese, and found two other experts to check the translated version for me to ensure the validity of the questionnaire.
Treatment Procedure
Design. The study was conducted in seven weeks. After the pretest in the first week, starting from the second week, one story was covered, which is introduced in two consecutive classes. In the seventh week, posttest was conducted. In the second week, the teacher introduced the story titled A Missing Man Found. In the third week, the story Bridge Thinking was covered. In the fourth week, The Salt Fish and the Eel was covered. In the fifth week, The Kettle That Would Not Walk was instructed. In the sixth week, the last story, The Cheese That Ran Away, was discussed. The instruction time lasted for five consecutive weeks, with two class periods in each week. Each period lasted forty-five minutes.
Procedure for Experimental Group (hereafter Group E). For Group E, a passage was divided into ten segments for discussion which were covered in two class periods, followed by a Comprehension Check after the reading process in 10 minutes.
These activities were completed in two class periods, that is, 90 minutes. The reading of the story and the discussion were presented first, followed by the Comprehension Check for the treatment story. All the stories were read aloud by the teacher. Students listened and read silently along in their own copies. It was decided that the researcher read aloud the story because some students had weak decoding skills and might not read aloud the passages smoothly. In addition, the researcher may also control the reading speed of each story (Liu & Chu, 2008).
During the reading process, the most important parts—queries and
discussions—were implemented. While the teacher was reading aloud the passage, she stopped at some points which may cause difficulty for the students. These points wee pre-determined by the teacher, and the totally ten stopping points formed ten segments for queries and discussion. In each segment, the teacher aroused students’
attention by raising an initiating query. For instance, the teacher asked question such as “What is the author trying to tell us here?” The teacher gave students one minute to think and discuss with other students, and then invited some students to express their ideas. After the students tried to answer the initiating queries, the teacher raised follow-up queries to encourage students’ further elaboration on the discussion topic.
An example of his kind of queries can be “Why do you think so?” Besides the two types of queries mentioned above, narrative queries concerning the character or the plot were included in some of the segments. The teacher asked questions such as
“What do things look for the stranger and the twelve men?” In doing so, the students were encouraged to think more deeply about the author’s intention or the messages embedded in the text. The ten segments were covered within two class periods, with each segment taking approximately 5 minutes to complete.
When the ten segments for discussion were all covered, the teacher handed out the Comprehension Check sheet, on which nine multiple choice questions and one open-ended question were included. The Comprehension Check sheet took 6 minutes to complete. After students finish the Comprehension Check sheet, the teacher helped them check the answers and invited some students to share their answers to the open-ended question with the whole class. This procedure took around 4 minutes.
A Sample Lesson. The reading passage of the day was A Missing Man Found (Appendix H with all treatment stories and queries). Table 3 describes the three categorized aspects of the ten queries used in this lesson.
Table 3.The Aspects on Which the Queries in A Missing Man Found focus Main idea What does the author tell here?
What is the author trying to say here?
What is the author trying to say here?