This chapter displays the results of the study. It is consisted of three sections, providing the answer to the three research questions. The first research question, answered in the first section, relates to the potential effect of QtA on reading comprehension in terms of three comprehension levels: factual, interpretive and responsive. The first and the second levels are presented based on students’ written recall. The third level is based on written response writing. The second section demonstrates the result for the second research question concerning the effect of QtA on reading motivation. The last section discusses the result of the students’
perceptions for QtA. In this section, students’ opinions about this approach will be presented.
Effects of QtA Lessons on Comprehension
This section presents the results of the potential effect of QtA on text retention and inferences in students’ written recall. As for the response to text, two raters graded the students’ response sheets by further categorizing the units into four types: (1) Meta-comment on content, (2) Meta-comment on language, (3) Reader-based response, and (4) Text-based response.
Effects of QtA Lessons on Text Retention
Text Retention in Recall. Table 7 displays the mean percentages of recalled
units by the two groups in the pretest and posttest.
Table 7.Mean Percentage in the Pretest and Adjusted Mean Percentage of Recall Units in Posttest
Pretest Mean Posttest Mean Group E (n = 29) 15.78% (S.D. = 12.86) 20.07a %(S.E. = 2.51) Group C (n = 32) 18.93% (S.D. = 18.66) 15.63a %(S.E. = 2.39) Note: Standard deviation and Standard Error in parentheses
As depicted in Table 7, in the pretest, the mean percentage of recalled units of Group E is 15.78, S.D. = 12.86, which was three points lower than that of Group C (Mean Percentage = 18.93, S.D. = 18.66). However, in the posttest, the adjusted mean percentage of recalled units, with pretest covariated, of Group E was 20.07, S.E. = 2.51, while that of Group C, with pretest covariated, dropped to an adjusted 15.63, S.E.
= 2.39. In addition, with pretest mean percentages covariated, an ANCOVA analysis was performed on the posttest mean percentages to examine whether there is a difference between groups in the posttest. The result of ANCOVA shows that there was not a significant difference between QtA group and the control group on posttest, F (1, 58) = 31.514, p = .207, η2 = .027 (See Table 8). QtA training does not make a difference on students’ recall of text content.
Table 8.ANCOVA on Mean Percentage of Recalled Units in the Posttest
Source df Mean Square F Sig.
Partial Eta Square
PreRx 1 5748.216 31.514 .000 .352
Group 1 297.606 1.632 .207 .027
Error 58 182.401
Total 61
Inference in Recall. When students were recalling the text content, they might not only retrieve ideas explicitly presented in the text, but also inferred ideas that are implicit based on what they had just read. For example, in the test story “The Old Woman and The Doctor,” there is a conclusive sentence, “The judge decided that the woman was right.” Some students did not directly present this idea; instead, they wrote “The judge believed what the woman said (# C12, C32).”Such proposition was counted as an inference unit. Table 10 displays the mean number of inference units of the two groups in the pretest and posttest recall. Unlike recalled units, which were transposed into percentages, inference units are reported in terms of frequency because there was no baseline for comparison between topics and between groups.
Therefore, Non-parametric test, Kolmogorov- Smirnov Z test, a test comparing the ranking of scores, was employed.
_____________________________________________________________________
Note: # E represents the student number from Group E, while # C represents the student number from
Group C.
Table 9.Mean Number of Inference Units in Pretest and Posttest Recall
Pretest Mean Posttest Mean
Group E (n = 29) 1.53 (S.D. = 1.82) 2.05 (S.D. = 2.49) Group C (n = 32) 2.45 (S.D. = 2.65) 1.58 (S.D. = 2.39) Note: Standard Deviation in parentheses
On average, students generated very sparse inferences, ranging 1 to 3 units of inference. As displayed in Table 10, in the pretest, Group E produced a mean of 1.53 units, S.D. = 1.82, while that of Group C is 2.45 units, S.D. = 2.65. Nevertheless, the mean inference units produced by Group E increased to 2.05 units, S.D. = 2.49 in the posttest, while the performance of Group C regressed, obtaining only a mean of 1.58 units, S.D. = 2.39.
Table 10.Kolmogorov-Smirnov Z Test on Scores of Interpretive Thought Units
PreI PostI
Kolmogorov-Smirnov Z test 1.377 .710
Asymptotic sig. （two-tailed） .045 .694
A Kolmogorov-Smirnov Z test was used to compare the inference units produced by the two groups in the pretest and the posttest. The results show that in the pretest, the mean inference units of the two groups are significantly different (Z = 1.377, p
= .045), while in the posttest, the two mean units of inference do not show significant difference (Z = .710, p > .05) (See Table 10). This indicates that although Group C outperformed Group E in the pretest, Group E did progress more than Group C in the posttest to a degree that narrowed the difference between the two groups, leading to insignificant difference.
Coding of Recalled Units and Inferences. A sample of recalled units and inferences on students’ Recall and Response Sheets is presented as follows (original Chinese version) to illustrate how the raters decided which are recalled units and which are inferences.
一個 美好的 早晨，農夫檢察動物 陷阱。在一個陷阱裡，他看到 美麗 的 老鷹。他想要殺了那隻老鷹，並且吃了牠。但那時，他想到 老鷹能抓 兔
子。他打開了陷阱，就讓那隻老鷹走了[Inference]。那隻老鷹 很感謝，所以
幫牠抓兔子 和老鼠。那個夏天後來，農夫 蓋了一座石牆，在一瞬間，石牆
要 倒 了 ， 但 農 夫 沒 注意 到 。 老 鷹看 到 農 夫 發 生 危 險，老 鷹救了農夫
[Inference] 。 當下農 夫 不 知 道 發 生什 麼 事[Inference] ， 下 去 時很 生氣 地
[Inference]拿起掉落的帽子，最後他才知道那座牆倒了[Inference]，才說老鷹
不斷地[Inference] 幫我。
(# E1) As presented in the sample, units that are underlined match the original text, so they were counted as recalled units. The italicized units, on the other hand, do not match the original text, and were therefore coded to as inferences.
Response Writing. As for the responsive thought units displayed in response writing, 122 data were included, with a lot of 58 response writing data from QtA group (29 in both pretest and posttest), and 64 from control group (32 in both pretest and posttest). I then coded the 122 participants’ response writing in the pretest and the posttest using open coding. Four categories emerged in the data: (1) Meta-comment on content, (2) Meta-comment on language, (3) Reader-based response, and (4) Text-based response. Among these four categories, the former two are not as directly related to the content; the latter two are.
Meta-comment on content includes comments on the content, such as “This story sounds like a fable I heard before (# E1),” “It’s such an interesting story (#
C17).” Meta-comment on language focuses on the use of language in the stories, such as “There are so many new words here (# E22),” “I learned new vocabulary through the reading process (# E10, C22).” Reader-based responses indicate the general ideas or morals that the students got after reading the stories. For example, some students stated, “It’s bad to be greedy (# C12),” or “Help others, and they may pay you back when you’re in need (# C14, C25).” As for text-based responses, students focused mostly on a certain plot or character and interpreted the events or behaviors so that they may comment on, such as “The doctor is so evil that he cheated the old woman’s money (# E22),” "The eagle is one who knows to pay back. It’s worth our respect (#
C28).”
Coding of Response Writing. A few samples of coded response writing on students’ Recall and Response Sheet are presented as follows (original Chinese version) to explain how the raters decided which types of response the units belong to.
“我覺得老鷹是一個知恩圖報的動物 [Text-based response]。農夫放老鷹 走，老鷹就救了農夫，改變了老鷹是壞心的刻版印象 [Text-based response]。”
(#E21) This excerpt is an example for text-based response. The response focuses on interpreting the eagle’s behavior and making judgment, which meets the definition of text-based response.
“做 人 要 腳 踏 實 地 [Reader-based response] ， 不 能 做 虧 心 事 [Reader-based response]。”
(#C13) These two propositions are coded as reader-based response, since they reveal the morals the reader got after reading the story.
“這個故事很像老鼠報恩的故事 [Meta-comment on content]。 救了之 前的恩人，這就是報恩 [Reader-based response]。”
(#E1)
“這個故事就好像白鶴報恩 [Meta-comment on content]，知恩圖報，動 物都知道，都有去回報幫助他的人，人類可以跟他們好好學習 [Reader-based response]。”
(#C25) The above two comments contain focuses on the content and are therefore coded as meta-comment on content.
“這篇文章好難啊 [Meta-comment on language]。”
(#C20) “因 為 看 不 懂 的 單 字 很 多 ， 所 以 我 也 沒 什 麼 興 趣 想 讀 下 去 [Meta-comment on language]。”
(#C19)
These two propositions reveal how the readers feel about the difficulty level of the stories and were thus coded as meta-comment on language.
Table 11.Frequency and Percentage of Units of Four Types of Response in Response Writing
As displayed in Table 11, several observations are found for response writing for Group E and Group C:
First, for Group E, the percentage of reader-based response rose from 37.68% (13 units out of the average 34.5 units in total) to 63.80% (18.5 units out of the total 29 units). The percentage increased by 26.12%. Similarly, the percentage obtained from Group C also rose from 54.81% (28.5 units out of the total 52 units) to 61.33% (23 units out of the total 37.5 units). The percentage increased by 6.52%. Still, the increase of 21.12% is much greater than Group C, 6.52%.
Second, for Group E, the percentage of the response concerning meta-comment on language in the stories dropped from 26.09% (9 out of the average 34.5 units in total) to 8.62% (2.5 out of the average 29 units in total). The percentage decreased by 17.47%. For Group C, on the contrary, the percentage rose from 13.46% (7 units out of the average 52 units) to 16% (6 units out of the average 37.5 units), which increased 2.54%.
Third, for Group E, the percentage of meta-comment on content increased from 0% (0 responsive units out of the average 34.5 in total) to 5.17% (1.5 units out of the average 29 in total). There is a five-percent increase in the percentage. For Group C, the percentage increased from 7.69% (4 units out of the average 52 in total) to 12%
(4.5 out of the 37.5 units). The increase is similar to that of Group E, approximately 5%.
Finally, for Group E, the percentage of the response concerning text-based response dropped from 36.23% (12.5 units out of the average 34.5 units in total) to 22.41% (6.5 units out of the total 29 units). The percentage decreased is 13.82%.
Likewise, the percentage obtained from Group C regressed from 24.04% (12.5 units out of the 52 units in total) to 10.67% (4 units out of the 37.5 units in total). The percentage dropped 13.37%, which is approximate to that of Group E.
The comparisons of the four types of response between pretest and posttest for the two groups are displayed in the following bar charts (See Figure 1 and Figure 2).
Figure 1.Percentage of Each Response Type for Group E
Figure 2.Percentage of Each Response Type for Group C
Meta-comment on content
Meta-comment on language
Reader-based response
Text-based response
Meta-comment on content
Meta-comment on language
Reader-based response
Text-based response
As depicted in Figure 1 and 2, both QtA group and control group show a similar gain on meta-comment on content in the posttest. Similarly, there is a similar drop on text-based response in both groups. However, while QtA group displays a sharp drop in the percentage of meta-comment on language, a slight raise is found in control group. In addition, a drastic increase is shown in QtA group on reader-based response, while the increase found in control group is comparatively insignificant.
Effects of QtA Lessons on Reading Motivation
In the present section, the result of the motivation for reading questionnaire (MRQ) is presented. The questionnaire, adopted from Setsuko’s MRQ, was adjusted to cater to this study, with 24 items in total. These items are scored on a seven-point Likert Scale. Table 13 displays the mean scores of 24 items in the MRQ for the two groups in the pretest and posttest. In the pretest, the mean score of Group E is 97.14 (S.D. = 36.203) and that of Group C is 102.78 (S.D. = 32.841). In the posttest, the adjusted mean score of Group E ascended to 99.69 (S.E. = 5.83) and that of Group C dropped to 100.62 (S.E. = 60.25). The Cronbach Alpha of the questionnaire in this study is .963, indicating a high degree of internal consistency among the 24 items.
Table 12.Mean Pre-Posttest Scores for Group on MRQ
Group Pretest Posttest
Group E (n = 29) 97.14 (S.D.= 36.203) 99.69a (S.E. = 5.83) Group C (n = 32) 102.78 (S.D.= 32.841) 100.62a (S.E. = 60.25) Note: Standard deviations and Standard errors are in parentheses.
An ANCOVA analysis, with pretest mean scores covariated, was performed on
posttest mean scores to explore the difference between groups. The result shows that pretest significantly contributes to the model, F (1, 58) = 112.746, p < .001. However, there is no significant difference between groups in their motivation at posttest, F (1, 58) = .954, p = .333, η2 = .016, indicating that QtA may not substantially arouse students’ reading motivation (See Table 13).
Table 13.ANCOVA Results on MRQ Posttest
Source df F Sig.
Partial Eta Square
PreTotal 1 225.419 .000 .795
Group 1 .954 .333 .016
Error 58
a. R Square = .795（Adjusted R Square = .788）
Results of the Students’ Perception of QtA
This section presents the analysis of the response data to the 6 open-ended questions in the QtA perception questionnaire (Appendix K, L).
Perceptions of QtA Lessons
As a supplement to the primary quantitative findings, Group E filled out a perception questionnaire to report their perceptions of QtA (See Appendix K, L). The questions in the questionnaire aim to probe three major aspects of this study: (1) perception of QtA and techniques in general, (2) the perception of material, and (3) perception of outcome.
The first aspect concerning the perception of QtA and techniques was examined by three questions in the questionnaire: 1) Do you like QtA approach? Do you have any suggestions? 2) What is the part in the teaching process that you like the most?
The least? and 3) Do you think queries raised by the teacher are too difficult?
The result of the first question is that 68.97% (N=20) of the students said they like the teacher’s way of instruction (# E2, E9, E10, etc.), indicating their positive attitude toward QtA approach. Only 31.03% (N=9) said they dislike QtA approach. In addition, 2 students suggested that fun activities can be included in the QtA lessons (#
E1, E9).
The answer to Question 2) What is the part in the teaching process that you like the most? The least? is depicted as follow (Table 14).
Table 14.Students’Favorite and Disfavored Processes of QtA (Q.2)
Students’ favorite part of QtA process Students’ disfavored part of QtA process
Comments Freq. Percent Comments Freq. Percent
a. The stories are process they like and dislike the most. For the parts they like the most, 44.83% (N=13) thinks that the stories are appealing. 31.03% (N=9) enjoyed the queries and discussion procedure. 10.34% (N=3) mentioned something important: the relaxing atmosphere can boost reading ability (# E17, E22, E29). As for the parts they dislike the most contain three points: the plots of the stories are too similar (10.34%, N=3), the queries are difficult to understand (10.34%, N=3), and the language in the stories is hard (6.90%, N=2).
The result to Question 3) Do you think queries raised by the teacher are too difficult? shows that up to 90% (N=26) of the students confirmed that the queries catered to the students’ proficiency, indicating that the queries are easy for them to
understand and suitable for discussion.
The above-mentioned three questions examine the students’ general response to QtA and its techniques, revealing that most of the students have positive perception toward QtA approach. They may have enjoyed the stories, the queries and discussion process, and the relaxing atmosphere brought about by QtA.
Table 15.Students’ Preference for QtA Materials and Techniques (Q.4)
Like: 72.41% (N=21) Dislike: 27.59% (N=8)
Reasons Freq. Percent Reasons Freq. Percent
a. The stories are
Table 15 summarizes the result of Question 4) Do you like these stories? Why or why not? This question concerns the second aspect, teaching material. After providing
Yes / No to the first question, they followed with reasons in statement. For this question, a majority, 72.41% (N=21) of the students stated that they have positive feelings about the treatment stories. As displayed in Table 15, students provided various reasons. Affectively, they think the stories are interesting or funny and thus they enjoy listening. Specifically, 48.28% (N=14) of the students stated that “I think the stories are interesting” (# E1, E26, E27, etc.). 10.34% (N=3) said “I enjoy listening to stories” (# E9, E22, 24). Another 10.34% (N=3) stated that “The stories are funny” (# E1, E9, E27).
Aside from affective feedbacks, some students offered cognitive feedback which explained why they like the material. They think the stories introduce culture and help them with their thinking. In addition, some students think the introduction of the vocabulary is very helpful. Specifically, 10.34% (N=3) of them said “The stories introduce foreign culture” (# E13, E3, E20). Another 3.45% (N=1) said that “I learned a lot of vocabulary from the stories” (# E20). A reason especially worth mentioning is “The stories help me contemplate” (3.45%, N=1) (# E6). This indicates that the student enjoyed the thinking elicited by the material.
Yet, still 27.59% of the students (N=8) said they did not enjoy the stories. Among these students, 17.24% (N=5) of them said they cannot fully understand the content (#
E1, E9, E27, etc.). Another 10.34% (N=3) thought the stories are too stupid, so they became less and less interested as the treatment procedure went on (# E19, E23, E29).
Table 16.Students’ Perception of QtA Impacts on English Reading Motivation (Q.5) Motivation Increased: 75.86% (N=22) Motivation Not Increased: 24.14% (N=7)
Reasons Freq. Percent Reasons Freq. Percent
a. The stories
continue reading” (# E11, E17, E19, etc.). 20.69% (N=6) expressed their favor for the teacher’s instruction. 13.79% (N=4) stated, “I learned new words through QtA training” (#E3, E10, E15, E21). Another 10.34% (N=3) of the students mentioned, “I never knew that English reading can be so much fun” (# E6, E11, E8). 3.45% (N=1) mentioned, “Reading with instructions is more motivating than reading alone”
(#E29). Another 3.45% (N=1) said “I understood all of the stories” (#E26). Only 10.34% (N=3) students said they would never be motivated to learn anything about English (See Table 16).
Table 17.Students’ Perception of QtA Impacts on English Reading Comprehension (Q.6)
Ability Increased:82.76% (N=24) Ability Not Increased: 17.24% (N=5)
Reasons Freq. Percent Reasons Freq. Percent
a. The stories you in reading comprehension?” 82.76% (N=24) of them expressed their feelings of their reading ability being improved a little. 41.38% (N=12) said that the two stories
introduce new vocabulary, so they improve their reading comprehension. However, this reason does not seem to directly relate to the effect of QtA. These students appear to attribute their progress in English to exposure to more vocabulary. In addition to learning new words, another 17.24% (N=5) think the teacher’s instruction is fun, which in turn increases their reading ability.10.34% (N=3) mentioned that “I learned to know how the author thinks” (# E6, E11, E27). 3.45% (N=1) said she had learned some reading strategies, such as “making inferences and predictions” (# E32).
Another 3.45% (N=1) also said, “I had benefited from the reading comprehension sheet” (# E20). As depicted in Table 17, although Reason a. refers to the material, reason b. (QtA instructional process), c. and d. (QtA’s effect that may transfer to reading skills for future reading) all directly relate to QtA, indicating that about 34%
of the QtA students think QtA improves their reading comprehension. Specifically, their perception of QtA may help explain why Group E had made progress in inference generation and response writing. On the other hand, there are still 17.24%
(N=5) of the students who stated that the vocabulary hindered them from cultivating interest in English reading (See Table 17).
To sum up, students may perceive QtA as an effective approach as reflected in their perceptions of higher reading motivation as well as increased reading comprehension in the posttest. Nevertheless, although this qualitative datum does indicate that students think themselves more motivated, the result of students’
motivation of reading questionnaire (MRQ) does not show a significant growth. Thus, there is a disparity in their perception and quantitative measure of reading motivation.
Oral Feedbacks from Control Group
As for the feedbacks from group C, the students’ opinions did not seem as
As for the feedbacks from group C, the students’ opinions did not seem as