This chapter, made up of three sections, presents the result of the study. The first section presents the answer to the first research question: Does QtA instruction make a difference in senior high school students’ reading comprehension as measured by three levels of short-answer questions, factual, interpretive and responsive? The second part comprises the answer to the second research question: Does QtA instruction make a difference in the quantity and quality of senior high school students’ post-reading response writing as indicated by number of words, number of thought units, and the four types of response (textual, personal, intellectual and incorrect)? The last part offers the answer to the third research question: What are students’ perceptions toward QtA instruction and self-assessment of ability growth in English?
Effects of QtA Lessons on Comprehension
The effects of QtA lessons on three types of comprehension questions (factual, interpretive, and responsive questions) are presented in this session. There are three questions for each type of questions. The total is nine questions.
Effect of QtA Lessons on Comprehension Questions
Table 9 displays the mean scores of 9 comprehension questions for two groups in total in the pretest and posttest. In the pretest, the mean score of experimental group (hereafter Group E) was 14.28 (SD = 5.91) and that of control group (hereafter Group C) was 13.97 (SD = 4.87). In the posttest, with pretest scores covariated, Group E obtained an adjusted mean of 16.11a (SE = .65) whereas Group C had 14.60a (SE = .64).
Table 9
Mean Scores for Group and Pre-Posttest on Comprehension Questions
Group Pretest Posttest Posttesta
Group E (n = 46) 14.28 (SD = 5.91) 16.20 (SD = 4.76) 16.11a (SE = .65) Group C (n = 48) 13.97 (SD = 4.87) 14.52 (SD = 5.67) 14.60 a (SE = .64) Note. a posttest mean score that is adjusted by pretest mean score
An ANCOVA analysis on the posttest comprehension score with pretest comprehension score as a covariate and group as an independent variable indicated that there was no significant difference between groups, F (1, 91) =2.71 (See Table 10). Nevertheless, pretest did show significant contribution to the model, F (1, 91) = 38.04, p < .001. With pretest variations accounted for, no difference was shown in the posttest. QtA lessons did not facilitate students’ ability to answer the 9 comprehension questions in general.
Table 10
ANCOVA Summary for Comprehension Questions
SS df MS F p
Covariate (pretest) 746.17 1 746.17 38.04 .000***
Group 53.08 1 53.08 2.71 .103
Error 1785.05 91 19.62
***p<.001.
Effects of QtA Lessons on Three Types of Comprehension Questions
Teasing out the comprehension total into three levels of question, the three rounds of ANCOVA analyses reveal the following findings.
Factual questions. As shown in Table 11, for the pretest of factual questions, the average score of the Group E (M = 5.54, SD = 2.41) was higher than that of Group C (M =
4.21, SD = 2.06). However, in the posttest, the trend goes in different directions. With the pretest difference partialed out, the adjusted mean of Group E dropped to 4.48a (SE = .32) while the adjusted average score of Group C ascended to 5.41a (SE = .31). The adjusted mean score of Group C in the posttest still exceeded that of Group E.
Table 11
Mean Scores for Group and Pre-Posttest on Factual Questions
Group Pretest Posttest Posttesta
Group E (n = 46) 5.54 (SD = 2.41) 4.72 (SD = 1.95) 4.48a (SE = .32) Group C (n = 48) 4.21 (SD = 2.06) 5.19 (SD = 2.48) 5.41a (SE = .31) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA was also performed to test the effect of the QtA intervention on factual questions. With the independent variable, group, controlled, a significant relationship is found between the covariate, pretest and the dependent measure, posttest, F (1, 91) = 12.50, p < .01, indicating a significant explanatory power in the posttest variance. The main analysis also showed a significant difference between groups in the posttest, F (1, 91) = 4.22, p < .05 (See Table 12). The results indicated that QtA lessons did not enhance the ability to answer factual questions. Rather, they decrease the literal understanding of text.
Table 12
ANCOVA Summary for Factual Questions
SS df MS F p
Covariate (pretest) 55.62 1 55.62 12.50 .001**
Group 18.78 1 18.78 4.22 .043*
Error 405.02 91 4.45
*p<.05. **p<.01.
Interpretive questions. In the pretest, the mean score of interpretive questions answered by Group E was 3.59 (SD = 2.47), which is similar to that of Group C (3.63, SD = 1.85). In the posttest, with the pretest scores covariated, Group E scored an adjusted average of 4.34a (SE = .29) whereas Group C achieved an adjusted mean of 4.49a (SE = .29) (see Table 13).
Table 13.
Mean Scores for Group and Pre-Posttest on Interpretive Questions
Group Pretest Posttest Posttesta
Group E (n = 46) 3.59 (SD = 2.47) 4.48 (SD = 2.33) 4.34a (SE = .29 ) Group C (n = 48) 3.63 (SD = 1.85) 4.35 (SD = 2.34) 4.49a (SE = .29 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA results on interpretive questions, again a significant relationship was found between pretest and posttest, F (1, 91) = 35.06, p < .001, indicating the contributing power of pretest to the model. In addition, there was no significant difference between groups on the posttest, F (1, 91) = .13 (See Table 14). After the QtA intervention, Group E did not show change in the answer to interpretive questions.
Table 14.
ANCOVA Summary for Interpretive Questions
SS df MS F p
Covariate (pretest) 139.19 1 139.19 35.06 .000**
Group .498 1 .498 .125 .724
Error 361.27 91 3.97
***p < .001.
Responsive questions. Table 15 displays the means of the answer to responsive questions for two groups in pre-posttests. In the pretest, Group E had lower mean score (M =
5.15, SD = 2.22) than that of Group C (M = 6.13, SD = 2.03). In the posttest, the adjusted mean score of Group E increased to 7.11a (SE = .27) while that of Group C decreased to 4.88 a, (SE = .27). Group E harvested a higher mean score than Group C in the posttest.
Table 15.
Mean Scores for Group and Pre-Posttest on Responsive Questions
Group Pretest Posttest Posttesta
Group E (n = 46) M = 5.15 ( SD = 2.22) M = 7.00 (SD = 1.81) 7.11a (SE = .27 ) Group C (n = 48) M = 6.13 (SD = 2.03) M = 4.98 (SD = 1.93) 4.88a (SE = .27 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA analysis on responsive questions presented in Table 16 shows that pretest variance significantly contributes to the explanation of posttest group difference, F (1, 91) = 5.59, p < .05. What’s more, there is a significant difference between groups on the posttest of responsive questions, F (1, 91) = 33.08, p < .001. Group E surpassed Group C in their answer to responsive questions after QtA treatment. QtA lessons did successfully enhance Group E students’ ability to answer responsive questions.
Table 16
ANCOVA Summary for Responsive Questions
SS df MS F p
Covariate (pretest) 18.71 1 18.71 5.59 .020*
Group 110.61 1 110.61 33.08 .000***
Error 304.27 91 3.34
*p < .05.***p < .001
In summary, QtA lessons contributed to great improvement on responsive questions.
However, no enhancement was found for interpretive questions and they lead to the decrease
in the scores of factual questions.
Effects of QtA Lessons on Written Response
Written response is analyzed in two dimensions: writing quantity in terms of number of words in writing, number of thought units in written response, and number of words per thought unit, and writing quality in terms of three kinds of reader-text interaction, textual, personal and intellectual.
Effects of QtA Lessons on Writing Quantity
Number of words. As indicated in Table 17, in the pretest response writing, Group E generated a mean of 85.20 words (SD = 31.32) which is lower than these produced by Group C (M = 90.73, SD = 31.75). In the posttest, the adjusted mean word written by Group E ascended to 99.14a (SE = 4.06) and the adjusted mean word written by Group C also scaled to 98.54a (SE=3.97).
Table 17.
Mean Scores for Group and Pre-Posttest on Number of Words
Group Pretest Posttest Posttesta
Group E (n = 46) 85.20 (SD = 31.32) 98.74 (SD = 27.36) 99.14a (SE = 4.06 ) Group C (n = 48) 90.73 (SD = 31.75) 98.92 (SD = 27.97) 98.54a (SE = 3.97 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA analysis indicates that there was no significant relationship between the covariate, pretest and dependent measure F (1, 91) = 2.40, posttest and that there was no significant difference between groups in the posttest, F (1, 91) = .01. (See Table 18). Group E did not surpass Group C on the number of words produced in response writing. As a consequence, QtA lessons did not assist the students in generating more words in written response.
Table 18
ANCOVA Summary for Number of Words
SS df MS F p
Covariate (pretest) 1812.92 1 1812.92 2.40 .125
Group 8.42 1 8.42 .01 .916
Error 68641.61 91 754.30
*p < .05.
Number of thought units. As shown in Table 19, in the pretest, Group E produced a higher mean units (M = 6.70, SD = 2.52) than Group C (M = 6.48, SD = 2.71). With the pretest variance covariated in the posttest, Group E generated lower adjusted mean units (M = 7.06a, SE = .29) than Group C (M = 7.51a, SE = .28).
Table 19
Mean Scores for Group and Pre-Posttest on Thought Units
Group Pretest Posttest Posttesta
Group E (n = 46) 6.70 (SD = 2.52) 7.09 (SD = 1.94) 7.06a (SE = .29 ) Group C (n = 48) 6.48 (SD = 2.71) 7.48 (SD = 2.20) 7.51a (SE = .28 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA analysis on the number of thought units for group reveals a significant contribution of pretest to the model, F (1, 91) = 13.05, p < .001. With the covariation, there was no significant difference between groups on the posttest, F (1, 91) = .27. No difference was found between Group E and Group C on thought unit production after the treatment (See Table 20). Therefore, QtA lessons did not enhance students’ writing quantity in terms of thought unit production.
Table 20
ANCOVA Summary for Thought Units
SS df MS F p
Covariate (pretest) 49.86 1 49.86 13.05 .000***
Group 4.82 1 4.82 1.26 .265
Error 347.77 91 3.82
***p < .001.
Words per unit in writing. Table 21 presents the results of number of words per thought unit in response writing. In the pretest, Group E produced a mean of 12.63 words (SD = 3.98) per thought unit, which is lower than that of Group C (M = 14.05, SD = 4.50). In the posttest, the adjusted mean words per unit produced by Group E escalated to 15.52a (SE = .75), while the adjusted mean words per unit written by Group C decreased to 13.29a (SE = .74).
Table 21
Mean Scores for Group and Pre-Posttest on Words per Unit
Group Pretest Posttest Posttesta
Group E (n=46) 12.63 (SD = 3.98) 15.21 (SD = 6.85) 15.52a (SE = .75 ) Group C (n=48) 14.05 (SD = 4.50) 13.58 (SD = 3.31) 13.29a (SE = .74 ) Note. a posttest mean score that is adjusted by pretest mean score
An ANCOVA analysis presents the significant relationship between the covariate, pretest, and the dependent measure, posttest, while controlling for the group variance, F (1, 91) = 11.53, p < .01, showing the explanatory power of covariate in the posttest variance. Most importantly, a significant difference in the posttest was detected, F (1, 91) = 4.42, p < .05 (See Table 22). Group E significantly exceeded Group C in generating more words per thought unit, which means QtA lessons were able to facilitate students to spawn more words per thought unit.
Table 22
ANCOVA Summary for Words per Unit
SS df MS F p
Covariate (pretest) 295.50 1 295.50 11.53 .001**
Group 113.26 1 113.26 4.42 .038*
Error 2332.59 91 25.63
*p < .05. **p < .01
To summarize, in terms of writing quantity, QtA lessons boosted students’ ability in generating words per thought unit despite that the text length, indicated by number of words and number of thought unit, was not affected.
Effects of QtA Lessons on Writing Quality
Another dimension of response writing, the quality of content tabulated as textual response, personal response, intellectual response and incorrect response were analyzed and compared as follows to examine the effects of QtA treatment.
Textual response. For textual response, Group E generated an average of 1.50 units (SD
= 1.81) and Group C generated 1.29 units (SD =1.52) in the pretest (See Table 23). In the posttest, Group E produced adjusted1.66a units of textual response (SE = .32), contrastively, Group C increased to an adjusted mean of 4.85a units (SE =.31).
Table 23
Mean Scores for Group and Pre-Posttest on Textual Response
Group Pretest Posttest Posttesta
Group E (n = 46) 1.50 (SD = 1.81) 1.67 (SD = 1.73) 1.66a (SE = .32 ) Group C (n = 48) 1.29 (SD = 1.52) 4.83 (SD = 2.47) 4.85a (SE = .31 ) Note. a posttest mean score that is adjusted by pretest mean score
An ANCOVA analysis showed that there was no significant contribution by the covariate,
pretest, to the model, F (1, 91) = .92. However, with the pretest covariated, there is a significant difference between groups in the posttest, F (1, 91) = 51.89, p < .001 (See Table 24). Group C largely surpassed Group E in their generation of textual response in the posttest.
This finding indicates that QtA lessons failed to aid in the generation of textual response; on the contrary, in comparison with control group, QtA might inhibit the generation of textual response.
Table 24
ANCOVA Summary for Textual Response
SS df MS F p
Covariate (pretest) 4.22 1 4.22 .92 .339
Group 237.51 1 237.51 51.89 .000***
Error 416.56 91 4.58
***p < .001.
Personal response. For personal response, the mean units generated in the pretest by Group E was 1.39 (SD =2.19) while that of Group C was 1.25 (SD = 1.82) (See Table 25). In the posttest, Group E had an adjusted average of .28a (SE = .09) personal response in their response writing and Group C had .08a (SD = .08). Both groups generated a meager number of personal response and they produced less personal response in their posttest than in pretest.
Table 25
Mean Scores for Group and Pre-Posttest on Personal Response
Group Pretest Posttest Posttesta
Group E (n = 46) 1.39 (SD = 2.19) .28(SD = .78) .28a (SE = .09 ) Group C (n = 48) 1.25 (SD = 1.82) .08(SD = .28) .08a (SE = .08 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA was also employed on the units of personal response to determine the effect of QtA, with pretest variation covariated. No significant impact from the covariate, pretest, was found, F (1, 91) = .12, on the model and no significant difference between groups was observed, F (1, 91) = 2.70, on the dependent measure, Personal Response, in the posttest (See Table 26). The treatment made no difference in making personal response.
Table 26
ANCOVA Summary for Personal Response
SS df MS F p
Covariate(pretest) .04 1 .04 .12 .728
Group .92 1 .92 2.70 .104
Error 30.95 91 .34
*p<.05.
Intellectual response. In the pretest, as shown in Table 27, the mean units of intellectual response in writing by Group E was 3.39 (SD = 2.51), slightly lower than that of Group C (M
= 3.67, SD=2.75). Nevertheless, in the posttest, the adjsuted mean unit of intellectual response in the response written by Group E soared to 4.84a (SE = .29), while the adjusted mean units in the written response by Group C dropped to 1.36a (SE = .28).
Table 27.
Mean Scores for Group and Pre-Posttest on Intellectual Response
Group Pretest Posttest Posttesta
Group E (n = 46) 3.39 (SD = 2.51) 4.80 (SD = 2.62) 4.84a (SE = .29 ) Group C (n = 48) 3.67 (SD = 2.75) 1.40 (SD = 1.23) 1.36a (SE = .28 ) Note. a posttest mean score that is adjusted by pretest mean score
Table 28 revealed the results of ANCOVA analysis on the posttest. Results showed that the covariate, pretest, did significantly relate to the dependent measure, posttest, F (1, 91) = 10.08, p < .01, contributing positively to the group variance in the posttest. Moreover, a significant difference was found between groups in the posttest, F (1, 91) = 75.13, p < .001 (See Table 28). Group E outperformed Group C in terms of the number of intellectual response generated in response writing after QtA intervention. Thus, QtA lessons are able to improve learners’ ability to generate more intellectual response in post-reading response writing.
Table 28
ANCOVA Summary for Intellectual Response
SS df MS F p
Covariate(pretest) 37.98 1 37.98 10.08 .002**
Group 282.96 1 282.96 75.13 .000***
Error 342.74 91 3.77
**p<.01***p<.001.
Incorrect response. As shown in Table 29, for the pretest, the mean units incorrect response generated by Group E was .41 (SD = .69) while that of Group C was .27 (SD = .76).
For the posttest, Group E reduced to an adjusted mean unit of .30a (SE = .19). However, Group C increased to a mean unit of 1.17a (SE = .19).
Table 29
Mean Scores for Group and Pre-Posttest on Incorrect Response
Group Pretest Posttest Posttesta
Group E (n = 46) .41 (SD = .69) .33 (SD = 1.03) .30a (SE = .19 ) Group C (n = 48) .27 (SD = .76) 1.17 (SD = 1.56) 1.19a (SE = .19 ) Note. a posttest mean score that is adjusted by pretest mean score
ANCOVA analysis with pretest scores as a covariate showed that there was a significant difference between groups, F (1, 91) =.10.77, p <.01(see Table 30) in the number of incorrect response found in response writing. However, there was no relationship between the covariate, pretest score and the dependent measure, posttest scores, F (1, 91) =3.72, indicating a weak explanatory power of covariate in the posttest between group variance. The increased number of incorrect response for Group C and the decreased number for Group E revealed the inhibiting effect of QtA in producing incorrect response. Indirectly, this suggests that the treatment facilitate students’ ability to make correct interpretation.
Table 30
ANCOVA Summary for Incorrect Response
SS df MS F p
Covariate(pretest) 6.40 1 6.40 3.72 .057
Group 18.52 1 18.52 10.77 .001**
Error 156.38 91 1.72
**p<.01.
In general, as far as writing quality was concerned, QtA lessons enhanced the ability to increase intellectual responses and decrease textual and incorrect response. The treatment made no difference on producing personal response.
Results of the Perception Questionnaire
To understand students’ perception of QtA after the intervention, a questionnaire tapping students’ preference of QtA features, perception of growth of QtA participation and the feasibility of future use, was administered. The following are the results of the questionnaire.
Preference toward QtA Approach
The first section explored participants’ preference toward QtA approach and the ranking of their favorite features. The result is displayed in Table 31. All the students (100%) held positive attitude toward the stories in QtA lesson. In total, 98 percent of the participants like QtA approach and have the intention to take more QtA classes in the future.
Table 31
Preference toward QtA Lessons
Items yes no total
1. Preference toward the stories in QtA lesson 46 (100%)
0 (0%)
N=46 (100 %) 2. Preference toward QtA approach in general 45
(98%)
1 (2%)
N=46 (100 %) 3. The intention to take more QtA classes in the future 45
(98%)
1 (2%)
N=46 (100 %)
To investigate the specific feature of QtA approach liked by the participants, students were asked to choose their favorite features in QtA approach and they were allowed to choose more than one feature. The result is shown in Table 32, ranking from highest to the least preference.
The top two features participants liked in QtA approach were students figuring out underlying message and teacher-posed queries, which were favored by 41 students out of 46 students (89 %). Some students said they “ no longer only look at the surface meaning of the text but am able to read the message the author tried to convey.” (E22) and other students perceived the advantage of teacher-posed queries to help them think, “I don’t like the traditional inflexible teaching method in Taiwan. In the old way, teachers just kept cramming things into us and there is absolutely no room for discussion. I like QtA method more because it helps me to learn to think and express my thought.”(E18)
The next feature that was preferred by the participants is whole class discussion and 83 percent of the students endorsed it. QtA lessons let them have the opportunities to take in various perspectives from their peers. They said, “I am able to see the same thing in various perspectives through discussion.”(E2) Besides, discussion also builds a lively atmosphere and better teacher-student interaction in the classroom. Students thought “Teaching is no longer so Table 32
Ranking of Favorite Features of QtA Approach
Items N= 46 (100 %)
1. Students figure out the message that the author wants to convey through the text in the story in whole-class discussion.
[students figuring out implicit message]
41 (89%)
2. The teacher poses open-ended but goal directed queries to guide students to comprehend the meaning of the text.
[teachers posing queries ]
41 (89%)
3. The whole class collaborates in discussion to construct the meaning of the story. [whole class discussion]
38 (83%) 4. Students pause to respond to the story with teacher asking
questions after reading a fragment of text. [students responding to the text segment]
36 (78 %)
5. Students discuss in a group of five to six members. [small group discussion]
17 (37%)
stifling. It is lively and interesting in class. “(E6) and “Even though we are in the class, there is no pressure and we would concentrate more.”(E21)
Following the above three features, 78 percent of the participants liked the feature of responding to the text segment after teacher asking question. When responding to text segment, students can better engage in reading. They said, “Through the prediction of the classmates in discussion, I would really want to know how the story will develop. This is very interesting. “(E23)
As for small group discussion, merely 37 percent of the students preferred this feature.
This result was further inquired among students and will be discussed in next chapter.
Conclusively, except for small group discussion, participants like most activities of the QtA approach including inducing underlying message, teacher-posed queries, whole class discussion and responding to the text segment.
Regardless of the fact that a large portion of students favor QtA approach, there are still some aspects that students showed concern toward QtA approach. The following are two major reasons why a few students did not really like QtA approach with example quote for each. For one thing, students cannot follow the content because of linguistic barrier or the teaching speed. They sometimes “can’t understand what the content is about because of the unfamiliar vocabulary and the teaching process goes too fast.”(E7) The other reason concerned them was that vocabulary and content-based knowledge from expository text was not explicitly taught like how they used to learn with expository textbook ,“Besides thinking, I can’t learn a lot from QtA lessons like vocabulary and other knowledge. Stories cannot provide us some content-based knowledge like the textbook.”
Perception of Growth in Ability through QtA Lessons
The second section investigated participants’ perception of growth in ability through QtA lessons. The results are presented in Table 33.
Up to 98 percent students believed their reading ability and thinking ability had improved through the QtA intervention (39% the most and 59 % more). Likewise, there are 98 percent participants who claimed QtA lessons enhanced their general English ability (15%
the most and 83% more). In addition to the above abilities, students also expressed their abilities of English speaking ability, reading speed, and vocabulary recognition had been improved along the process.
Feasibility of QtA Materials and Lessons
The third part probed participants’ attitudes toward the feasibility of QtA lessons. The results are presented in Table 34.
Table 34
Table 34