As shown in the preceding section, eight types of skills were measured in 2002-2007 SAET and DRET. To analyze the examinees’ overall performance on each reading skill in both SAET and DRET, the passing rates on each item type were examined. A two-way ANOVA was applied to the SAET and DRET to see if there were significant differences on students’ average passing rates among various question types, and to see whether these differences were consistent throughout the six years. The passing rates are used to show the proportion of the examinees who got an item correct. In the ANOVA analyses, the item types and the year of exams were used as the factors. Descriptive statistics for the passing rates on each item type on the SAET from 2002 to 2007 and the results of ANOVA are presented in the following section, followed by those of the DRET.
SAET
Descriptive statistics for the passing rates on each reading comprehension test item from 2002 to 2007 SAET are summarized in Table 9. Table 11 shows that, in the 2002 SAET, only five types of reading skills were identified and the total mean of the passing rate was 60.8 (SD=12.33). The passing rates for “Recognizing and Understanding the Main Idea” items were the highest of all, which belonged to global reading skill types, followed by “Recognizing Cohesive Devices” items (Mean=75),
“Recognizing and Interpreting Details” items (Mean=61.1111, SD=10.9), “Word Inference from Context” items (Mean=58, SD=1.41), and “Recognizing Implications and Making Inferences” items (Mean=45, SD=7.07).
In the 2003 SAET, five types of reading skills were identified. The total mean of the passing rates was 51.3333. Items on “Recognizing and Understanding the Main Idea,” a global reading skill, received the highest passing rates (Mean=61.3333),
Table 11. Average Passing Rates of Items Measuring Different Reading Skills in 2002-2007 SAET and DRET
followed by items on “Recognizing Presuppositions Underlying the Text” (Mean=51), items on “Recognizing and Interpreting Details” (Mean=50.75), items on “Word Inference from Context” (Mean=44), and items on “Recognizing Implications and Making Inferences” (Mean=41).
As shown in Table 11, only three types of reading skills were identified in the 2004 SAET. The total mean of the passing rates was 58.33. Items on “Word Inference from Context,” which belonged to the local reading skill, were the best performed, with the highest passing rates of 61, followed by items on “Recognizing and Interpreting Details” (Mean=58.64), and items on “Recognizing and Understanding the Main Idea“(Mean=54).
In the 2005 SAET, only four types of reading skills were identified. The total mean of the passing rates was 55.88. “Recognizing Implications and Making Inferences” items, which belonged to the global reading skill level, received the highest passing rates (Mean=66), followed by “Word Inference from Context” items (Mean=58), “Recognizing and Interpreting Details” items (Mean=55.2), and
“Recognizing Functional Value” (Mean=54.5).
Table 11 also shows that, in 2006 SAET, only four types of reading skills were identified. The total mean of the passing rate was 56. “Recognizing Implications and Making Inferences” items, which was a global reading skill, had the highest passing rates (Mean=76). This was followed by “Recognizing and Interpreting Details” items (Mean=59.7), “Word Inference from Context” (Mean=41), and “Recognizing Functional Value “(Mean=40.5).
In the 2007 SAET, 6 types of reading skills were identified, and the total mean of the passing rate was 57.75. Items on “Recognizing and Understanding the Main Idea”, which was a global reading skill, received the highest passing rates (Mean=77),
followed by items on “Recognizing Functional Value” (Mean=67), items on
“Recognizing and Interpreting Details” (Mean=60.44), items on “Recognizing Presuppositions Underlying the Text” (Mean=54.5), items on “Word Inference from Context” (Mean=43), and items on “Recognizing Implications and Making Inferences” (Mean=41).
In general, in the SAET from 2002 to 2007, the ranking of passing rates among all item types were not consistent each year. In 2002, 2003 and 2007, items on
“Recognizing and Understanding the Main Idea” ranked the highest, while items on
“Word Inference from Context” ranked the highest in 2004, and items on
“Recognizing Implications and Making Inferences” ranked the highest in 2005 and 2006. In other words, from 2002 to 2007, more global-level reading skills tend to have the highest passing rates. As for the worst performed items each year, items on
“Recognizing Implications and Making Inferences” ranked the lowest in and 2002, 2003 and 2007 while items on “Recognizing and Understanding the Main Idea”
ranked the lowest in 2004, and items on “Recognizing Functional Value” ranked the lowest in 2005 and 2006. The results showed that from 2002 to 2007, the items which had the lowest passing rates all tested global reading skill. One thing worth noting is that items on “Recognizing and Understanding the Main Idea” had the highest passing rates in 2002, 2003, and 2007 while ranked the lowest in 2004.
To yield statistical support to these observations about the inconsistent passing rates given by each item type throughout 2002-2007, a two-way ANOVA was conducted. This is done to see whether the differences on passing rates among items measuring different reading skills were significant, and to see whether these differences exhibited consistency throughout the years. The results of the ANOVA analysis were show in Table 12. The results revealed that there was no significant
effect on the variable of the item type (F=1.169, p>.05), which indicated the passing rates did not vary according to the types of skills measured. In addition, there was no effect on the variable of the year (F=.728, p>.05), which indicated that the passing rates do not vary throughout the years. Also, no significant effect was found on the variable of the interaction of item type and year (F=.927, p>.05), which indicated that the discrepancy of the average passing rates on each item type would not differ throughout years and each of these years showed a similar pattern. Furthermore, from Table 11, it can also be seen that different item types appeared in different years.
Hence, it’s hard to find a general pattern of item types from 2002 to 2007 SAET.
Table 12. The ANOVA Analysis of the Passing Rates in the SAET Dependent Variable: Passing Rates Intercept 113352.846 1 113352.846 561.259 .000
Item Type 1416.596 6 236.099 1.169 .334 DRET from 2002 to 2007 is presented in Table11. As illustrated in Table 11, in 2002 DRET, the total mean of the passing rates was 45.27. “Recognizing Cohesive Devices” items received the highest mean (Mean=57), followed by “Word Inference from Context” items (Mean=54), “Recognizing Functional Value” items (Mean=51),
“Recognizing Text Organization” items (Mean=50), “Recognizing and Interpreting Details” items (Mean=45.5), “Recognizing and Understanding the Main Idea”
(Mean=42), “Recognizing Implications and Making Inferences” (Mean=31), and
“Recognizing Presuppositions Underlying the Text” items had the lowest passing rates (Mean=30).
For the 2003 DRET (as shown in Table 11), the total mean of the passing rates was 45.40. Items on “Recognizing Functional Value” were the best-performed, with a mean of 54.5 (SD=26.16), followed by items on “Recognizing and Understanding the Main Idea” (Mean=49), items on “Word Inference from Context” (Mean=48), items on “Recognizing and Interpreting Details” (Mean=46.38), items on “Recognizing Cohesive Devices” (Mean=43), and items on “Recognizing Implications and Making Inferences” (Mean=30.5), which got the lowest average passing rates.
As shown in Table 11, in 2004 DRET, the total mean of the average passing rates was 39.9091. Items on “Recognizing Functional Value” (Mean=48) obtained the highest passing rates, followed by items on “Recognizing Cohesive Devices”
(Mean=44.5), items on “Recognizing and Interpreting Details” (Mean=37.83). Items on “Recognizing Implications and Making Inferences” had a quite low passing rate of 27 only, the lowest in 2004 DRET and also the lowest in the DRETs over the past 6 years.
From Table 11, it can be seen that, in the 2005 DRET, the total mean of the average passing rates was 40.1. Items on “Recognizing and Understanding the Main Idea” (Mean=56.75) received the highest passing rate, followed by items on “Word Inference from Context” (Mean=50.5), and items on “Recognizing and Interpreting Details” (Mean=46.4).
As shown in Table 11, in 2006 DRET, the total mean of the average passing rates was 41.09. Items on “Recognizing Functional Value” (Mean=68) got the highest passing rate, followed items on “Recognizing Implications and Making Inferences”
(Mean 45.5). Items on “Recognizing and Interpreting Details” (Mean=39) ranked the
third, followed by items on Word Inference from Context (Mean=31). Items on
“Recognizing Presuppositions Underlying the Text” got the lowest passing rate of 28.
As illustrated in Table 11, in 2007 DRET, the total mean of the average passing rates was 45.82. Items on “Recognizing and Interpreting Details” (Mean=39) were the best performed, followed by items on “Recognizing Implications and Making Inferences” (Mean=40) and items on “Recognizing Presuppositions Underlying the Text” were the worst performed.
In sum, in the DRET from 2002 to 2007, similar to the results in the SAET, the ranking of passing rates of different item types were not consistent throughout the years. Items on “Recognizing Cohesive Devices” ranked the highest in 2002, items on
“Recognizing and Understanding the Main Idea” ranked the highest in 2003 and 2005, while items on “Recognizing Functional Value” ranked the highest in 2004 and 2006 and items on “Recognizing and Interpreting Details” ranked the highest in 2007. As for items that ranked lastly, items on “Recognizing Presuppositions Underlying the Text” ranked the lowest in 2002, 2006, and 2007, items on “Recognizing Implications and Making Inferences” ranked the lowest in 2003 and 2004, and items on
“Recognizing and Interpreting Details” ranked lastly in 2005. Thus, it’s difficult to determine whether the reading skills tested in the DRET exhibited certain patterns.
To yield statistical support to these observations about the inconsistent passing rates given by each item type throughout 2002-2007, a two-way ANOVA was conducted. This is done to see whether the differences on passing rates among items measuring different reading skills were significant, and to see whether these differences exhibited consistency throughout the years. The results of the ANOVA analysis of the DRET were shown in Table 13.
Table 13. The ANOVA Analysis of Passing Rates in the DRET SAET. The results of the ANOVA showed that there was no effect on the variable of item type (F=1.918, p>.05), the year (F=.626, p>.05), and the interaction between the item type and year (F=.505, p>.05). Thus, the ANOVA analyses also indicated that in the DERT, the passing rates did not vary according to the types of reading skills measured on each time, the year, or the interaction of item types and year. However, if we examine the passing rates of different item types throughout 2002 to 2007 DRET, the passing rates of item types were not consistent throughout the years. For example, “Word Inference from Context” had a passing rate of 54 in 2002, 48 in 2003, 50.5 in 2005, but dropped to only 31 in 2006 DRET. In addition, not all types of skills were the most or the least difficult ones. Since different kinds of skills appeared in different years, the passing rates of the specific skills cannot be compared.