The Reading Skills Measured in SAET & DRET
In this section, the major findings concerning the research questions presented previously are reported and discussed. A total of 167 reading comprehension test items, 93 on the SAET and 74 on the DRET, were analyzed by utilizing the revised Nuttall’s taxonomy as the coding scheme. Each test item was classified according to the reading skill tested. Frequencies were run to investigate the frequency and distribution of reading level and skill type of reading comprehension test item in the SAET and the DRET. Table 6 presents the results of the items classified into two reading levels in both SAET and DRET from 2002 to 2007. According to Table 6, 70.7% aimed at testing bottom-up skills while 29.3% emphasized measuring top-down skills.
Table 6. Reading Level Tested in the SAET & DRET
Reading Level Count %
1. Reading for Plain Sense (Bottom-Up Skills) 118 70.7%
2. Reading into Discourse (Top-Down Skills) 49 29.3%
Total 167 100.0%
The results of item frequency on both SAET and DRET illustrated in Table 7 showed that, in general, “Recognizing and Interpreting Details” is the more frequent item on 2002 to 2007 SAET and DRET while “Recognizing Text Organization” is the least tested. In terms of readings skills at local and global level, more than half of the items of the items under analysis (around 70.7%) aimed at measuring the examinees’
reading skills at a more local level, with 9% on items testing “Word Inference from Context, 3% on items testing “Recognizing Cohesive Devices,” and 58.7% on items testing “Recognizing and Interpreting Details.” As shown in Table 6, Only 29.3%
were dedicated to testing reading skills at the global level, with items on
“Recognizing Functional Value,” 0.6% on “Recognizing Text Organization,” 3.6% on
“Recognizing Presuppositions Underlying the Text,” 9% on “Recognizing Implications and Making Inferences,” and 8.4% on “Recognizing and Understanding the Main Idea.”
Table 7. The Frequencies of Reading Skills Measured in the SAET & the DRET
Note. The percentages were calculated based on the total number of reading comprehension test items on 2002-2007 SAET and 2002-2007 DRET.
Frequencies were run separately to SAET and DRET to investigate the similarities and differences in the frequency and distribution of each reading level and
Item Type Count %
Skills necessary for reading for plain sense (Bottom-up Skills)
1. Word Inference from Context 15 9.0%
2. Recognizing Cohesive Devices 5 3.0%
3. Recognizing and Interpreting Details 98 58.7%
Skills necessary for reading beyond plain sense and into discourse (Top-down skills)
4. Recognizing Functional Value 13 7.8%
5. Recognizing Text Organization 1 0.6%
6. Recognizing Presuppositions Underlying the Text 6 3.6%
7. Recognizing Implications and Making Inferences 15 9.0%
8. Recognizing and Understanding the Main Idea 14 8.4%
Total 167 100.0%
skill type of reading comprehension test item between the SAET and the DRET from 2002 to 2007. Table 8 presents the results regarding the reading level tested in the SAET & the DRET from 2002 to 2007.
Similarities and Differences between the SAET & the DRET
According to Table 8, in the SAET, 62.4% tested bottom-up skills while only 37.6% were devoted to measuring top-down skills. As for the DRET, 60.8% aimed at measuring bottom-up skills while only 39.2% were dedicated to top-down skills.
Reading Skills Tested in the SAET. As shown in Table 9, the research data revealed that only seven types of reading skills were identified on 2002-2007 SAET, Table 8. Reading Level Tested in the SAET & DRET from 2002 to 2007
Category Reading Level Count %
SAET 1. Reading for Plain Sense (Bottom-Up Skills) 68 73.1 2. Reading into Discourse (Top-Down Skills) 25 26.9
Total 93 100.0
DRET 1. Reading for Plain Sense (Bottom-Up Skills) 50 67.6 2. Reading into Discourse (Top-Down Skills) 24 32.4
Total 74 100.0
including “Word Inference from Context,” “Recognizing and Interpreting Details,”
“Recognizing and Interpreting Details,” “Recognizing Functional Value,”
“Recognizing Presuppositions Underlying the Text,” “Recognizing Implications and Making Inferences,” and “Recognizing and Understanding the Main Idea.” The most frequent item tested in the SAET is “Recognizing and Interpreting Details” (61.3%, 57 items). Items on “Word Inference from Context” came second (10.8%, 10 items), followed by items on “Recognizing and Understanding the Main Idea” (8.6%, 8 items). Items on “Recognizing Functional Value” (7.5%, 7 items) and items on
“Recognizing Implications and Making Inferences” (7.5%, 7 items) were the third frequent items, followed by a very small number of occurrences of items on
“Recognizing Presuppositions Underlying the Text” (3.2%, 3 items). Only one item was coded as “Recognizing Cohesive Devices” (1.1%). On 2002 to 2007 SAET, none was labeled as “Recognizing Text Organization items.” In terms of the skills on the SAET, around 73.1% aimed at measuring skills at the local level. As for skills at the global level, only 26.9% of items tested reading skills at the global level.
Reading Skills Tested in the DRET. As illustrated in Table 9, items measuring
“Recognizing and Interpreting Details” were the most frequent item type (55.4%, 41 items). Items labeled as “Recognizing Implications and Making Inferences” (10.8%, 8 items) came the second. The third frequent item type contained “Recognizing Functional Value” (8.1%, 6 items) and “Recognizing and Understanding the Main Idea” (8.1%, 6 items), followed by a very small number of occurrences of items on Table 9. Reading Skills Measured in both SAET & DRET
Item Type SAET DRET
3. Recognizing and Interpreting Details 57 61.3% 41 55.4%
Skills necessary for reading beyond plain
“Recognizing Text Organization” (1.4%, 1 item), which is the least frequent item type on 2002-2007 DRET. Different from the results of the items on 2002-2007 SAET, all the eight skills on the revised Nuttall’s taxonomy were found on the DRET items from 2002 to 2007. “Recognizing Text Organization,” which was not found on 2002-2007 SAET, was measured on the DRET. Similar to the SAET, test items in the DRET also focused on measuring students’ abilities to recognize and interpret details in the passages while placing less emphasis on testing higher-order skills, which require readers to have a more global understanding of texts. Overall, 67.6% aimed at testing local reading skills while only 32.4% measured global reading skills.
Similarities and Differences between the Reading Skills Measured in the SAET
& the DRET
Similarities. As shown in Table 8, in terms of the reading level tested, both examinations revealed a similar pattern: more than 60% of items emphasized on testing the examinees’ ability to use bottom-up reading skills. Much fewer percentages were used to measure the examinee’s ability to employ more top-down skills. In addition, as shown in Table 9, in terms of frequency, both examinations revealed a similar pattern: the most frequent items were items on “Recognizing and Interpreting Details” (61.3% on the SAET & 55.4% on the DRET), with much less number of items devoted to measuring other skills. Similarly, both exams had more than 60% of items testing local level reading skills. In addition, in both examinations, three types of items had occurrences of less than five times over the past six years, including items on “Recognizing Cohesive Devices” (1.1 %, 1 item in the SAET and 5.4%, 4 items in the DRET), items on “Recognizing Presuppositions Underlying the Text” (3.2%, 3 items in the SAET and 4.1%, 3 items on the DRET), and items on
“Recognizing Text Organization” (0% in the SAET and 1.4%, 1 item in the DRET).
This indicated that in both examinations, certain types of reading skills received less
emphasis. As can be seen from Table 8, the distribution between the two examinations showed that both examinations consisted of at least three item types each year, ranging from three to eight among the six years. In terms of the distribution of items in the SAET, there were five types in 2002, five types in 2003, three types in 2004, four types in 2005, five types in 2006, and six types in 2007. As for the distribution of items in the DRET, eight types were found in 2002, six types in 2003, four types in 2004, three types in 2005, five types in 2006, and six types in 2007.
Hence, it’s hard to determine whether there was a pattern of skill distribution as the skill types fluctuated year by year.
Differences. Based on close examination on the frequency of item in each year, three kinds of major differences between the SAET and the DRET were found: (1) frequency, (2) occurrences and (3) distribution of the reading skills. Firstly, in terms of frequency as shown in Table 7, the most noticeable difference between the SAET and the DRET is that the SAET had more items on “Recognizing and Interpreting Details than the DRET,” 5.9% more on the overall percentage. In addition, the SAET also had more items on “Word Inference from Context” than the DRET, 5% more on the overall percentage. To examine closely the differences between the two types of examinations, the frequency of item types year by year were further examined, which were shown in Table 10. Although the most frequent item in both exams was
“Recognizing and Interpreting Details” item, more than 50% of items were devoted to measuring “Recognizing and Interpreting Details” in the SAET each year while in the DRET only five years (2002, 2003, 2004, 2006, 2007) had more than 50% of items on
“Recognizing and Interpreting Details.” The frequency of “Recognizing and Interpreting Details” item dropped slightly to 45.5% in 2005 DRET. Regarding occurrences of reading skills shown in Table 8, “Recognizing Text Organization” was
tested in the DRET (2002) but not tested in the SAET. Items which appeared every year in 2002-2007 SAET are those on “Word Inference from Context” and
“Recognizing and Interpreting Details” whereas only those on “Recognizing and Interpreting Details” appeared consistently in the DRET from 2002 to 2007. In addition, items on “Recognizing Implications and Making Inferences” occurred nearly every year except in 2004 SAET and in 2005 DRET. Moreover, items on
“Recognizing and Understanding the Main Idea” occurred nearly every year except in the 2005 SAET; while item of this type only occurred in the 2002, 2003, and 2005 DRET. Thus, the results indicated that certain items were favored in the SAET: e.g., items on “Word Inference from Context,” “Recognizing and Interpreting Details,”
“Recognizing Implications and Making Inferences” and “Recognizing and Understanding the Main Idea.” On the other hand, the DRET had more items on
“Recognizing and Interpreting Details” (occurred every year), “Word Inference from Context” (occurred in 2002, 2003, 2005, 2006) and “Recognizing Functional Value”
(occurred in 2002, 2003, 2004, and 2006). Another difference between the two examinations was in the distribution of item types for each year. The results showed that on the SAET, two item types occurred each year, including items on “Word Inference from Context,” and items on “Recognizing and Interpreting Details.” As to the DRET, only items on “Recognizing and Interpreting Details” were tested every year. In sum, the results indicated items on “Recognizing and Interpreting Details”
were the majority on both SAET and DRET. Thus, it indicated that both exams emphasized measuring examinees’ abilities to identify and interpret specific information in the reading passages, which belonged to the more local level comprehension.
Table 10. Reading Skills Measured in 2002-2007 SAET & DRET
Skills necessary for Reading Beyond the Plain and into Discourse (Top-down Skills)
Skills necessary for Reading Beyond the Plain and into Discourse (Top-down Skills)