CHAPTER ONE INTRODUCTION
This chapter provides the background and the rationale of this study. The first section presents the background and motivation behind the research whereas the second section identifies the purposes and research questions. The significance of the study and the organization of the thesis are described in the last two sections.
Background and Motivation
English reading plays a crucial role in secondary and higher education in the EFL (English as a Foreign Language) context in Taiwan because it is one of the major inputs students receive while learning English and is a major skill measured particularly in nationwide examinations. College entrance examination has a great impact on senior high school students and teachers because it brings consequences on learners’ future study and thus will either directly or indirectly influence English instruction. College Entrance Examination Center (henceforth CEEC) in Taiwan implements a two-stage examination—Scholastic Achievement Test (SAT) and Department Required Test (DRT). In the first stage of screening, students who take the SAT and meet the qualifications set by a particular university are recommended by their high school to the university. SAT aims to assess students’ basic scholastic
knowledge and abilities1. DRT, implemented in 20022, is designed to identify students that possess the ability in certain subject area required by a particular university.
Usually, a university requires the DRT test results of three to six subjects. Based on the requirements of their favorite schools, students then can choose the subjects they
1The SAT is designed to meet the need of filtering potential candidates for universities at the early stage. The aims and focuses of the SAT are as follows:
(1) To see whether candidates have the required high school level abilities in foundational subjects.
(2) To see whether candidates have acquired the required abilities for college education.
(3) General-education oriented: To see whether candidates have abilities to combine the knowledge with real life experience or to integrate it with different fields.
(4) To see whether candidates have abilities of comprehension and application of the knowledge.
2 DRET was originally called Joint College Entrance Examination (henceforth JCEE) before 2002. It was implemented in 2002 when the new college admission system was introduced.
want to take the test for. The English achievement test in the second stage is thus comparatively more difficult than that of the first stage and aims at discriminating more proficient students in English subject area.
On both types of English tests, reading comprehension has been a major
component in measuring students’ reading ability. Generally, a reading comprehension section contains a series of passages, each followed by three to five multiple-choice questions. Those questions are assumed to tap different reading or thinking skills.
Whether or not the reading comprehension questions truly assess an examinee’s reading ability is a question to be explored. The CEEC makes annual reports on item analysis of both SAET (Scholastic Achievement English Test) and DRET
(Department Required English Test), yet studies on the qualitative analysis of reading comprehension questions have not been sufficiently conducted. The statistical
analyses conducted by CEEC revealed that most of the items had good statistical values, but test items with good statistical values might not necessarily be good items (Huang, 1994).
Recently, two studies (Hsu, 2005; Lu, 2002) examined some qualitative factors such as the reading skills measured on each test item (by using Mo’s classification) and the factors affecting students’ reading test performances in both SAETs and DRETs separately. However, the scheme (Mo, 1987) they used for analysis does not clearly show the cognitive process levels in reading that each test item tries to
measure. Cognitive abilities such as reasoning and analyzing are aims and foci of the DRET in particular3. The present study, viewing reading as a cognitive activity
3The DRT is designed to meet the need of filtering proficient candidates for universities. The aims and focuses of the DRT are as follows:
(1) To see whether candidates understand the important knowledge/concepts in the disciplines.
(2) To see whether candidates have abilities to read, to judge, to infer, and to analyze the materials.
(3) To see whether candidates have abilities to express their ideas.
(4) To see whether candidates have abilities to apply the knowledge/concepts in the disciplines.
(Bernhardt, 1991), attempts to apply another framework—the Revised Bloom’s Taxonomy, which contains four knowledge types (Factual, Conceptual, Procedural, and Metacognitive Knowledge) and six cognitive process levels (Remember, Understand, Apply, Analyze, Evaluate, and Create), to analyze the reading comprehension test items on both SAET and DRET.
Bloom’s Taxonomy has been widely used in curriculum, instruction, or
assessment design and evaluation, especially on test item analysis across subject areas (Airasian & Miranda, 2002; Aviles, 1999; Chen, 2004; James , 2002; Kastberg, 2003).
In year 2001, Anderson and Krathwohl, removing the defects found in the original version, proposed a revision for the original Taxonomy. The major difference between the two versions is that the revised Taxonomy includes two dimensions: knowledge and cognitive process dimensions, which provide a clear framework for test designers to know what to test and how to test it. However, among the studies that applied Bloom’s Taxonomy on test item analysis, little research could be found on ESL/EFL reading tests (Lee, 2004) and no research on the SAET and the DRET reading comprehension test item analysis was found. Thus, it is worthwhile to conduct a research on test content analysis to investigate the reading comprehension sections in both types of nationwide examinations by utilizing the revised Taxonomy. Moreover, some quantitative analyses of examinees’ performances on different types of test items will be carried out to describe learners’ strengths and weaknesses in English reading and to further provide suggestions for English reading instruction.
Research Questions
The purpose of the study is threefold. First, it aims to analyze and compare qualitatively and quantitatively the reading comprehension test items on the SAET and the DRET from 2002 to 2006. The decision to select questions taken from tests between 2002 and 2006 is based on the assumption that, over a five-year period,
changes may occur in the content of the questions and the cognitive levels tested. This study explores whether the entrance exams are well designed to adequately and fairly measure students’ reading ability. Second, it intends to explore the differences
between two types of examinations and changes in test design in the past five years.
Third, examinees’ performances on different question types will also be investigated to further provide suggestions on test design and on EFL reading instructions in senior high schools. Specifically, the study explorers the following questions:
1. What cognitive process levels and knowledge types in the Revised Bloom’s Taxonomy were measured in the SAET and the DRET reading comprehension sections?
2. How did the SAET and the DRET differ in measuring the cognitive process levels and knowledge types?
3. How did the examinees in general perform on reading comprehension questions of different cognitive process levels and different knowledge types in the SAET and the DRET?
4. How might high scorers differ from low scorers in their performances on reading comprehension questions that measured different cognitive process levels and different knowledge types in the SAET and the DRET?
Significance of the Study
This study is significant in two main aspects. First, the study promotes a better understanding of the content and construct of the reading comprehension questions in the SAET and the DRET in the past five years. It seeks to examine the changes in both test design, to see if the test items do measure various cognitive levels in English reading. It should be noted that this study did not try to describe students’ cognitive processing while taking the tests, but to identify the possible cognitive process levels and knowledge types intended to be tested on the items.
Second, through examining test takers’ performances on each question type, implications on English reading instruction can be yielded. Senior high school teachers may then know what specific cognitive skills in reading are most needed for students with different proficiency levels and understand what cognitive skills students need to possess in order to pass the entrance exams. Meanwhile, from the analyses, teachers can learn to design questions or classroom activities to promote learners’ thinking skills such as reasoning and analyzing in reading; teachers can also learn to construct test items/assessments in assessing those abilities.
To conclude, based on the analyses and comparisons of the data from SAETs and DRETs, this study attempts to provide a greater understanding of the test item design in nationwide examinations and then to give practical implications for reading comprehension test item writing as well as for reading instruction in senior high schools.
Organization of the Thesis
The remainder of this study is structured as follows. Chapter Two reviews the literature regarding the rationale of reading, the studies of reading comprehension tests, the revision of Bloom’s Taxonomy and its application on testing and assessment.
In Chapter three, the methodology, including the materials, the instruments, and the procedure of data analysis, is presented. Chapter Four elaborates the results for the various analyses. Finally, Chapter Five discusses the major findings and concludes the study with pedagogical implications, limitations of the current study, and suggestions for future research.
