2010年至2017年英語科國中基測與會考克漏字測驗分析 - 政大學術集成

全文

(1)國⽴政治⼤學英國語⽂學系碩⼠在職專班碩⼠論⽂. 指導教授：尤雪瑛博⼠ Advisor: Dr. Hsueh-ying Yu. 立. 政治大. ‧. ‧ 國. 學. 2010 年⾄ 2017 年英語科國中基測與會考克漏字測驗分析. y. Nat. er. io. sit. Analysis of the Cloze Test Items in English Basic Competence Test and. n. Comprehensive Assessment Program from a v 2010 to 2017. i l C hengchi Un. 研究⽣：彭凡家撰 Name: Fan-chia Peng 中華民國⼀百零七年六⽉ June, 2018. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(2) Analysis of the Cloze Test Items in English Basic Competence Test and Comprehensive Assessment Program from 2010 to 2017. A Master Thesis Presented To Department of English National Chengchi 政治University. 大. 立. ‧. ‧ 國. 學 er. io. sit. y. Nat n. al. ni C h Fulfillment In Partial engchi U. v. of the Requirements for the Degree of Master of Arts. By Fan-chia Peng June, 2018. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(3) Acknowledgements. First of all, I would like to show my deepest gratitude to my thesis advisor, Dr. Hsueh-ying Yu of the Department of English of the National Chengchi University. With her inspiring and patient guidance, I could complete my thesis. Besides, when I was in the time of frustration, I could always have some relief from her words of wisdom and her consistently encouragement. My sincere thanks also extend to my thesis committee members, Dr. Chieh-yue Yeh and Dr. Hsi-nan Yeh. With their insightful suggestions and comments on the. 政治大. 立 more complete and persuasive. manuscript, I could make my thesis. ‧ 國. 學. I would like to show my appreciation to all my classmates in ETMA. I was. ‧. grateful with their company. Thanks to them, I had more confidence to fight for the. y. Nat. er. io. sit. huge stress of the studies during the times of several exhausted but beautiful summers. n. in ETMA. My special thank goesato i v of the thesis, for her l Grace Hsu, the coder. Ch. assistance to my research.. n engchi U. Lastly, I am much obliged to my husband Tsung-hsin Liu and my family for their support and love from the beginning to the end of this wonderful journey.. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(4) TABLE OF CONTENTS Acknowledgements…………………………………………………………………...iii Table of Contents …………………………………………………………………….iv List of Tables ………………………………………………………………………. ...v List of Figures ………………………………………………………………………..vi Chinese Abstract …………………………………………………………………….vii English Abstract …………………………………………………………………….viii Chapter One: Introduction ......................................................................................... 1 Background and Motivation ...................................................................................... 1 Significance of the Study ........................................................................................... 4. 政治大. 立............................................................................... 5 Chapter Two: Literature Review. ‧ 國. 學. Testing and Assessment in Curriculum ..................................................................... 5. ‧. Approaches to Language Testing........................................................................... 6. sit. y. Nat. er. io. The Development of Cloze Test ................................................................................ 8. n. a. v. l C............................................................................ Measurement of the Readability 8 ni hengchi U. Measurement of the Reading Comprehension ....................................................... 9 Types of Modified Cloze Tests ............................................................................ 10 What Kind of Language Skill Does Cloze Measure? .............................................. 15 Relevant Research on Cloze Test in Taiwan ........................................................... 17 BCT and CAP .......................................................................................................... 18 BCT ...................................................................................................................... 19 CAP ...................................................................................................................... 19 Chapter Three: Methodology ................................................................................... 21 Materials .................................................................................................................. 21. iv. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(5) Instrument ................................................................................................................ 22 Modification of Jonz's Subcategories ...................................................................... 27 Data Analysis ........................................................................................................... 35 Chapter Four: Results and Discussion..................................................................... 39 Results of the Five Categories ................................................................................. 39 The Results of Jonz’s 37 Subtypes in BCT and CAP .............................................. 44 The Distribution of Jonz’s 5 Categories in Each Year of BCT and CAP ................ 49 Results of the Distribution of Jonz’s 37 Subtypes in BCT and CAP....................... 53 Chapter Five: Conclusion.......................................................................................... 59 Summary of the Major Findings .............................................................................. 59. 政治大. Implications.............................................................................................................. 61 立. ‧ 國. 學. Limitations and Suggestions .................................................................................... 62. ‧. References .................................................................................................................... 63. sit. y. Nat. n. al. er. io. Appendix A…………………………………………………………………………..67. i Un. Ch. v. engchi Appendix B .................................................................................................................. 68 Appendix C…………………………………………………………………………...71 Appendix D .................................................................................................................. 76 Appendix E .................................................................................................................. 77. v. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(6) LIST OF TABLES Table 3. 1 Distribution of Reading Passages and Test Items in Cloze Test ................ 22 Table 3. 2 Jonz's (1990) Subtypes of Cloze Items ....................................................... 24 Table 3. 3 Modified Taxonomy of the Current Study ................................................. 34 Table 4. 1 The Distribution of Items in the Five Categories ………………………...40 Table 4. 2 The Distribution of Jonz’s Subtypes in BCT and CAP .............................. 45 Table 4. 3 The Distribution of Jonz’s Five categories from 2010 to 2017 .................. 50 Table 4. 4 The Numbers of Words and Readability in BCT ....................................... 52 Table 4. 5 The Numbers of Words and Readability in CAP ....................................... 52 Table 4. 6 The Distribution of Jonz’s 37 Subtypes in BCT and CAP ......................... 54. 治. 政 in CAP ...................................................... Table 4. 7 The Passing Rates of the Items 56 大. 立. ‧. ‧ 國. 學. n. er. io. sit. y. Nat al. Ch. engchi. i Un. v. vi. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(7) LIST OF FIGURES Figure 2. 1 The Flowchart of Jonz's Interpretation of Bachman's Categories……..…13 Figure 3. 1 Data Analysis Procedure of the Study ....................................................... 35. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat al. Ch. engchi. i Un. v. vii. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(8) 國⽴政治⼤學英國語⽂學系碩⼠在職專班碩⼠論⽂提要. 論⽂名稱：2010 年⾄ 2017 年英語科國中基測與會考克漏字測驗分析指導教授：尤雪瑛博⼠研究⽣：彭凡家論⽂提要內容：. 立. 政治大. ‧ 國. 學. 對台灣國中⽣⽽⾔，2001 年實施⾄ 2013 年的國中基本學⼒測驗以及 2014 年. ‧. 開始實施的國中教育會考，是兩項重要的⾼中⼊學考試。其中，英⽂閱讀測驗題. y. Nat. sit. er. io. 組中每年會有⼆篇的克漏字測驗，克漏字測驗能夠測驗受試者英⽂學習的整合能. n. a. iv. l C ⼒，且在台灣被廣泛地使⽤於各項考試中，本研究主要⽬的在於：(1)研究從 2010 Un h engchi. 年⾄ 2017 年基測與會考的克漏字試題主要測驗了哪些語⾔能⼒; (2)分析這 8 年的克漏字試題分別在基測與會考中測驗的能⼒為何，以及討論其相同與相異之處。本研究使⽤ Jonz (1990)的克漏字分類法為其研究⼯具以探討其分類結果。本研究結果顯⽰：(1) 2010 年⾄ 2017 年基測與會考著重於受試者要能找出跨越句⼦與句⼦間⽂本線索的能⼒; (2) 2010 ⾄ 2013 年的基測，著重於檢視受試者時態句法的能⼒; (3) 2014 ⾄ 2017 年的會考，著重於檢視受試者篇章結構的能⼒。根據本研究發現，筆者針對未來研究⽅向及教學實務提出了建議。關鍵字：克漏字測驗、國中教育會考、國中基本學⼒測驗 viii. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(9) Abstract To every ninth-grade student of junior high school, the Basic Competence Test (BCT) from 1999 to 2013 and the Comprehensive Assessment Program (CAP) from 2014 are the two most important senior high school entrance examinations. Among the reading tests of both exams, two of them are the cloze tests. The cloze test targets at measuring the candidates’ performances on the integrated ability in language, and it is widely used in the different testing contexts in Taiwan. The research thus aims to investigate: (1) from 2010 to 2017, what kind of language knowledge in the cloze tests were measured in BCT and CAP, and (2)治 the similarities and differences of the. 政. 大. 立 in BCT and CAP. To achieve the purpose, this cloze item types tested and designed. ‧ 國. 學. research analyzed the cloze items in BCT and CAP in the past 8 years, and Jonz’s. ‧. taxonomy was utilized as the instrument to classify the target test items.. y. Nat. er. io. sit. The analysis results of the present study are shown as follows: (1) the primary. a. n. v l C to 2017 was about language knowledge tested from 2010 n i the higher-order knowledge hengchi U. that focused on the ability to integrate the contextual clues across sentential level; (2) the most frequently-tested items in BCT was the syntactic knowledge in tense; and (3) the most frequently-measured item in CAP was the textual relationship in discourse. Based on the findings of this research, the researcher yielded suggestions for the future research and implications for teaching.. Key words: cloze test, Basic Competence Test, Comprehensive Assessment Program ix. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(10) 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat al. Ch. engchi. i Un. v. x. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(11) CHAPTER ONE INTRODUCTION Background and Motivation The Basic Competence Test for Junior High School Students (BCT, hereafter) was the main standardized entrance exam for every ninth-grade graduate to take from 1999 to 2013. In 2014, the Comprehensive Assessment Program for Junior High School Students (CAP, hereafter) replaced BCT as the new test type. The test result is of vital importance for it decides the future school to each graduate. Both BCT and CAP are the large-scale high-stakes national tests that require a large amount time of preparation for both test designers and test-takers. The test items have therefore received much attention.. 立. 政治大. ‧ 國. 學. To well understand BCT and CAP, it is essential to realize the purpose of both. ‧. national exams and the principles of test construction. According to the Research. sit. y. Nat. Center for Psychological and Educational Testing at National Taiwan Normal. er. io. University (RCPET, hereafter), the test construction of BCT and CAP are based on. al. n. iv. C which the Grade 1-9 Curriculum Guidelines in 2008. As for the Un h e n wash iannounced. gc. purpose of both exams, they are summative achievement tests which aim to evaluate learners’ academic acquisition after three-year junior high school education. The relationship between official policy and the national summative assessment is highly related to EFL teaching in Taiwan. The 12-year basic education policy have implemented since 2014, and it was the year BCT was replaced by CAP. There is a change of the difficulty level in CAP. According to RCPET (Appendix A), BCT is designed at the level of easy, and CAP is at the level of appropriate which means the proportion of the difficulty is equally distributed among the test. The shift of difficulty from BCT to CAP is a big impact to teaching pedagogy as well as material development and selection.. 1. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(12) BCT and CAP include two sections: single multiple choice questions and group questions, and the test type in both exams is multiple-choice question which is accompanied with four options. Group questions include two parts: reading comprehension test and cloze test. The section of group questions puts focus on evaluating the test takers’ reading ability. The reading articles in group questions are authentic and multi-constructed with dialogues, descriptions, poems and comics with different genres. From the sample questions in the past exams in CAP, we find that the reading passages were longer than BCT and the target cloze items were not a word or phrase anymore. The options included more words or even a sentence. Therefore, the test. 治. 政with complicated takers need to have the ability of dealing 大 information and inferring 立. ‧ 國. 學. between sentences to decide which answer is correct. However, the advanced ability. ‧. to comprehend the complicated information is not stated in the competence indicators. sit. y. Nat. of Grade 1-9 Curriculum Guidelines (Appendix B) at the stage of junior high school.. er. io. On the contrary, this advanced ability meets the competence of new Grade 1-12. al. n. iv. n C which Curriculum Guidelines (Appendix C) h e n is going h i U to implement in 2018.. gc. The cloze test is used to evaluate learners’ command of discourse structure or linguistic knowledge (Bachman 1982). The evaluation of discourse structure puts focus on the ability of searching for the correct answer beyond the sentential level to make the text coherent. The linguistic knowledge emphasizes the grammatical points of tense forms, referential determiners, and other syntactic elements. The cloze test thus is an integrative test because it involves several types of linguistic knowledge in making the text logical and reasonable. For different age of language learners, cloze test is one of the most difficult parts in reading test because it targets at integrated comprehension skills. Besides, it is used widely in many testing contexts in Taiwan. 2. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(13) which is from high-stakes exams to quizzes in class. Learners must pay more attention on the cloze items; otherwise, they cannot handle the complex tasks successfully. In Taiwan, some research analyzes cloze test in senior high schools. For the characteristics of the cloze procedure, Yang (1996) investigates what the principles the local teachers used to generate the cloze items in monthly exams. For the analysis of cloze items, Cheng (2007) categorizes cloze items of the main entrance exams to college, Scholastic Aptitude English Test (SAET, hereafter) and Department Required English Test (DRET, hereafter), into five cohesion types according to the classification of Halliday and Hasan (1967). Chou (2009) utilizes Jonz's (1990) taxonomy and a modified classification procedure to categorize items in a section. 治. 政" in SAET大 called "Zong-He-Ce-Yian" as "cloze test and DRET. However, little 立. ‧ 國. 學. research has been conducted in the field on the cloze test at the stage of junior high. ‧. school even though the cloze test is a crucial testing type whether in formal entrance. sit. y. Nat. exam or monthly exams at school.. er. io. The new test type of senior high school entrance exam, CAP, differs greatly from. al. n. iv. n C hclosely examined the previous test. The new test type is e n h i U by teachers, parents and. gc. students because it is a high-stakes test. The change shows an important perspective for the teachers in junior high school to analyze the language knowledge involved in cloze tests. The present study is thus motivated to investigate the cloze test for the better understanding of CAP, which in turn may cast light on teaching and learning. The major research questions addressed in the present study are as follows: 1. What kind of language knowledge is measured by cloze items of BCT and CAP from 2010 to 2017? 2. What are the differences between cloze items of BCT and CAP from 2010 to 2017?. 3. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(14) Significance of the Study Since the cloze test needs more integrated language ability for students, the significance of the present study can be illustrated in three aspects. The first is to discover the language knowledge on cloze test in senior high school entrance exams from 2010 to 2017. Secondly, the study aims at the characteristics of cloze items in both tests. The differences of cloze items between BCT and CAP will be compared in the present study. Lastly, for pedagogical application, since most teachers are more familiar with BCT and is still figuring out the content of CAP, the present study can give teachers some suggestions for their teaching activities in junior high school, and further design different lessons to address the important language knowledge to. 治. 政learning. enhance assistance in students' language 立. 大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 4. i Un. v. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(15) CHAPTER TWO LITERATURE REVIEW In this section, literature review on this topic is further divided into five parts. The first part is the relationship between testing and assessment in the curriculum design and approaches to language testing. The second part is the development of cloze test and the elaboration of four major types of cloze test. The third part is about what kind of language skill cloze tests measures. The fourth part is about the research on cloze test in Taiwan. The last part is the introduction of BCT and CAP. Testing and Assessment in Curriculum The well-constructed curriculum offers a proper framework to promote learning. 治. 政 efficiency. According to Nicholls and Nicholls (1972), 大 the stages of curriculum 立. ‧ 國. 學. design are (1) setting objectives, (2) devising methods and choosing materials, (3). ‧. conducting assessment and (4) giving feedback to the objectives. In order to achieve. sit. y. Nat. the defined goals, each stage interacts with one another. The curriculum design should. er. io. be a continuous cyclical process, which needs to be planned and modified constantly. n. al. C h(Tyler, 2010).U n based on feedbacks from assessments en hi. iv. gc. The role of testing and assessment is one major element for constructing an appropriate curriculum in teaching. In order to inspect students' learning as well as teachers' curriculum design, testing thus provides feedbacks to teachers for evaluating the effectiveness in teaching, examining the appropriateness of the textbook and reflecting the effectiveness of the syllabus. The test result thus provides clear information for teachers or designers to re-evaluate the objectives, teaching materials or teaching methods to modify the curricula. The relationship between curriculum and assessment is dynamic and interactive, and the assessment should not be parted from the language learning process (Brown, 1995; Brown, 2005: 252; Dick, Carey＆Carey, 2002: 2). Hughes (2003) stated that an. 5. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(16) accurate measurement of test takers' target skill is essential to a language classroom, as it provides formative and summative information about the learning progress and achievement, which in turn gives the instructor beneficial feedback in how and what to teach. The selection of test depends on different curriculum purpose. For inspecting students' progress in learning, formative assessment is a proper choice to evaluate their weakness and strength. Aside from providing the feedback for curriculum modification, student can also monitor their learning progress by formative assessment. Summative assessment, on the other hand, is used to measure what has been achieved by the end of a semester or a year which is called formal test (Hughes,2003).. 立. 政治大. ‧ 國. 學. Approaches to Language Testing. ‧. From the chronological order, language testing can be classified into four main. sit. y. Nat. approaches:(1) the essay-translation approach, (2) the structuralist approach, (3) the. er. io. integrative approach and (4) the communicative approach (Heaton, 1975).. al. n. iv. The essay-translation approachCishcommonly U n at pre-scientific stage in e n h i used. gc. language testing. The tests consist of essay writing, translation and grammatical analysis. The approach can apply to examinees of any level. However, the tests have a heavy literary and culture bias as well as the teachers’ subjective judgment. On the contrast, the behaviorism teaching approach during the period of the 1940s and 1950s is associated with the psychometric-structuralist in language testing. The structuralist approach views language learning as the systematic acquisition of a set of habits. It draws attention on a set of structure linguistics: phonology, vocabulary and grammar. The tests of four abilities (reading, writing, listening and speaking) are also separated from one another because testing one thing at a time is emphasized in. 6. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(17) this approach. The approach in language testing put focus on the mastery in separate elements or skills instead of integrated ability. However, having a good command of using language in a larger context cannot be measured through discrete-point test. The students are not able to have ability to communicate outside the classroom in the real world (Richards & Rodgers, 2003). Furthermore, it is found that there is no positive relationship between learners' performances on discrete-point test and their language ability in communication (Richards & Rodgers, 2003). Hence, in the 1970s and the 1980s, new language teaching methods start growing, and put emphasis on integrative language ability. This new era of language teaching method is called integrative-sociolinguistic. 治. 政integrative大approach, providing a real period (Spolsky, 1996). Starting from the 立. ‧ 國. 學. context for the communication purpose is the main characteristics in this period. The. ‧. focus of testing shifted on the testing of language in context with communicative. sit. y. Nat. effect. The focus is different from the previous approach that put emphasis on a. er. io. discrete-point item. A discrete-point test focuses on one grammatical point at a time,. al. n. iv. n whereas the integrative test measuresCmore h e n thanhone i Upoint at a time. (Oller, 1979;. gc. Hadley, 2000; Hughes, 2003). In this approach, the language testing techniques are designed to increase learners’ contextualized knowledge. The test designers aim to test learner's ability by utilizing different aspects of language knowledge instead of isolating pieces. The testing formats also followed this principle to change. Many scholars pointed out the cloze tests and dictations were considered as the two types of integrative tests (Oller, 1979; Bailey,1998; Brown, 2001; Brown, 2005). Lastly, the communicative approach is focus on how language is performed in communication. In this approach, most tasks are related to students’ real life, and the effectiveness of communication is the target purpose instead of the language forms.. 7. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(18) The measurement of communicative approach is based on the concept of divisibility hypothesis in language. The learners thus have several profiles to measure the different language performances. The Development of Cloze Test This section addressed the development of the cloze test, including three parts: measurement of the readability, measurement of leaners’ reading comprehension, and types of modified cloze test. Measurement of the Readability The readability of a text affects the difficulty of a test. The readability means the difficulty level of a reading passage. The readability of a passage can be measured by. 治. 政 vocabulary sentence length, the complexity of sentences, 大 difficulty, and others. Dale 立. ‧ 國. 學. and Chall (1948) and the Flesh (1948) are the two most widely used formulas.. ‧. However, Taylor (1953) proposed the cloze procedure because he thought that the. sit. y. Nat. previous two formulas used to measure the readability of a prose that dealt with some. er. io. superficial aspects of the text. The cloze procedure integrates most of the potential. n. al. n C h of a prose. elements that may influence the readability en hi U. iv. gc. The invention of the cloze procedure opens a crucial page of language testing, that is, the total language context. The total language context indicates that "the ability to identify, learn and produce any language symbols depends heavily on the variable degrees to which it is associated with everything else by larger and meaningful overall combinations " (Taylor, 1953, p. 418). The performance in a cloze passage represents learners’ understanding of the text. Oller (1979) suggests that teachers can select the appropriate level of cloze passages for learners based on their current level. Generally, two common methods are used to score a cloze passage which represent learners' understanding of a specific passage: The exact word method and the acceptable word method. The exact word method is. 8. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(19) that the test takers need to provide the exact word as the original word from the original passage. Conversely, the acceptable word method indicates that any response which is grammatically correct and make sense in the context is regarded as correct answers. Measurement of the Reading Comprehension Reading comprehension is treated as an important ability in language learning. The process of reading comprehension covers all the meaningful elements of a texts that are integrated into a coherent whole during text process (Kintch and van Dijk ,1978). The reading ability involves organizing and arranging the written information to get the whole pictures of the texts. The reading comprehension occurs. 治. 政 and integrate when the readers make sense out of passages, separately presented ideas 大立. ‧ 國. 學. into a unitary whole (Koler, 1973; Wittrock, 1975). Reading comprehension test is. ‧. used widely from class quizzes to high-stakes examination. There are numerous ways. y. sit. er. io. questions (Heaton, 1998).. Nat. of testing reading comprehension, ranging from multiple-choice items to open-ended. al. n. iv. n Since the result of the cloze testCcan h ebe h i Uas the measurement to present n utilized. gc. learners’ understanding of a passage, researchers have been interested in the relationship between cloze test and reading comprehension test. Bormuth (1969) conducted a study for 4th to 6thgrade students. The nine reading passages with readability index between 4.5 to 6.6. of Dale＆Chall’s formula were adopted and modified into a cloze test. At the same time, seven multiple choices are provided to measure learners' reading comprehension. The study shows the results of the cloze test are highly related with the reading comprehension test. Dupuis (1980) also have a similar result for the tenth-grade students (senior high school level) in the study. Dupuis intended to investigate whether cloze procedure is also an effective instrument in measuring learners' comprehension in different genres.. 9. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(20) She selected two short stories and one novel, and adopted the cloze test by every nth word deletion to construct a cloze passage. Each passage was modified into a cloze passage accompanied with some reading comprehension questions. The study reveals a highly-connected relationship between the cloze test and the reading comprehension test in both genres. In Taiwan, Huang (1992) conducted a study to compare the result of the two types cloze tests with reading comprehension in TOEFL. The cloze procedure of two types cloze tests in this study were the C-test and multiple choice. The study revealed that both cloze tests effectively predicted learners’ performance in reading comprehension.. 治. The relationship between cloze test政 and reading 大 comprehension test is highly. 立. ‧ 國. 學. related as shown in the above studies. The similar conclusion provides a notion that. ‧. cloze tests can be used to test both micro and macro-level language skills. However,. sit. y. Nat. in Greene’s study (2001) of evaluating the reasoning ability by a cloze passage with. er. io. deleted discourse markers, he concluded that cloze test is a reliable tool to evaluate. al. n. iv. n C hwhen the focus learners’ reasoning ability of a passage e n h i U is on the connection between. gc. sentences, but it cannot be used to evaluate learners content knowledge in reading comprehension of a passage. Alderson (2005) made a similar conclusion that cloze procedure is word-based, it thus may not evaluate some reading skills. Types of Modified Cloze Tests Bailey (1998) pointed out “the process of deleting word to create a cloze is known as mutilation” (p.62). According to the way how a cloze item is generated, four general types of cloze tests are commonly used: fixed-ratio deletion, the C-test, the rational deletion and multiple choice. The following section will present the introduction of the four general types of the cloze test.. 10. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(21) The first type of cloze test is called fixed-ratio deletion (nth word deletion). Every nth word is deleted no matter what word it is in a selected passage. The procedure tends to test various items types, including the governed constraints in local grammatical knowledge and long-range textual ones. The advantage of the fixed ratio deletion is that the test constructors can easily and objectively generate cloze items; however, the items may be either too difficult or meaningless to the learners. Also, the different deletion frequencies (6th, 8th or 10th) of the same passage may produce different difficulty levels of the cloze test (Alderson, 1979, 1980). The second type is C-test. Two types of C-test have been identified. The first type of C-test is a type of language test in which the students read a brief paragraph in. 治. 政 the target language with the first two sentences are left 大 intact. Starting from the third 立. ‧ 國. 學. sentence, the individual letters of every other word are replaced by the blanks (Bailey,. ‧. 1998). Another type of C-test keeps the first sentence remain intact only, and the. sit. y. Nat. second half of every second word is deleted from the second sentence. The. er. io. advantages of the C-test are that only exact scoring is necessary and it possibly covers. al. n. iv. n the other hand, the C h of ability.UOn a wider range of topics, styles, and levels en hi. gc. disadvantage of C-test is that it is harder to read than a cloze passage, and the correct answers can often be found in the surrounding text (Hughes,2003). The third type is the rational deletion that is also called selective deletion, which means the words are deleted based on some rational decisions. Since the fixed-ratio deletion does not test on target item in language, the rational deletion suggests a new test type for test developers to control over the types of words deletion to measure the target language ability. Rational cloze tests are as " professionally developed" tests in general intend to measure high-order reading processing by fine-tuning the items (Bensoussan, 1990; Storey, 1997).. 11. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(22) Bachman’s study (1982) suggested that “cloze tests are not necessarily monotonic tests of general and specific traits.” The result showed that rational cloze was capable of measuring higher order skills: cohesion and coherence in the text. In Bachman's study in 1985, he conducted another study about the proportions of different cloze items types deleted by fixed-ratio deletion and rational deletion. Bachman developed a classification framework for cloze item types according to the different required level of the context to fill in the blank, which are (1) Within Clause, (2) Across Clause, Within Sentence, (3) Across Sentence, Within Text, and (4) Extratextual. The result revealed that fixed-ratio deletion tends to produce items of (1) Within Clause and (4) Extratextual. The two types refer to the blanks that can be. 治. 政 filled in simply by using clause-level grammatical knowledge or world /extra textual 大立. ‧ 國. 學. knowledge. Contrast to fixed-ratio deletion, the rational deletion produces more items. ‧. of (2) Across Clause, Within Sentence, and (3) Across Sentence, Within Text. Types (2). al. er. io. provided by the text beyond clause or sentence.. sit. y. Nat. and (3) indicate that the test-takers are required to fill in the items by using the clues. n. iv. n C hstudy in 1990. A similar result is found in Jonz’s e n h i U He applied Bachman's (1985). gc. four-category analytical approach on seven cloze passages. Figure 2.1 presents the flowchart of Jonz’s interpretation about Bachman’s four categories.. 12. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(23) Can closure be affected if the clause is isolated from the text? No. Category 1 "Within Clause". Yes. No Can closure be resolved with support of textualYes clause outside the clause?. Category 4 "Extratextual". No. Yes Is the clue within the same orthographic sentence?. Yes. No Category 3 "Across Sentences, Within Text ". Category 2 "Across Clause, Within Sentence". 立. 政治大. ‧. ‧ 國. 學 sit. y. Nat. n. al. er. io. Figure 2.1 The Flowchart of Jonz's Interpretation of Bachman's Categories. iv. C h Clause into He further divided the Category 1 - Within U n two subcategories: (a). engchi. Clause-level lexis, and (b) Clause-level syntax. He proposed that the division is to uncover the evidence that cloze tests measure both the high-order (lexical choice) and low-order (syntactic knowledge) comprehension. Besides, he proposed a classification key of the subtypes under the five categories. The classification key provides a precise source to classify the cloze items. The present study adopts Jonz’s framework to analysis the cloze items in BCT and CAP from 2010 to 2017, and the complete classification will be presented in chapter 3. The previous three types of cloze test require test-takers to fill in the blanks, in terms of scoring, two methods are adopted: the exact word and the acceptable-word. Comparing these two methods, the exact word is easier for scoring; however, it may. 13. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(24) not actually evaluate student’s performance in using the target language. On the other hand, the acceptable-word method provides more opportunities for learners’ creativity in using the language, but it increases the difficulty and time of scoring. From the disadvantages of above scoring methods, multiple-choice cloze test becomes a popular test format, that is each cloze item is followed by four alternative options, in many formal and large-scale examinations because of the easy scoring. The fourth modified test type is multiple-choice cloze test. Panackal and Heft (1978) compared learners' performance on Scholastic Aptitude Test (SAT, hereafter) by open-ended cloze test and multiple-choice cloze test. The result reveals that both test types are comparable to SAT, and multiple-choice cloze test can be used to replace the open-ended cloze test.. 立. 政治大. ‧ 國. 學. Hinofotis and Snow (1978) used 25 multiple choice items and 25 open-ended. ‧. cloze test items to investigate whether multiple choice cloze test work equally well as. sit. y. Nat. open-ended cloze test. The study showed that multiple choice items are easier in terms. er. io. of item facility indices, and open-ended cloze test is better in terms of item. al. n. iv. n C h that even there discrimination indices. They concluded e n h i U are different performances on. gc. both tests, the multiple choice is a more promising evaluative tool because of ease of scoring. Although the multiple choice cloze test is relatively easy and convenient in scoring, constructing a multiple choice cloze test is more complicated than constructing an open-ended cloze test (Hinofotis & Snow, 1978; Wu, 2002). Wu (2002) pointed out the deletion and alternatives have an effect on the test, and the limitation of the alternatives affect test takers' comprehension processing. In order to construct a multiple choice cloze test with validity, it is necessary to check item facility and discrimination indices and carry out the pretest (Hinofotis & Snow, 1978). By doing so, the improper cloze items should be discriminated and modified.. 14. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(25) What Kind of Language Skill Does Cloze Measure? Cloze tests are widely used for measuring text comprehension (Sharp, 2009; Schmitt & Sha, 2009; Miller, Dewitt, McCleeary & O'Keefe, 2009), proficiency (Keshavarz & Salimi, 2007), collocational knowledge (Stuart & Eve, 2009; Keshavarz & Salimi, 2007), as a test of translation (Ito, 2004), and a pedagogical tool (Dastjerdi & Talebinezhad, 2006; Lee, 2008). The cloze procedure has been long studied in the divergent opinion about what kind of language ability the cloze procedure measure; whether it measures learners' higher or lower-order skills. Some argued learner’s overall language proficiency is able to be tested and obtained from the learning result by the cloze test (Chavez-Oller. 治. 政 On the 大 et all, 1994; Bachman, 1982; Sasaki, 2000). other hand, some argued the cloze 立. ‧ 國. 學. test can only test local language skills because of the insensitivity (Alderson, 1979;. ‧. Shanahan, 1982; Markman, 1985).. sit. y. Nat. Chavez-Oller et al. (1994) found that the cloze items are sensitive to long-range. er. io. constraints, and the constraints across sentence boundaries facilitate performances on. al. n. iv. n C h to the clues some cloze items. The items which refer e n h i U beyond sentential level can. gc. discriminate high achievers from low achievers. Bachman (1982) proposes that the cloze test items can be classifies into three traits: syntactic, cohesive and strategic. The syntactic trait which depended on clause-level. The cohesive trait means the closure is on interclausal and intersentential cohesive structure (as described by Halliday and Hasan in 1976). The strategic trait refers to “long-range” patterns of coherence. The study indicates that the rational deletion can be used to test on specific abilities and measure the higher-order skills. Sasaki (2000) investigated the effects of cultural schemata on test-taking processes of cloze tests and used immediate retrospective introspection as one of the. 15. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(26) data sources. The study revealed that participants did use information beyond the sentence level to answer. However, some studies suggested that cloze test is not able to test learners’ integrative ability, it only tests local syntactic skills. The changes of the deletion rates also change the measurement in different abilities. Shanahan et al. (1982) conducted a study to examine whether the cloze passages are able to measure learners’ ability from the information across sentence boundaries. The researchers deleted every 5th word in the passage, and used sequential and scramble cloze passages. Besides, they added one more type of cloze passage in the study: they inserted a single sentence into the original cloze passage. They assumed that the learners could not use the. 治. information across sentence boundaries政 under the three 大 types of cloze passages. The. 立. ‧ 國. 學. result revealed that there are no differences in learners’ performances. The test takers. ‧. who have only a single sentence also have good performances like those have the. y. sit. Nat. same sentence embedded in meaningful context.. er. io. Markman (1985) adopted the fixed ratio deletion to examine the intersentential. al. n. iv. th th n C h procedure. U sensitivity of the rational deletion cloze e n h i He deleted every 7 and 9. gc. word in both sequential and scramble texts, and placed adjacent sentences in nonadjacent position to destroy the intersententail ties in scramble texts. The result shows there is no differences in performance on both cloze tasks. A similar conclusion in the above studies claim that the cloze procedure measure the lower-order skills. In the fixed-ratio deletion, the test constructors are not able to control the test items in specific abilities. In this way, only the cloze items at clausal and sentential level can be generate. Therefore, Alderson (1980) suggested that if test constructors carefully design the deletion closure and make sure the clues for the cloze items are beyond sentences, the cloze test is capable of measuring language knowledge across sentence boundaries.. 16. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(27) Relevant Research on Cloze Test in Taiwan The cloze test has been adopted in various contexts, which is from monthly exams to high-stake tests, and some research addressed the topic on cloze tests in Taiwan (Yang,1996; Huang, 1997; Change, 1994; Chou,2009 ). Yang (1996) analyzed the cloze items in monthly exams of secondary school, and interviewed with local teachers about the principles they use to generate cloze items. He indicated that the cloze tests have often been misused in high schools in Taiwan because "most teachers are not aware of the true characteristics of the cloze procedure and tend to interpret it as gap-filling in a continued discourse without concerning themselves with contextual dependence of test items" (p.62). In Yang's. 治. 政 cloze tests大tend to serve as discrete point study, he found that many multiple-choice 立. ‧ 國. 學. test rather than integrative test because too many local items are tested instead of. ‧. cross-sentential contextual clues.. sit. y. Nat. Huang (1997) studied the cloze test items of Joint College Entrance Exam (JCEE). er. io. from the year 1982 to 1993 by a qualitative analysis. He divided cloze items into two. al. n. iv. n item could be restored within C h item meansUthe types: global and local items. The local en hi. gc. clause or sentence level. The global item indicates that the contextual clues for the cloze item are beyond sentence. The result revealed that global items only accounted for 40 (16.33%) out 245 items. JCEE put much emphasis on local items, and the modified cloze test is regarded as discrete-point test. Change (1994) found the similar conclusion that the cloze items in Taiwan focus more on local items. He suggested test constructors should balance both local and global items to measure learners’ overall language ability. Chou (2009) applied Jonz's taxonomy to evaluate the cloze items of the two college entrance exams in the section of Zong-He-Ce-Yian in SAET and DRET from 2004 to 2008. The finding of the study reveals that 78% and 69% of the. 17. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(28) Zong-He-Ce-Yian in SAET and DRET are classified as local items, and 22% and 31% in SAET and DRET are classified as global items (pp.79-80). These two nation-wide exams tend to measure examinees' discrete-point skills rather than integrative skills (p.85). To balance the cloze items on local and global skills in cloze test, test constructors can use some criteria to check. Wu (2002) classified item generation approaches into three types: syntactic-, lexical, and discourse-oriented approaches. The previous two approaches are considered as items for local skills, and discourse-oriented approach is considered as global items. Besides, Bachman’s taxonomy (1985) can be used for evaluating whether a cloze. 治. 政 oriented. item is discrete-point oriented or integrative 大Yang (1996) pointed out that (1) 立. ‧ 國. 學. Within clause and (4) Extratextual are associated with discrete-point skills because. ‧. (1) Within clause deals with contextual clues only within clause level, and. sit. y. Nat. (4) Extratextual requires the knowledge without inferring from the text. Types. er. io. (2) Across Clause, Within Sentence and (3) Across Sentence, Within Text deal with. al. n. iv. n C h skills. In most interclausal or intersentential integrative e n h i U local context, the. gc. multiple-choice cloze test and the procedure of rational deletion are adopted, the test constructors should be more aware of what kind of the language knowledge in cloze items aim to measure as well as the balanced distribution of the measurement in global and local language knowledge. BCT and CAP In Taiwan, the main standardized entrance exam of senior high school for ninth-grade students from 1999 to 2012 was BCT. Starting form 2014, CAP is the new test type. The goals of academic attainment testing construction of BCT and CAP are according to the competence indicators in Grade 1-9 Curriculum Guidelines which is related to the English learning objectives within 7th to 9th grade. The format of. 18. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(29) reading test in these two exams consists of two parts: single multiple choice questions and group questions. Group questions include reading comprehension test and cloze test. The cloze test is conducted by rational deletion in multiple choice questions with four options in both exams. The test genres for cloze test include written passages, dialogues, ads, chats, and other authentic ones. BCT BCT was a main entrance exam of senior high school from 1999 to 2013 for junior graduates. From 1999 to 2011, BCT had been conducted twice a year in May and July. Starting from 2012 to 2013, BCT was changed to be conducted only once in a year in June. BCT is designed at the level of easy according to the official. 治. 政 announcement of Research Center for Psychological 大and Educational Testing. The 立. ‧ 國. 學. result of the test performance is counted in the way of norm-reference and presented. ‧. by the scale scores. The total scale scores of BCT is 412 which include 80 scores in. y. sit. er. io. in composition from 2007.. Nat. each five subjects from 2009 (before 2009, 60 scores in each subjects) and 12 scores. al. n. iv. n C h for 70 minutes. There are 45 test questions in total e n h i U The design of the item types. gc. in BCT are the multiple-choice questions with four options. There are 10 or 11 group questions designed in the standardized exam. The group questions are arranged by reading comprehension test and cloze test randomly. The number of the cloze test of group questions are mainly constructed by 2 passages with totally 6 or 7 cloze items in the test each year. CAP In 2014, BCT was replaced by the new test type CAP. Before the formal practice of CAP, a sample test was conducted in 2013. According to RCPET, the objectives of the CAP are as follows: (1) CAP mainly focuses on basic, key and important concepts. 19. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(30) of English, (2) the various genres and topics are included; (3) the emphasis is focus on natural English language usage instead of complicated and fragmental memory. For the degree of difficulty, CAP is at the level of appropriate which means the proportion of four difficulty levels is equally distributed among the test. The result of the test performance is counted in the way of criterion-reference and presented by three levels: mastery (A++, A+ and A), basic (B++, B+ and B) and improvement needed (C). Besides, the English test contains two parts in CAP, listening test and reading test, with a total 60 of multiple choice questions. From the sample test in 2013 to formal test in 2014, these two parts are combined into one same exam paper, and the. 治. 政 from 2015,大the listening test and reading test time is 80 minutes. However, starting 立. ‧ 國. 學. test are separated into two independent test sections which are presented in two. ‧. different exam papers. The test time for listening test is 25 minutes for 20 multiple. al. er. io. multiple choice questions with four options.. sit. y. Nat. choice questions with three options, and for the reading test is 60 minutes for 40. n. iv. n C h from BCT and Above all, there are some changes e n h i U CAP, and the differences. gc. certainly bring some impacts on teaching and learning. The current study is thus motivated to investigate what kind of language knowledge of both exams aim to measure by adopting Jonz’s taxonomy. Furthermore, the current study also analyzes the differences of items types in the cloze tests of BCT and CAP from 2010 to 2017.. 20. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(31) CHAPTER THREE METHODOLOGY The present study aims to analyze and observe what kind of the language abilities measured in the cloze test section of the two national senior high school entrance exams in Taiwan, BCT and CAP. Jonz’ taxonomy (1990) is adopted as the instrument to classified the two types of materials into five categories, and then each cloze item is classified into one subtype under each category according to their test targets. In this chapter, the materials, instrument and procedure of data analysis are presented. Materials. 治. The materials in the present study政 are collected大 from BCT and CAP. The test. 立. ‧ 國. 學. items are obtained from the website of Comprehensive Assessment Program for. ‧. Junior High School Students (http://cap.ntnu.edu.tw). The data of both exams include. sit. y. Nat. the test items in the past 8 years. The current study will analyze the cloze items from. er. io. 2010 to 2013 in BCT, and analyze the cloze items from 2014 to 2017 in CAP. Table. al. n. iv. n Ch 3.1 presents the number of reading passages i Unumber of test items in BCT and e n andh the. gc. CAP from 2010 to 2017.. 21. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(32) Table 3. 1 Distribution of Reading Passages and Test Items in Cloze Test Data. Number of Reading Passages. Number of Test Items. 2010-1 BCT. 2. 6. 2010-2 BCT. 2. 7. 2011-1 BCT. 2. 6. 2011-2 BCT. 2. 7. 2012 BCT. 2. 6. 2013 BCT. 2. 6. 2014 CAP. 2. 7. 2015 CAP. 2. 6. 2016 CAP. 2. 7. 2017 CAP. 2. 8. Total. 20. 66. Year (1st or 2nd). 立. 政治大. Instrument. ‧ 國. 學. The present study aims to find what kind of language knowledge is measured in. ‧. the standardized national entrance exams of senior high school in Taiwan. The. y. Nat. n. al. er. io cloze items in BCT and CAP.. sit. researcher applies the taxonomy in Jonz’s study in 1990 to analyze the features of. Ch. engchi. i Un. v. In 1990, Jonz analyzes cloze passages based on Bachman's framework (1985) to categorize cloze items into four types according to the level of context required for recovering the missing word. However, Bachman’s study does not provide further explanation for how and why each item is assigned to a specific category. Therefore, Jonz develops his operationalization rationale for the interpretation of cloze deletions of Bachman’s categories. To have a deeper understanding of the test items, Jonz (1990) further divides two levels under Category 1-Within Clause: Clause-level syntax and Clause-level lexis. Jonz proposes an assumption that the lexical choice represents the high-order knowledge, and the syntactic knowledge represents the low-order one (p.70).. 22. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(33) The present study adopts Jonz’s model that includes two levels in Category 1 to collect more data for better understanding of the cloze items in local context. Hence, each cloze test item will be assigned into 5 categories in the current study: (1) Clause-Syntax, (2) Clause-Lexis, (3) Across Clause, Within Sentence, (4) Across Sentence, Within Text, and (5) Extratextual (C1, C2, C3, C4, C5 hereafter). Furthermore, Jonz lists several subtypes of grammatical structure under the five categories. The subtypes under each category provide not only the more precise source to classify the language elements that the cloze items target at but the better understanding on what kind of language knowledge is measured (pp.76-77). Table 3.2 presents the categorization of each subtype provided by Jonz.. 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 23. i Un. v. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(34) Table 3. 2 Jonz's (1990) Subtypes of Grammatical Points Tested in Cloze Items (pp.76-77) Clause-level Syntax 1-S-a. determiner without text antecedent. 1-S-b. conjunction below clause level. 1-S-c. role-marking preposition. 1-S-d. relative word in headless clause. 1-S-e. Complementizer. 1-S-f. auxiliary. 1-S-g 1-S-h. negation copula BE＆other intensive relationships. 1-S-i. syntactic there or syntactic it＋Be. 1-S-j. syntactic elements in cleft or pseudocleft. Clause-level Lexis 1-L-a 1-L-b. referential determiner or deictic within clause 政治大 pre-＆post determiners. 1-L-c. pronoun (coreferent with clause). 1-L-d. indefinite proform (including you as indefinite). 1-L-e. preposition other than role-marker. 1-L-f. collocation with clause. 1-L-g. reiteration with clause. 1-L-h. multipart item al Ch qualifier in AdjP＆AdvP. 立. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. 1-L-i. Across Clause, Within Sentence. engchi. i Un. v. 2-a. referential determiner or deictic across clauses. 2-b. conjunction at clause level. 2-c. pronoun (coreferent across clauses). 2-d. relative word in headed clause. 2-e. collocation across clauses. 2-f. reiteration across clauses. 2-g. subordinator. 2-h. •. it of extraposition. Across Sentence, Within Text 3-a. referential determiner or deictic across sentences. 3-b. conjunction between sentences. 3-c. pronoun (coreferent across sentences). 3-d. collocation across sentences. 24. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(35) 3-e. reiteration across sentences. 3-f. textual discourse marker. Extratextual 4-a. speech-role pronoun without textual antecedent. 4-b. lexical item with little or no local clue Most subtypes in Jonz’s taxonomy are covered in grammar books, like pronoun,. auxiliary and negation. The following sections are the further explanations of some subtypes that students are not so familiar with when they are learning English, which are: 1-S-e complementizer, 1-S-h copula BE＆other intensive relationships, 1-L-h multipart item, 2-h it of extraposition ,and 3-f textual discourse marker. 1-S-e: complementizer The complementizer within a clause includes two types: subject complement. 政治大. and object complement. Subject complement is a word or phrase that follows a 立. ‧ 國. 學. linking verb to describe or identify the subject. The linking verbs are like “am, is, are,. ‧. appear, look, smell, and others.” A subject complement could be an adjective, a noun,. y. Nat. n. al. er. io. (1) That pie looks delicious.. sit. an adverb, a pronoun, or others.. Ch. (2) Mosquitoes are everywhere.. engchi. i Un. v. In Example 1, “delicious” is the complementizer to describe how that pie looks. In Example 2, “everywhere” is an adverb that identifies where the mosquitoes are. On the other hand, object complement is utilized to describe or identify the object. (3) The President appointed her Secretary of State. In Example 3, “her” is the object of the transitive verb “appoint.” In addition to “her”, we need a word to complete the meaning of the verb. The noun “Secretary of State” completes the meaning of the clause and functions as a complement to the object “her.”. 25. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(36) 1-S-h: copula BE＆other intensive relationships The intensive relationship means to focus on the subject, and the following intensive verbs serves the function as the subject complement. (4) Jenny is a doctor. (5) Jason looks very young for his age. In the above examples 4 and 5, the usage of copula be and intensive relationship, such as “am” and “look” function as the join of an adjective or a noun to each subject. In Example 4, the focus of the sentence is Jenny and what she is. In Example 5, the focus of the sentence is Jason and how he looks. Copular verbs, such as “taste, look, smell” also known as the linking verb to play the role of subject complement. 1-L-h: multipart item. 立. 政治大. ‧ 國. 學. A multipart item refers to the items which contain more than one, such as phrasal. ‧. verbs, “make up, turn on, take off.” A phrasal verb includes a simple verb plus a. sit. y. Nat. particle, or in rare cases two particles, but it must be considered a single lexical item.. er. io. (6) He took off his rain coat.. n. al. C hin 5 minutes.U n (7) The plane is going to take off en hi. iv. gc. (8) He took two weeks off in December to have a vacation in France. In the above three examples, each sentence contains the same phrase verb “take off”; however, there are three different meanings that learners should comprehend depend on the other components in the sentence. 2-h: it of extraposition This subtype refers to a construction in which a clause that acts as a subject is moved or extraposed to the end of the sentence, and replaced by dummy “it” in the original position. (9) It is true that you have been misled. (10) It is exciting to watch the baseball game at baseball field in person.. 26. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(37) The subjects in examples 9 and 10 are replaced by dummy “It”, and the following “that- clause” and infinitive “to” bring out the complete meaning of the two subjects. By doing so, the writer can avoid a problem of having a heavy subject in writing. 3-f: textual discourse marker Discourse markers are words or phrases used to show how discourse is constructed. They connect and organize what we say or write, or to express attitude. (11) A: I like Maggie. She is adorable. B: So, you’ll ask her out then. (12) My father promised to have a family trip this summer. However, he changed his mind last week. In Example 11, the expression “So” shows what is said is the result of what was said. 治. 政consequence. before. It serves the function as a logical 大 In Example 12, the discourse 立. ‧ 國. 學. marker “however” is used to begin a new sentence which contracts two ideas.. ‧. The other subtypes of Jonz’ taxonomy are covered by grammar courses at the. sit. y. Nat. stage of junior high school; for example, determiners “the” and “that”, conjunction. er. io. “or” and “but”, negation “no” and “not”. In addition, learners are also required to. al. n. iv. master the principles of basic tensesCfrom U n at this stage; for instance, h e nEnglish h i courses. gc. present simple tense, past tense, future tense, present and past progressive, and present and past perfect tense. Modification of Jonz's Subcategories Jonz (1990) analyzes eight cloze passages that are developed either by fixed-ratio or rational deletion. When the cloze passages are applied on the fixed-ratio deletion, every nth word is deleted. However, BCT and CAP adopt the procedure of rational deletion which is different from Jonz’s. As a result, it may cast a problem that not each cloze item of BCT and CAP can be classified properly into Jonz’s taxonomy. The rational deletion is adopted in BCT and CAP, and the cloze tests in both exams are designed as multiple choice questions, that is, the test-takers must choose. 27. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(38) the correct answer from four options. Therefore, the test developers are able to control the types and elements of the word/ sentence deletion to test the target points. On the contrary, the standard cloze procedure requires the test-takers to fill in the blanks by themselves. As the case stands, some cloze items in Jonz’s subtypes cannot be found in BCT and CAP if they do not represent critical language knowledge. The test constructors in local context may also delete more than one word or even a complete sentence in an item. Therefore, the researcher makes six modifications from Jonz’s taxonomy in the present study. The first modification is to distinguish 2-b: conjunction at clause level from 2-g: subordinator. In general, conjunction includes three subtypes: coordinating,. 治. 政 conjunctions subordinating and correlative. Coordinating 大 connect two equal units of a 立. ‧ 國. 學. sentence together, such as “and, but, nor, yet. On the other hand, subordinating. ‧. conjunctions connect two unequal parts of a sentence: main clause and subordinating. al. er. io. following example shows two parts of a sentence.. sit. y. Nat. clause. The subordinating conjunctions are like “before, after, when, because.” The. n. iv. n C h an umbrella (13) I got wet because I didn’t bring e n h i U with me this morning.. gc. In Example 13, “I got wet” is the main clause and “because” connects the subordinating clause “I didn’t bring an umbrella with me this morning.” Two unequal parts conveys the relation of cause and effect. The third one is correlative conjunctions which are always used in pairs, such like “both/and, neither/ nor and not only/ but also”, often tested at graders 7 to 9. They join similar important sentence elements, so they are classified into coordinating conjunction in the current study. Therefore, 2-b: conjunction at clause level is modified as 2-b: coordinator, and 2-g: subordinator remains the same. The second modification is in Category 5: Extratextual. According to Jonz's taxonomy, Category 5 is known as Extratextual, and two subtypes are provided under. 28. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(39) this category: (1) speech-role pronoun without textual antecedent, (2) lexical item with little or no local clue. (14) He had to do a lot of _THINGS_ he didn't like to do _JUST_ because he was going to go away to college. (From Jonz, 1990, p.78) In Example 14, the test takers need to fill up the blanks, and the test item is categorized under the second type, indicating lexical item with little or no clue. However, as the cloze procedure is applied as multiple choices in BCT and CAP, the test-takers do not have to fill in the answers by themselves. Therefore, the adjustment in the category of Extratextual of Jonz’s study in the standardized cloze procedure has to be made.. 政治大 For the SLA learners, in order 立 to pursuit the goal of authenticity in language ‧ 國. 學. testing, some test formats like charts, ads, time tables, and maps in daily life are also. ‧. provided for cloze tests design. The test-takers need to acquire the ability of. Nat. er. io. sit. y. understanding the maps to find out the correct direction, or have the ability of taking. n. an order from a menu at a restaurant. a One example of ads v from BCT is presented as follows:. i l C hengchi Un. 29. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(40) 立. 政治大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. (15) Mr. Lee: Wow! You look great in _25_ glasses! Its glasses are very famous. How much is this pair? Mr. Kao: Only 500 dollars. a l did you get the glasses? Mr. Lee: Really? Where iv n Ch i Uhaving a sale these days. Mr. Kao: At B.B DepartmenteStore. n g c hIt’s ＊ 25.(A) Glass House (B) O.L.S. (C) Rian’s (D) NUMA (From BCT 2011-1st) The cloze test in Example 15 is presented by a format of a written dialogue with extra textual of four ads. From the dialogue, there are two interlocutors discussing some shopping items and prizes. For Question 25, the clue provided is beyond the sentential level, that is, the examinees must understand the following sentences accompanied with the ads below to choose the correct answer. Therefore, when the genre design includes two parts: written passage and picture (ads, charts, maps and others), the category is categorized into the modified Category 5– Extratextual. In addition, there are only some key words in adds or charts, and they don’t have the continuous text. 30. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(41) and contextual clues. Hence, the subtype 4-b: lexical item with little or no local clue is modified as 4-b: lexical item with little or no clues from the text in the present study. The third modification is to add one more subtype to Category 1. The application of tense is widely tested in local testing context which ranges from classroom quizzes to nationwide senior entrance examination in BCT and CAP. The usage of tense requires the test-takers to select the correct tense based on the clues provided within a clause or across clauses. Below is an example from BCT: (16). ‧ 國. 學. In the old times, people in the town _18_ their houses on both sides of the river. The houses in each block shared a public washing room. On Mondays, people came and washed their clothes together there. They hung their clothes 政治大 at the back of each house. ＊立 18. (A) build (B) are building (C) have built (D) built (From BCT 2010-2) In Example 16, Question 18 is provided with the clue “In the old times” within the. ‧. sit. y. Nat. same clause. Based on the clue, test-takers are able to choose the most appropriate. er. io. tense – past tense – for the item. In this case, the tense of the item is decided at the. n. a. v. i clause level of Category 1: Clausel –Clevel Syntax. However, there is no proper Un. hengchi. subtype among Category 1 for the option of tense. On the other hand, the lexis level part of Category 2: 1-L-a referential determiner or deictic within clause involves the identification of the relation between elements within a clause, like the first speaker and the interlocutor, the time and the space (here and now). Therefore, 1-L-a referential determiner or deictic with clause, is moved to Category 1 under the level of syntax in the current study. The fourth modification is to re-categorize about some subtypes C1- syntax and C2 - lexis. The subtypes which is associated to grammar knowledge like “pronoun” and “referential determiner or deictic within clause” are moved to C1. To have more. 31. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(42) clear analysis of the cloze items in C2, the subtypes which relate to lexical usage, such as “collocation” and “reiteration” remain in the level of lexis. The fifth modification is made in C4. In attempt to relate leaning to real communication, it is crucial for learners to have more appropriate language performances in authentic situation. The test constructors not only include written passages but daily dialogues in cloze tests. The conversational cloze test is a comprehensive tool to evaluate examinees' knowledge in pragmatics. One example of pragmatics from BCT is presented below: (17). ‧. ‧ 國. 學. Gisele: My dad’s taking me to play the Dye-out game this Saturday. He asked if you would like to come. Opal: _ 22_. 治 The game is played by two teams Gisele: It’s a popular game these 政 days. 大 and can be played 立 inside or outside. You win the game by taking the other team’s castle or “killing” all the players on the other team. 22. (A) But I’m not good at it (B) Your Dad has already invited me ＊ (C) But I’ve never heard of it before (D) I’d love to join you this Saturday (From BCT 2013). sit. y. Nat. er. io. The clue of Question 22 is provided by the following sentences of Gisele’s clear. al. n. iv. explanation about the Dye-out game.CAs U n Opal elicits clarification by h ethe caseishows,. ngch. showing a statement of uncertainty. In this case, the cloze item is related to the knowledge of speech act in pragmatics; that is the statement functions as a requirement for further information about the Dye-out game in the conversation. However, there is no proper subtype for the usage of pragmatics; hence, a new test type of modification in Category 4: Across Sentence, Within Text, 3-g: pragmatics is added to the present study. The sixth modification is made in Category 4. The cloze items of local context in the past mainly focus on the syntactic elements more than discourse structure; however, reading skills are included in the cloze section in recent years. The. 32. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(43) test-takers need to find out the main idea across the sentences from the whole text. One example from CAP is presented below: (18) . 學. ‧. ‧ 國. . Englishman Robert Scott is known for leading two trips to Antarctica. The first one made him a star; the second 38. . After his first successful trip, Scott decided to be the first person to stand on the South Pole. However, 39. . He would be in a race with Roald Amundsen, from Norway. Both left their countries by ship in June of 1910 and arrived in Antarctica in January of 1911. About ten months later, both teams started their trips down to the South Pole. 40. . Amundsen’s team used dogs, and Scott’s team used horses. Because horses weren’t good at traveling on snow, it took Scott’s team 77 days to arrive at the South Pole. They got there on January 17, 1912, and were surprised to find that Amundsen was ahead of them. The news hit Scott very hard, but what he didn’t know was that 41. . His team began their long trip home with broken hearts. After days of terrible weather and little food, Scott lost his men one after another, and he himself was the last one to meet the end of his life. No one on his team lived to go 治 learned through Scott’s diary. back home and tell their story.政 It was only 大 38. (A) hurt his health 立 (B) opened his eyes ＊ (C) cost him his life (D) made his dream come true (From CAP 2016). sit. y. Nat. In Example 18, to answer the Question 38, the test-takers need to understand the. er. io. following description about the second trip to Antarctica, and then go back to find out. n. a. v. l C the best option for its main idea. As the case stands,Uit’s n i a big shift in cloze test. Test. hengchi. takers must acquire the ability of understanding how sentences are united together to build up a coherent text and identifying the main idea of the text. Therefore, a modification in Category 4: 3-h discourse structure is included in this study. Table 3.3 illustrate the modified taxonomy of the present study.. 33. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(44) Table 3. 3 Modified Taxonomy of the Current Study1 Clause-level Syntax (C1) 1-S-a. determiner without text antecedent. 1-S-b. conjunction below clause level. 1-S-c. role-marking preposition. 1-S-d. relative word in headless clause. 1-S-e. complementizer. 1-S-f. auxiliary. 1-S-g 1-S-h. negation copula Be＆other intensive relationships. 1-S-i. syntactic there or syntactic it ＋Be. 1-S-j. syntactic elements in cleft or pseudo cleft. 1-L-a. referential determiner or deictic with clause. 1-L-c. pronoun (coreferent with clause). 1-L-d. indefinite proform (including you as indefinite). 1-L-e. preposition other than role-marker. 立. 政治大. ‧ 國. 1-L-g. reiteration with clause. 1-L-h. multipart item qualifier in AdjP＆AdvP a. ‧. collocation with clause. n. l. Ch Across Clause, Within Sentence (C3) e. er. io. sit. y. Nat. 1-L-f. 1-L-i. 學. Clause-level Lexis (C2) pre-＆post determiners 1-L-b. ngchi. i Un. v. 2-a. referential determiner or deictic across clauses. 2-b. coordinator. 2-c. pronoun (coreferent across clauses). 2-d. relative word in headed clause. 2-e. collocation across clauses. 2-f. reiteration across clauses. 2-g. subordinator. 2-h. it of extraposition. Across Sentence, Within Text (C4) 3-a. referential determiner or deictic across sentences. 3-b. conjunction between sentences. 3-c. pronoun (coreferent across sentences). 1. Mark the modifications by boldface and italic.. 34. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.

(45) 3-d. collocation across sentences. 3-e. reiteration across sentences. 3-f. textual discourse marker. 3-g. pragmatics. 3-h. discourse structure. Extratextual - written passage and picture (ad, chart, map and others) (C5) 4-a. speech-role pronoun without textual antecedent. 4-b. lexical item with little or no clues from the text Data Analysis This section explains the data analysis framework of coding system and possible. problems that may encounter during the data analysis. The procedure of the current study comprises the following steps, as illustrated in Figure 3.1.. 立. 政治大. ‧. ‧ 國. 學. Categorizing each cloze item into two taxonomies individually. Nat. n. sit. er. io. al. y. Discuss the discrepancy. Ch. engchi. i Un. v. Reaching consistency and agreement on each cloze item analysis. Keying-in the result of the analysis. Analyzing the cloze items based on the modified Jonz’s framework. Figure 3. 1 Data Analysis Procedure of the Study. 35. DOI:10.6814/THE.NCCU.ETMA.003.2018.A07.