利害關係、需求與回沖效應:以台灣之大學英文畢業門檻為例

全文

(1)國立台灣師範大學英語學系博士論文 Doctoral Dissertation Graduate Institute of English National Taiwan Normal University. 利害關係、需求與回沖效應：以台灣之大學英文畢業門檻為例 Stakes, Needs and Washback: An Investigation of the English Benchmark Policy for Graduation and EFL Education at Two Technological Universities in Taiwan. 指導教授: 葉錫南博士 Advisor: Dr. Hsi-nan Yeh. 研究生:朱秀瑜 Hsiu-yu Chu. 中華民國九十八年七月 July, 2009.

(2) 中文摘要本研究旨在探討英文畢業門檻之回沖效應(washback effect)、相關之「利害關係」(stakes)，以及其他可能之中介因素 (mediating factors)。此外，基於「將學生需求納入考試，有助於產生正向回沖效應 (positive washback)」的假設 (Saif, 1999, 2006)，本研究亦對學生之英語需求(needs)進行調查，以期進一步了解學生之英語需求、英文畢業門檻，以及大學英文教育三方之間的關係，是否符合產生正向回沖效應之條件。本研究並試圖以 Green (2007)的 model of washback 作理論上之驗證及整合。本研究進行之場域為台灣兩所設有相同英文畢業門檻及類似配套措施的科技大學。研究方法則採取質性與量化並用。資料之收集包括: 訪談(老師、行政主管、學生及雇主)、教室觀察、教材、全民英檢考古題，以及學生及教師之問卷調查。資料來源多元化，以利於進行各項分析及交互驗證(triangulation)。結論發現英文畢業門檻對教師及學生之回沖效應均低，且多為負面。藉由 Green 的 model of washback 和 Saif 的假設檢視研究結果後，發現相關之「利害關係」太低，以及門檻不符學生對英語之實際需求實為主因。除此之外，儘管台灣社會對於實施英文畢業門檻已具相當程度之共識，但各方關係人(stakeholder) 對此政策仍有不盡相同的意見及看法。根據上述發現，本研究提出具體之建議: 大學英文畢業門檻不同於一般之入學考試，應著重於學生之多樣性及未來畢業後之考量。因此統一之考試標準，如通過全民英檢某級數，恐不符合目前實際狀況，學生應該給予更大之彈性以通過畢業門檻(如採用分級制，不同程度者有不同之進步要求)。而教師們在配合學生需求上則需要花費更大之心力，將學生需求透過測驗及教學之改進，以達到最佳之回沖效應(strong and positive washback)。. i.

(3) ABSTRACT This present study investigated the washback effects of the English exit exam, the stakes of the graduation benchmark policy and possible mediating factors at two technological universities in Taiwan, which had similar English benchmark policies for graduation. Students’ English language needs were also investigated for their relationship with the graduation benchmark requirement and the EFL higher education based on the assumption that positive washback is likely to be generated if students’ language needs are taken into consideration. Both qualitative and quantitative approaches were employed for the present study. Qualitatively, classroom observations and interviews with different stakeholders were conducted and the teaching materials and GEPT test samples were collected for all the related issues. Quantitatively, the Teachers’ Questionnaire (TQ) and Students’ Questionnaire 1 (SQ1) were conducted to explore issues on washback and stakes. Students’ Questionnaire 2 (SQ2) was used to collect information on students’ language needs. Data from different sources were analyzed and triangulated throughout the research project to ensure validity of the results and avoid possible biases. The results show that the washback on teaching was limited to only a “superficial” level, such as teaching test-taking strategies, delivering test information, using GEPT mock tests and so on. The washback on learning was also very little and mostly negative, because the benchmark was too high for a majority of students, while it was of little help to a few high-achieving students in terms of their eagerness to learn English. The perceived stakes of the English exit exam and its make-up measures were generally low among teachers and students. Although most teachers and low-achieving students worried about the graduation benchmark, they tended not to believe the possible consequences to suffer. The low stakes of the graduation ii.

(4) benchmark policy were associated with its limited washback on teaching and learning. The investigation into students’ actual English language needs in the future job market, the EFL higher education and the graduation benchmark requirement show that the three areas did not match well with each other, reducing the possible positive washback that the English exit exam could generate on teachers and students. Also, no effective mediating factors were found to affect the washback effects except that teachers with more administrative experience tended to perceive more pressure from the school administration, and students with low English proficiency worried more about the English exit exam. The findings of the present study were reviewed through Green’s (2007) model of washback and were fully explained, confirming the important roles of test stakes, test difficulty and language needs as reflected in the test design whenever washback is to be taken into consideration. It is also suggested that the English benchmark for graduation should no longer be one fixed line which appears to be high for low-achieving students but low for high-achieving ones. Accordingly, renovated curriculum and tests that aim at students’ language needs and their progress in relation to their entry level might be able to push the currently weak and negative washback towards a healthier dimension.. iii.

(5) ACKNOWLEDGEMENTS I would like to express my gratitude to all the people who have helped me with the completion of this dissertation, although I may not be able to name them all here. My deepest thanks go to my advisor, Dr. Hsi-nan Yeh, for his knowledgeable and patient guidance throughout the formation of this dissertation. I would also like to thank my committee members, Dr. Wu-chang Chang, Dr. Yuh-show Cheng, Dr. Viphavee Vongpumivitch and Dr. Chih-min Shih, for their valuable contributions to this project. My special thanks go to Dr. Shih for his constant encouragement. I also appreciate all the participants in the study for their generous offers to share, especially those teachers who allowed me to sit in their classes to observe. To preserve their anonymity, I may not be able to name a few of them, but my sincere gratitude is with them. Without their participation, this project could not even have been started. I will never forget my dear classmates in the PhD program at NTNU. Over these years striving for our common goal, we have gone through all the sad and happy moments together. I could not have come this far without such a cherished friendship. Hsiu-chuan, Shu-li, Hsiao-hui, Li-yuan, Ping-ju, Shu-ren, Ching-hua, Wei-yu, Kang-ren and Kuan, thank you so much for being such good pals. Finally, I would like to dedicate this dissertation to my family. I am greatly indebted to my parents for their selfless love and full support for my study, career and family life. I am also grateful to my husband, Shinn-shyong, for being understanding and helpful with my computer work and household chores. My only apology goes to my two little ones, Albert and Victoria. During the past few years, the time I could spend with them was limited, but they have always been considerate and promised to be good. iv.

(6) Table of Contents Chinese Abstract .................................................................................................... i English Abstract .................................................................................................... ii Acknowledgements.............................................................................................. iv Table of Contents ...................................................................................................v List of Tables........................................................................................................ xi List of Figures .................................................................................................... xiii List of Transcription Symbols............................................................................ xiv CHAPTER ONE: INTRODUCATION .....................................................................1 Background and Rationale.....................................................................................1 Issues in previous washback studies..............................................................2 The graduation benchmark policy in Taiwan’s EFL higher education ..........3 Mediating factors that affect washback effects............................................10 Students’ English language needs and EFL higher education in Taiwan.....13 Purpose of the Study ............................................................................................14 Research Questions..............................................................................................15 Significance of the Study .....................................................................................15 Chapter Summary ................................................................................................16 CHAPTER TWO: LITERATURE REVIEW .........................................................17 Washback on Teachers .........................................................................................17 Washback on teaching content and teaching methods .................................19 Washback on teachers’ attitudes and feelings ..............................................23 Washback on Students .........................................................................................24 Washback on students’ attitudes and perceptions ........................................24 Washback on students’ learning outcomes...................................................26 Direction and Intensity of Washback ...................................................................27 Factors That Affect Washback .............................................................................30 Test factors and prestige factors...................................................................30 Personal factors............................................................................................34 Teachers’ beliefs about teaching ..........................................................34 Teachers’ beliefs about testing .............................................................36 Learners’ beliefs about language learning ...........................................37 Learners’ beliefs about testing .............................................................38 Micro-context factors...................................................................................39 Macro-context factors ..................................................................................40 Language Needs and Washback...........................................................................40 Chapter Summary ................................................................................................45 CHAPTER THREE: METHODOLOGY................................................................46 Research Context .................................................................................................46 Instruments and Participants ................................................................................50 The general plan for the research instruments and participants...................50 Questionnaires..............................................................................................51 Theoretical basis for the TQ and SQ1..................................................52 The Teachers’ Questionnaire (TQ) on washback and stakes: pilot study ......................................................................................................................54 v.

(7) Students’ Questionnaire 1 (SQ1) on washback and stakes: pilot study58 Students’ Questionnaire 2 (SQ2) on needs ..........................................61 Interview guides...........................................................................................62 Observation forms........................................................................................63 Data Collection Procedures..................................................................................65 Observation ..................................................................................................65 Collecting teaching materials and the GEPT samples .................................66 Interviews with administrators.....................................................................67 Students’ Questionnaire 1 and 2 (SQ1 and 2) and follow-up interviews.....67 The Teachers’ Questionnaire (TQ) and follow-up interviews......................69 Interviews with students’ future employers .................................................70 Data Analysis Procedures ....................................................................................70 Analysis of the questionnaire data ...............................................................71 Scoring of the Teachers’ Questionnaire (TQ) ......................................71 Scoring of Students’ Questionnaire 1 (SQ1)........................................71 Statistical analysis of the TQ and SQ1.................................................72 Students’ Questionnaire 2 (SQ2)..........................................................72 Analysis of the interview data......................................................................72 Interview data on washback and stakes ...............................................73 Interview data on language needs ........................................................73 Analysis of the observation data ..................................................................75 Analysis of the GEPT and teaching materials .............................................75 Chapter Summary ................................................................................................75 CHAPTER FOUR: RESULTS AND ANALYSIS: WASHBACK ON TEACHING ..................................................................................................................................77 Washback of the English Exit Exam on Teaching ...............................................77 Evidence from the interview data ................................................................77 Teachers’ selection for their teaching materials...................................77 Teachers’ delivery of test-taking strategies and GEPT related information.......................................................................................79 Teachers’ worries about their students’ performance...........................80 Limited superficial washback on teaching...........................................81 Summary ..............................................................................................82 Evidence from the observation data.............................................................82 Teacher A2’s Freshman English ..........................................................83 Teacher A3’s Sophomore Aural-oral English ......................................83 Teacher A6’s English Tutorial (make-up course) ................................84 Teacher B1’s Intermediate English and Practice (Elementary) ..........84 Teacher B2’s Freshman English(Advanced 3).....................................85 Teacher B3’s Sophomore English and Practice (Advanced 1) ............85 Teacher B4’s Freshman English (Intermediate 1) ...............................86 Teacher B4’ and B6’s Practical English (make-up course).................86 Summary ..............................................................................................87 Evidence from the teaching and testing materials .......................................87 School A’s teaching and testing materials............................................88 School B’s teaching and testing materials ...........................................88 Analysis and findings...........................................................................90 Summary ..............................................................................................92 vi.

(8) Evidence from the Teachers’ Questionnaire (TQ) survey data ....................92 Summary ..............................................................................................96 Synthesis of the findings..............................................................................96 Mediating Factors for Washback on Teaching.....................................................97 Common ground for the limited washback on teaching ..............................97 Teachers’ doubts about “testing leading teaching” ..............................98 Teachers’ concerns about students’ individual difference....................98 Little interference in teaching from school administration..................99 Summary ............................................................................................100 Investigation of possible mediating factors ...............................................100 Top-down external pressure ...............................................................101 Teacher’ perceived external pressure via administrative experience.104 Internal factors to affect washback on teaching.................................106 Teachers’ and administrators’ beliefs about the purposes of testing ................................................................................................106 Teachers’ perceived teaching roles ............................................109 Teachers’ beliefs about the best teaching method ......................110 Macro factors .....................................................................................111 Summary ............................................................................................114 Chapter Summary ..............................................................................................115 CHAPTER FIVE: RESULTS AND ANALYSIS: WASHBACK ON LEARNIING ................................................................................................................................116 Washback of the English Exit Exam on Learning .............................................116 Evidence from the interview data ..............................................................116 Too difficult a goal to accomplish......................................................117 Waiting until the last minute to boost scores .....................................117 Low motivation for taking the GEPT ................................................119 Summary ............................................................................................120 Evidence from Students’ Questionnaire 1 (SQ1) survey data....................120 Summary ............................................................................................123 Synthesis of the findings............................................................................124 Mediating Factors for Washback on Teaching...................................................125 Underlying problems with students’ English in the TVE system ..............125 Students’ general low achievement in English ..................................125 Discrepancies between high school and vocational high school graduates ................................................................................126 Socio-economic status and English proficiency ........................128 Quantitative evidence.................................................................128 Limitations of college English education ..........................................128 Summary ............................................................................................129 Investigation of possible mediating factors ...............................................130 Evidence from the interview data ......................................................130 Students’ perceived external pressure ........................................130 Internal factors to affect washback on learning .........................132 Other micro and macro factors ..................................................136 Summary ....................................................................................140 Evidence from Students’ Questionnaire 1 (SQ1) survey data............141 Students’ perceived impact of the English exit exam on their vii.

(9) out-of-school practice ............................................................142 Students’ perceived impact of the English exit exam on their efforts .....................................................................................143 Students’ worries about the graduation benchmark ...................143 Summary ....................................................................................144 Synthesis of the findings....................................................................145 Chapter Summary ..............................................................................................146 CHAPTER SIX: RESULTS AND ANALYSIS: STAKES AND STATUS...........147 Purposes of the English Benchmark Policy for Graduation ..............................147 Administrators’ and teachers’ views of the graduation benchmark policy 148 A benchmark set to meet problematic proficiency descriptions ........148 Quantitative evidence.........................................................................149 A “backdoor” as a necessary evil.......................................................150 Students’ views of the make-up course......................................................151 The “cons” of a make-up course as a backdoor.................................152 The “pros” of a make-up course as a backdoor .................................153 The actual intention behind the policy.......................................................154 Administrators: The GEPT used for management purposes..............154 Teachers: The GEPT used as an incentive for learning .....................155 Students: Taking the GEPT to fulfill social expectations ..................158 Some negative voices from students..................................................158 Summary ............................................................................................160 Perceived Stakes and Status of the Graduation Benchmark Policy ...................161 Teachers’ perceived stakes and status of the graduation benchmark policy ....................................................................................................................161 Analysis of the interview data............................................................161 Summary ....................................................................................163 Analysis of the Teachers’ Questionnaire (TQ) survey data ...............163 Summary ....................................................................................165 Synthesis of the findings....................................................................166 Students’ perceived stakes and status of the graduation benchmark policy ....................................................................................................................166 Analysis of the interview data............................................................166 Not fatal to fail the test...............................................................166 An open backdoor ......................................................................167 Unawareness of the related rewarding policy............................169 Less effort in the exit exam than in the College Entrance Exam ................................................................................................169 An additional finding .................................................................170 Summary ....................................................................................171 Analysis of the Students’ Questionnaire 1 (SQ1) survey data ...........171 Summary ....................................................................................173 Synthesis of the findings....................................................................173 Chapter Summary ..............................................................................................173 CHAPTER SEVEN: RESULTS AND ANALYSIS: LANGUAGE NEEDS........176 Overview of Students’ English Language Needs...............................................176 Students’ actual English language needs in college and the job market ....176 viii.

(10) Analysis of the Students’ Questionnaire 2 (SQ2) survey data ...........177 Analysis of the interview data with students’ future employers ........183 Perceived students’ English language needs in the job market..................186 Overview of the EFL Higher Education and Graduation Benchmark Requirement ........................................................................................................................188 EFL higher Education at both schools .......................................................188 Analysis of the observation and interview data for the required courses and their teaching materials ...........................................................189 Analysis of the testing materials for the required courses .................190 Graduation benchmark requirement for both schools................................192 Analysis of the GEPT old tests ..........................................................192 Analysis of the observation data in the make-up course and their teaching and testing materials................................................193 Comparison Across Students’ English Language Needs, EFL Higher Education and Graduation Benchmark Requirement ....................................193 Chapter Summary ..............................................................................................199 CHAPTER EIGHT: DISCUSSION AND CONCLUSION..................................202 Discussion of the Results and Analysis .............................................................202 Research Question One: What is the washback of the English exit exam on college teachers and students? Is the washback affected by any mediating factors?...................................................................................................202 Research Question Two: How do college teachers and students perceive the stakes and status of the graduation benchmark policy? Is the washback of the English exit exam influenced by their perceptions of the stakes or status of the graduation benchmark policy?...........................................208 Research Question Three: Does the EFL higher education, as well as the graduation benchmark requirement, meet students’ English language needs?.....................................................................................................210 Theoretical Implications ....................................................................................211 Pedagogical Implications ...................................................................................214 Limitations of the Present Study........................................................................219 Suggestions for Future Studies ..........................................................................220 Concluding Remarks..........................................................................................221 REFERENCES.........................................................................................................224 Appendix A Appendix B Appendix C Appendix D Appendix E Appendix F Appendix G Appendix H Appendix I Appendix J Appendix K Appendix L. Table of approximate score comparability.........................................235 Teachers’ Questionnaire (English Version)........................................236 Teachers’ Questionnaire (Chinese Version) .......................................241 Students’ Questionnaire 1 (English Version) .....................................246 Students’ Questionnaire 1 (Chinese Version).....................................251 Students’ Questionnaire 2 (English Version) .....................................256 Students’ Questionnaire 2 (Chinese Version).....................................263 Interview Guide for Teachers.............................................................271 Interview Guide for Students .............................................................274 Interview Guide for Administrators ...................................................277 Interview Guide for Future Employers ..............................................279 Observation form for required English course...................................282 ix.

(11) Appendix M Appendix N Appendix O Appendix P Appendix Q Appendix R Appendix S Appendix T Appendix U Appendix V Appendix W Appendix X Appendix Y Appendix Z. Observation form for the make-up course .........................................284 Letter to teachers requesting observation ..........................................286 Consent form for teachers to be observed..........................................288 Observation schedule .........................................................................289 Letter to administrators requesting interviews...................................290 Consent form for participants to be interviewed................................291 Participants’ background information................................................292 Letter to students to be interviewed ...................................................294 Letter to teachers requesting interviews ............................................295 Letter to students’ future employers requesting interviews ...............296 Samples of School A and School B’s exams and quizzes..................297 Teacher participants’ background information for the TQ.................300 Student participants’ background information for SQ1 and SQ2 ......301 Tables of TLU task characteristics.....................................................303. x.

(12) List of Tables Table 1. Bachman and Palmer’s model of language ability ....................................44. Table 2. A brief description of School A and School B...........................................49. Table 3. The research instruments and participants for the present study...............51. Table 4. A comparison of Alderson and Wall’s hypotheses (rephrased) and Spratt’s categorizations of areas of washback effects.............................................53. Table 5. Washback effects in three dimensions.......................................................54. Table 6. The structure of the revised Teachers’ Questionnaire (TQ) ......................56. Table 7. The results of factor analysis and the reliability test on the revised Teachers’ Questionnaire (TQ) ...................................................................57. Table 8. The structure of the preliminary Students’ Questionnaire 1 (SQ1) ...........59. Table 9. The results of factor analysis and the reliability test on the revised Students’ Questionnaire 1 (SQ1) ...............................................................60. Table 10. The overall structure of Students’ Questionnaire 2 (SQ2) ........................61. Table 11. Data collection schedule............................................................................65. Table 12. Distribution and numbers of student respondents for SQ1 and SQ2 ........68. Table 13. Modified model of language ability from Bachman and Palmer (1996) ..74. Table 14 Descriptive statistics for teachers’ perceived impact of the graduation benchmark on three dimensions.................................................................93 Table 15. Repeated-measures ANOVA on the three dimensions of teachers’ perceived impact of the English exit exam ...............................................94. Table 16. A list of mean scores for the TQ items on the three dimensions of teachers’ perceived impact........................................................................................94. Table 17. Descriptive statistics for students’ perceived impact of the graduation benchmark on three dimensions (excluding students who already passed the benchmark before college) ................................................................121. Table 18. Repeated-measures ANOVA on the three dimensions of students’ perceived impact of the English exit exam .............................................122. Table 19. A list of mean scores for the SQ1 items on the three dimensions of teachers’ perceived impact ......................................................................123 xi.

(13) Table 20. Regression of students’ demographic and personal factors on their perceived impact of the English exit exam on out-of-school practice ....142. Table 21. Regression of students’ demographic and personal factors on their perceived impact of the English exit exam on efforts .............................143. Table 22. Regression of students’ demographic and personal factors on their worries about the graduation benchmark .............................................................144. Table 23. Descriptive statistics for teachers’ perceived stakes and status of the English exit exam ....................................................................................164. Table 24. Paired-samples t-test on teachers’ perceived stakes and status of the English exit exam ....................................................................................164. Table 25. A list of mean scores for the TQ items on teachers’ perceived stakes and status of the English exit exam................................................................165. Table 26. Descriptive statistics for students’ perceived status of the English exit exam (all student participants) ..............................................................172. Table 27. A list of mean scores for the SQ1 items on students’ perceived status of the English exit exam ..................................................................................172. Table 28. Comparison across teachers’, students’ and administrators’ views on washback-related issues ..........................................................................175. Table 29. Descriptive statistics for college students’ language needs in SQ2.........177. Table 30. Comparison of components of language ability across TLU tasks for students in the future job market, EFL higher education and graduation benchmark requirements .........................................................................195. Table 31. The TLU tasks in Table 30 ......................................................................196. xii.

(14) List of Figures Figure 1. A general model of the graduation benchmark requirement for English proficiency in Taiwanese colleges and universities ....................................8. Figure 2. Tension between the pedagogic force and the administrative force as a result of the benchmark policy .................................................................10. Figure 3. A general model of the tension between assessment for the administrative purposes and assessment for the pedagogic purposes ..............................12. Figure 4. Model of washback, incorporating intensity and direction ......................32. Figure 5. Systematic approach to designing and maintaining language curriculum.... ..................................................................................................................41. Figure 6. Distribution of teachers’ perceived impact of the English exit exam on the three dimensions .......................................................................................93. Figure 7. Students’ perceived impact of the English exit exam on their out-of-school practice and their efforts, as well as their worries about the graduation benchmark (excluding students who already passed the benchmark before college) .................................................................................................121. Figure 8. Teachers’ perceived stakes and status of the English exit exam.............164. Figure 9. Students’ perceived status of the English exit exam...............................172. xiii.

(15) List of Transcription Symbols The following symbols are used in the transcription excerpts throughout the dissertation: [. ]. words in the brackets representing the missing words assumed by the researcher and added in the excerpt to make clear the utterance. (. ). words in the parentheses representing notes or explanations provided by the researcher for the previous utterance. **********. representing omission of one or several exchanges in the interview. {sighing}. sighing in the interview. xiv.

(16) CHAPTER ONE. INTRODUCTION. Background and Rationale Tests are believed to be a powerful instrument to bring about changes in language teaching and learning (Shohamy, 1993). This is usually referred to as washback. Washback has long been mentioned in the literature in language testing, but the phenomenon has only started to attract attention from researchers in recent years. Ever since Popham (1987) proposed that “measurement-driven instruction is the most cost-effective way of improving the quality of public education” (p.679), various definitions have been offered for the notion of washback. Bailey (1996) characterizes washback as “the influence of testing on teaching and learning” (p.259). Shohamy, Donitsa-Schmidt, and Ferman (1996) describe it simply as “the connection between testing and learning” (p.298). According to Messick (1996), washback refers to “the extent to which the introduction and use of a test influences language teachers and learners to do things they would not otherwise do that promote or inhibit language learning” (p.241). Hughes (1989) uses a different term backwash 1 to refer to “the effect of testing on teaching and learning” (p.1), asserting that testing can have either a beneficial or a harmful effect on teaching and learning. He later discusses the mechanisms by which washback works. He states that “in order to clarify our thinking on backwash, it is helpful, I believe, to distinguish between participants, process and product in teaching and learning” (Hughes, 1993, p.2). Here, the participants refer to the stakeholders involved in the test, mainly the students, teachers and administrators; the process refers to any actions taken by the participants, which may contribute to the process of learning and the product refers to 1. Alderson and Wall (1993) pointed out that the phenomenon is referred to as “backwash” in general education circles, but it has come to be known as “washback” in British applied linguistics. However, both terms basically refer to the same notion in the literature. 1.

(17) what is learned and the quality of the learning. Issues in previous washback studies Ever since Alderson and Wall’s (1993) Sri Lankan Impact Study, which investigated the washback effects of a new O-level English examination in that country, more and more empirical studies have come forth in this field. According to Spratt’s (2005) review of washback studies, empirical studies over the last decade on areas affected by washback have covered the following: curriculum, materials, teaching methods, feelings and attitudes, and learning. However, except a few general theoretical derivations, such as Alderson and Wall’s (1993) Washback Hypothesis and Bailey’s (1996) Basic Model of Washback, these studies have not yet brought forth a comprehensive conclusion of the washback phenomena. It is probably because the tests under investigation in the previous studies were in various forms and with different foci, and the educational contexts of these tests being investigated were not exactly the same. However, a more basic problem still lies in that studies on this newly-developed research subject have not yet accumulated to a substantial amount to support or build up more hypotheses or theoretical models on washback. As a result, there is a strong need for more empirical studies on different tests and educational contexts in order to enrich the scope of the washback research, to testify the existing washback hypotheses, and furthermore, to reach a more comprehensive model of the washback effect. Most of the tests that have been investigated in the previous studies were new tests involved in some sort of education reform, such as New O-level English Examination in Sri Lanka (Wall and Alderson, 1993), Hong Kong Certificate of Education Examination in English (Cheng, 1997), The Basic Competence Test in English in Taiwan (Chen, 2002), the National Matriculation English Test in China (Qi, 2.

(18) 2004) and the EFL National Oral Matriculation Test in Israel (Ferman, 2004). Other types of tests investigated in the previous studies include some existing entrance exams, such as the college entrance exams in Japan (Watanabe, 1996, 2000) and the Technological and Vocational Education entrance exam in Taiwan (Shih et al., 2000, in Chinese); and some standardized language tests, such as the TOEFL (Alderson and Hamp Lyons, 1996) and IELTS (Read and Hayes, 2003; Saville and Hawkey, 2004) and so on. In addition, previous washback studies have placed too much emphasis on the teachers’ side in proportion to the students’ side. There should be equal attention paid to the washback effect on learning as well as on teaching. The graduation benchmark policy in Taiwan’s EFL higher education As previously mentioned, many new public exams were planned and implemented with an attempt to improve language teaching and learning. However, most of the time, the actual outcome did not seem to match the intended washback, or the changes were only superficial (Shohamy, 1993; Cheng, 1997), namely, the changes were often instrumental and due to people’s fear of authority. However, for many decision-making bodies, such as the Ministry of Education or school administrations, tests are still treated as a panacea for educational innovation. This is especially true in many Asian countries, where examinations have long played an important role in their cultural and educational history. Chen et al. (2005) in a study on Taiwanese students’ language learning motivation even added one more category, the required orientation, to the existing two categories, the integrative orientation and instrumental orientation, to signify the important role tests have played in most Taiwanese students’ language learning experience. With this exam-oriented tradition, it was not surprising to see that the Ministry of Education (MOE) of Taiwan embodied an English proficiency benchmark policy for college 3.

(19) undergraduates in its 2005-2008 Administration Guidelines to promote globalization in Taiwan’s tertiary education. Under the current education system in Taiwan, there are two types of undergraduate programs. One type is the four-year university undergraduate programs, which recruit mostly high school graduates. The other type belongs to colleges or universities of technology in the Technological and Vocational Education (TVE) system, which offers (1) four-year undergraduate programs, mainly for vocational high school graduates, and (2) two-year undergraduate programs, particularly for five-year junior college (starting after the junior high school) graduates and two-year junior college (starting after the vocational high school) graduates. Students graduated from any of the above undergraduate programs will be awarded a bachelors’ degree. According to the MOE’s 2005-2008 Administration Guidelines, it was expected that by the year 2007, up to 50% of the four-year university graduates should have passed the Intermediate Level of the General English Proficiency Test (the GEPT) and 50% of university or college of technology graduates passed the Elementary Level of the test, which was still unaccomplished by 2008. Meanwhile, the MOE also required each university and college to set their own English benchmark for graduation, using the GEPT or other language tests, such as the TOEFL, TOEIC and IELTS. The General English Proficiency Test is a local test which has been developed by the Language & Training & Testing Center (LTTC) in Taiwan since 1999 and is administered in five levels, Elementary, Intermediate, High-Intermediate, Advanced and Superior. Each of the first four levels is administered in two stages: the first stage includes the listening and reading components and the second stage includes the speaking and writing components. Examinees must pass the first stage before proceeding on to the second. The Superior Level is an integrated test of all four skills. 4.

(20) Each level, based on the LTTC’s GEPT-CEFR 2 linking study (Wu and Wu, 2007), approximates to the CEFR’s A2-Waystage (Elementary), B1-Threshold (Intermediate), B2-Vantage (Higher-Intermediate), C1-Effective Operational Proficiency (Advanced) and C2-Mastery (Superior) respectively. A table of approximate score comparability across the GEPT and several widely-used language tests and their mapping with the CEFR can be found in Appendix A. For more information about the GEPT test, please refer to the LTTC’s website at http://www.lttc.ntu.edu.tw/. Under the English benchmark policy for graduation, the GEPT test, compared to other standardized tests, is the most-frequently taken test for Taiwanese college students to fulfill the graduation benchmark requirement nowadays. In a similar vein to most of the previous washback studies, the MOE of Taiwan is trying to boost college students’ English proficiency by implementing a benchmark policy and providing massive subsidies 3 to support the policy. As a result, more and more universities and colleges have implemented or are implementing standardized English exit exams on their students ever since. National Taiwan University (NTU), for example, even set its English proficiency benchmark as early as in the 2002 academic year. In general, most four-year universities and colleges set their graduation benchmark for non-English majors at the first stage of the High-intermediate or Intermediate level of the GEPT, while most technological and vocational universities and colleges set theirs at the fist stage of the Intermediate or the second stage of the Elementary level. For English majors in both systems, the benchmark is usually higher than that for their non-English major peers. The graduation benchmarks are. 2. 3. CEFR, The Common European Framework of Reference for Languages: Learning, teaching, assessment (Council of Europe, 2001) provides a common basis for the description of language courses, syllabuses, and qualifications. In recent years, the MOE of Taiwan has provided massive subsidies to support English teaching and learning in universities and colleges, such as the annual grant for The Program of Uplifting TVE (Technological and Vocational Education) Students’ English Proficiency and the Xteach Project for almost every university and college here in Taiwan. 5.

(21) usually set by each university and college at the above levels in accordance with the social expectations as well as the MOE policy; that is, supposedly, four-year university graduates should reach an English proficiency level which approximates to the CEFR-B1 Level (roughly the TOEFL CBT scores 137 or above), and students in the technological and vocational track are expected to have lower English proficiency at the CEFR-A2 Level (roughly the TOEFL CBT scores 90 or above) than their four-year university counterparts. This trend has reflected the strong demand of English in Taiwan’s higher education and, what is more, the power of the MOE authority. As Shohamy (1993) put it, “using tests to solve educational problems is a simplistic approach to a complex problem. It works on people’s fear of authority” (p.19). In her point of view, the power of tests is too strong to bring about any meaningful educational changes, since the changes are often instrumental. Unfortunately, even though it is still unknown whether the English proficiency benchmark policy has brought about any meaningful educational changes, there has been a striking discrepancy between the MOE’s expectation and students’ actual performance over the years, with the latter being much poorer than the former. For example, National Taiwan University set its graduation benchmark at the first stage of the GEPT High-Intermediate level, but the average passing rate for the first three years of the implementation was only 35%. However, NTU students’ performance was far better than students of other universities and colleges in Taiwan. For students in some technological and vocational colleges, the passing rate is lower than 3% with the benchmark set at the first stage of the GEPT Elementary level. Inevitably, almost every university and college which has set its English benchmark for graduation has had to provide other options for those students who can not meet the requirement before graduation. The most common way is that students 6.

(22) who are unable to pass the English exit exam will have to take extra courses to fulfill the graduation benchmark requirement instead. Some schools offer an internal test for students to choose to take on campus in addition to the external tests. A few others lowered the passing scores originally set for the GEPT test 4 . In general, most universities and colleges have used external tests (the GEPT and its equivalents) as their exit exams, but the supporting measures have varied from school to school. Figure 1 shows a general model of the graduation benchmark requirement for English proficiency in Taiwanese colleges and universities. As pointed out by Wu (2007), a serious problem has arisen with these supporting measures for the graduation benchmark requirement; that is, the criteria set for each of the measures for the graduation benchmark requirement did not seem to align with one another. There is no evidence to show that a student passing an internal exam or an extra course has the same level of required English proficiency for graduation as those who passed the external GEPT test. To sum up, most college students have not been able to pass the required English exit exam. Instead, they have to take the internal English exit test or an extra course to fulfill the graduation benchmark requirement. However, there seems to be an alignment problem between the MOE-favored benchmark and each individual school’s substitute measures.. 4. The perfect scores and passing scores for the Elementary, Intermediate and High-intermediate levels of the GEPT are as follows (effective before 2008): Level Elementary Intermediate Highintermediate. Scores perfect passing perfect passing perfect passing. Reading 120 80 120 80 120 80. Listening 120 80 120 80 120 80. Speaking 100 80 100 80. Writing 100 70 100 80 100 80. A new scoring system went into effect in 2008, which requires a total score of 160 or higher on listening and reading, with neither component gaining less than 72, to pass the first stage of each of the three levels. In addition, a speaking component has been added to the second stage of the Elementary level of the GEPT. 7.

(23) Figure 1.. A general model of the graduation benchmark requirement for English proficiency in Taiwanese colleges and universities. Translated by Wu (2007) from the original in Chinese. Reproduced with permission from Wu.. At this point, it might be time to think about the following paradoxes: Are the English proficiency benchmarks set too high for most college students today or are they actually unqualified in their English proficiency? Has this policy brought about. 8.

(24) any meaningful educational changes or was it only a simplistic approach to a complex problem? It is actually not hard to find in the previous literature that the paradoxes seem to have resulted from the tension between assessment for pedagogic purposes and assessment for administrative purposes, with the latter usually winning out (McCay, 2006; cited in Wu, 2007). Though there might be some agreement between the two forces, more disagreement seems to have pulled the two forces apart. If the washback effects intended by the administration are to be achieved, those who serve the pedagogic purposes (the teachers) and those who are pedagogically involved (the students) will have to give in to the administrative force and move towards it. However, exactly how much further the pedagogic force will be pulled towards or away from the administrative force still remains a question for the present study. Figure 2 shows the tug-of-war between the two forces as a result of the benchmark policy. The overlap between a solid circle and any dotted circle represents the agreement between the two forces. The overlap could range from a large portion to none, suggesting that the pedagogic force may yield to or be entirely ungoverned by the administrative force. As for how the pedagogic force will be pulled towards or away from the administrative force, it is necessary to bring up another important issue here. In the next section, the mediating factors that affect the washback effects will be presented.. 9.

(25) Figure 2. Tension between the pedagogic force and the administrative force as a result of the benchmark policy Mediating factors that affect washback effects Previous studies have suggested that washback is a “complex and elusive phenomenon” (Spratt, 2005, p.21). When we expect to make changes through tests, we should take the whole education context into consideration and not rely on the test alone (Shohamy, 1992; Alderson &Wall, 1993; Messick, 1996; Wall, 1996; Watanabe, 2004). In other words, it is important to know how the possible mediating factors might affect the washback effects and not to depend on the power of test per se to create changes. Watanabe (2004) suggests the following factors that seem to be mediating the process of washback: (1) test factors (e.g., test methods, test contents, skills tested, etc.); (2) prestige factors (e.g., stakes of the test, status of the test, etc.); 10.

(26) (3) personal factors (e.g., teachers’ educational backgrounds, their beliefs about the best methods of teaching/learning, etc.); (4) micro-context factors (e.g., the school setting in which the test preparation is being carried out) and (5) macro–context factors (e.g., the society where the test is used). In a similar vein, Spratt (2005) categorizes the factors identified by the empirical studies as influential in affecting washback into four groups, namely, (1) teacher-related factors, such as teacher beliefs (about the exam, about what constitutes effective teaching methods, etc.), teachers’ attitudes towards the exam, teachers’ education and training, teachers’ personalities and their willingness to innovate; (2) resources, such as whether or not customized materials and exam support materials are available to teachers, etc.; (3) the school, such as school atmosphere, learning traditions, etc.; (4) the exam itself, such as its proximity, its stakes, the status of the language it tests, its purpose, the formats it employs, etc. Figure 3 proposes a general model of the tension between the pedagogic force and the administrative force as driven by the possible mediating factors after the implementation of an external exam. The two circles start with something in common before the implementation of the exam, shown as their overlapping with each other. It is suggested that the increased (or reduced) portion of the overlapping part between the two circles being pushed after the implementation of the exam accounts for the washback effects of the exam affected by the possible mediating factors.. 11.

(27) Figure 3.. A general model of the tension between assessment for the administrative purposes and assessment for the pedagogic purposes. In terms of the graduation benchmark policy here in Taiwan, the prestige factor seems to have played a far more critical role and deserve more attention than other mediating factors. The rationale here is that it seems difficult for most students to pass the external English exit exam, but they can still fulfill the graduation benchmark requirement through other substitute measures that do not seem as difficult as the external test. As a result, the stakes of the graduation benchmark policy seem to be reduced on the ground that the consequences of failing the external exit exam are not fatal and can be made up by other “shortcuts” to fulfilling the graduation benchmark requirement. This assumption is theoretically based on one of Alderson and Wall’s (1993). 12.

(28) fifteen Washback Hypotheses 5 that “tests that have important consequences will have washback; and conversely” (p.120). Some supporting evidence can also be found in previous washback studies, such as Shohamy (1993), Stoneman (2005) and Shih (2007). Shohamy investigates the impact of three tests in Israel and one of her conclusions is that the strength of the impact varied, depending on the type of test and on other variables such as stakes of the test. Stoneman studies the impact of an exit English test on Hong Kong undergraduates and concludes that test stakes were probably a more important motivating factor than the test’s international status for students. In a similar situation to the present study, Shih investigates the washback of the GEPT on English majors in Taiwan and finds that the GEPT generated little washback largely due to its lack of immediate importance for students. Accordingly, the stakes of the graduation benchmark policy, in addition to other possible mediating factors, deserve more attention in the present study. Students’ English language needs and EFL higher education in Taiwan The probably low stakes of the graduation benchmark policy have raised the issue about how stakes may influence washback. However, it would be more meaningful to see how such a benchmark policy, if in any other ways, would be able to bring about the intended, preferably positive, washback effects. Saif (1999, 2006) claims that positive washback can be achieved by designing the test in question in accordance with the stakeholders’ language needs. Conversely, it is hypothetically assumed that a test with no linkage to the stakeholders’ language needs may not bring about positive washback. In terms of the English benchmark policy for graduation in Taiwan, the administrative force is entirely relying on one test to bring about changes in the EFL higher education, but exactly how the GEPT, serving as an exit test,. 5. For all the fifteen Washback Hypotheses, please refer to Chapter Two. 13.

(29) reflects colleges students’ language needs has never been a major issue, not to mention its substitute measures. It is thus worth thinking whether there is a connection between students’ poor performance on the test and the ignorance of students’ needs in the English benchmark policy for graduation. Meanwhile, it is also worth thinking how the EFL higher education meets college students’ English language needs in Taiwan, especially when there are students who have already passed the English exit exam before entering college, students who are expected to pass the English exit exam after entering college, and students who do not seem to be able to pass the benchmark before graduation. It would then help capture a broader picture of the research context if the connection is investigated between the goal of the EFL higher education in Taiwan and college students’ diverse English language needs. Purpose of the Study The total number of colleges and universities in Taiwan, as of the 2008-2009 school year, was 147 (69 four-year universities and colleges; 78 technological universities and colleges), among which more than 100 schools or departments had adopted the GEPT as their graduation benchmarks (United Daily News, May 29, 2008). This present study chose two technological universities in Taiwan to focus on, which had set similar English graduation benchmark policies by the time this research project was conducted. This present study modeled the washback investigation on some of the previous studies (e.g. Chen, 2002 and Shih, 2006) but paid equal attention to both teaching and learning in view of the fact that most washback studies have targeted on the former and ignored the latter. It then focused on the relationship between the washback effects of the English exit exam and the stakes of the graduation benchmark policy, 14.

(30) among other possible mediating factors. Students’ English language needs were also investigated to see their relationship with the graduation benchmark requirement. It is assumed that if the policy reflects stakeholders’ language needs, it would bring about more positive washback. The EFL higher education was also studied to see if students’ diverse language needs have been met in college English curriculum as well as the graduation benchmark requirement for English. Research Questions Three research questions are formulated for this present study: 1. What is the washback of the English exit exam on college teachers and students? Is the washback affected by any mediating factors? 2. How do college teachers and students perceive the stakes and status of the graduation benchmark policy? Is the washback of the English exit exam influenced by their perceptions of the stakes or status of the graduation benchmark policy? 3. Does the EFL higher education, as well as the graduation benchmark requirement, meet students’ English language needs? Significance of the Study This study is significant both academically and pedagogically. Academically, this study contributes to the present washback literature some empirical findings about a college English exit exam as well as the related graduation benchmark policy, which has only rarely been explored in studies such as Stoneman (2005) and Shih (2007). This present study focuses on the impact of the English benchmark policy for graduation on non-English majors in contrast to Shih’s study on English majors. This study also, in addition to the many other mediating factors that have been raised in the previous studies, pays special attention to the stakes issue, as it has been brought up in Stoneman’s study. It is strongly believed that the stakes of the 15.

(31) English exit exam as well as the overall graduation benchmark policy play an important role acting on the washback effects on both teachers and students. Furthermore, college students’ English language needs are also treated in the present study, trying to extend Saif’s (1999, 2006) research finding that positive washback could be achieved by taking stakeholders’ needs into consideration when designing the test in question. It is assumed that a test or policy with considerable relation to the stakeholders’ language needs will bring about positive washback or vice versa, and this assumption was tested in the present study. It is hoped that the investigation of the washback effects and stakes of the graduation benchmark policy, as well as the analysis of students’ language needs and their relations with the graduation benchmark requirement and the goal of the EFL higher education in Taiwan, might bring together different important issues in previous washback studies and reveal some general relations among them. Pedagogically, the findings of this present study may provide some suggestions to the MOE of Taiwan about how such a policy could be better adjusted. In other words, in addition to its primary goal of enriching the scope of the washback research, this study was also conducted for real educational purposes here in Taiwan. Chapter Summary In this chapter, various definitions of washback effects and major issues in the previous empirical studies on washback have been discussed, followed by a brief introduction of the graduation benchmark policy in Taiwan’s EFL higher education. In order to examine the possible problems lying behind the policy itself and find workable solutions, three research questions were proposed for the present study, which was conducted on two technological universities in Taiwan.. 16.

(32) CHAPTER TWO. LITERATURE REVIEW. As introduced in Chapter One, this chapter reviews previous studies on washback on teachers, washback on students, washback direction and intensity, and factors that affect washback. Students’ language needs in relation to washback are then discussed with special attention to Bachman and Palmer’s (1996) model of language abilities. Washback on Teachers Previous washback studies have paid far more attention to teachers and teaching than to learners and learning, even though the following theoretical assertions on washback have placed equal emphasis on both sides. Alderson and Wall (1993) propose the Washback Hypothesis (pp. 120-121), assuming that “teachers and learners do things they would not necessarily otherwise do because of the test” (p.117). They further specify the Washback Hypothesis with fifteen statements regarding the influence of testing on teaching and learning as follows: (1) A test will influence teaching. (2) A test will influence learning. (3) A test will influence what teachers teach. (4) A test will influence how teachers teach. (5) A test will influence what learners learn. (6) A test will influence how learners learn. (7) A test will influence the rate and sequence of teaching. (8) A test will influence the rate and sequence of learning. (9) A test will influence the degree and depth of teaching. 17.

(33) (10) A test will influence the degree and depth of learning. (11) A test will influence attitudes to the content, method, etc., of teaching and learning. (12) Tests that have important consequences will have washback; and conversely. (13) Tests that do not have important consequences will have no washback. (14) Tests will have washback on all learners and teachers. (15) Tests will have washback effects for some learners and some teachers, but not for others. It is evident that the fifteen smaller washback hypotheses have placed symmetrical emphases on both teaching and learning. In Hughes’s model (1993) on how washback works, he distinguishes between participants, process and product in teaching and learning. As mentioned earlier in Chapter One, the participants refer to the stakeholders involved in the test, such as students, teachers, administrators, materials developers and publishers; the process refers to any actions taken by the participants which may contribute to the process of learning and the product refers to what is learned and the quality of the learning. Here learning, as compared to teaching, is even more the center of attention in the washback model. Bailey (1996) proposes a basic model of washback, which combines Alderson and Wall’s Washback Hypothesis and Hughes’s model. In his model, both washback to the program and washback to the learner have been equally taken care of in that the former refers to washback on what teachers teach, how teachers teach, the rate and sequence of teaching, and the degree and depth of teaching, while the latter refers to washback on what learners learn, how learners learn, the rate and sequence of learning, and the degree and depth of learning. However, due to lack of empirical evidence, Alderson and Wall (1993) has 18.

(34) criticized their Washback Hypothesis for being “unduly simplistic and makes too many untested assumptions” (p.119). Ever since Alderson and Wall’s claim, an increasing number of empirical studies on washback effects have appeared in the literature. As indicated by Watanabe (2004, p.21) in an article discussing the methodology in washback studies, however, the area of “washback to the program,” namely, washback on teaching, has been relatively well explored; while “washback to the learner” has received less attention in the literature. According to Spratt’s (2005) review of the empirical studies on washback during the past decade, an apparently large proportion of the previous studies have looked into the washback effects on the teacher and teaching, which, as classified by Spratt, have covered curriculum, materials, teaching methods, feelings and attitudes. Washback on teaching content and teaching methods Studies on the different aspects of teaching affected by the washback effects have come to different conclusions. Very often a contrast has been perceived between the teaching content and the teaching methods, with the former showing evidence of washback and the latter reflecting no sign of washback. For example, Wall and Alderson (1993) in their Sri Lankan Impact Study claim that, according to their cross-country observations of classroom teaching towards a new O-level English examination introduced to Sri Lanka in 1988, there was no impact on either teaching content or teaching methodology in lessons where the textbook was used (75% of all the lessons), while in lessons where the textbook was not used (25%), there was indeed impact on teaching content but no impact on teaching methodology. In brief, it seems that the exam in this context affected only a small part of teachers on how they chose the teaching content but affected no teachers on how they taught in the classroom. Similar results are also found in Cheng’s (1995) 19.

(35) study on the influence of the introduction of an integrated and task-based approach into the existing Hong Kong Certificate of Education Examination in English (HKCEE). Cheng finds that textbooks played a very important role in the teaching of English in Hong Kong secondary schools. That is, when teaching the new syllabus, teachers did adopt different types of activities from the ones they had used before the introduction of the new exam; however, these obvious changes made in teaching lay in the different activities designed in the textbooks, which had come from the textbooks publishers’ understanding of the new HKCEE. In this sense, the washback effect of the HKCEE on teachers’ teaching in Hong Kong secondary schools was “superficial” because teachers seemed to teach what came next in the textbook. In a study on the 1999 pilot version of the National Matriculation English Test (NMET) for Senior III students in China, Qi (2004) compares the test constructors’ beliefs and teachers’ beliefs in the exam and finds there were mismatch and partial match between the two. The mismatch was that the test constructors intended to de-emphasize formal linguistic knowledge in the exam while the teachers still stuck to the teaching of linguistic knowledge towards the exam. The partial match was that the test constructors intended to encourage teachers and learners to spend more time on language use through listening, reading and writing activities, and these skills were actually drilled in the Senior III English course. However, once again, the skills intended by the test constructors were included in the content, but not in the methodology of teaching. Qi argues that the NMET failed to realize the teaching methodology considered by the test constructors because most of its items were still in the multiple-choice format. She also suggests other factors that are thought to hinder operation of intended washback, such as teachers’ beliefs about language teaching and test preparation, teachers’ education background, teaching style, teaching experience and inadequate communication between test makers and test users. 20.

(36) However, as Spratt (2005) points out, the perception that washback affects teaching content but not teaching methods is not fully supported by the findings of some studies, such as Alderson and Hamp-Lyons (1996) and Watanabe (1996). The results of these two studies show that “whether the exam affects methods or not may also depend on factors other than the exam itself, such as the individual teacher” (Spratt, 2005, p.16). In Alderson and Hamp-Lyons’ study (1996), two teachers were observed teaching both the TOEFL preparation course and the non-TOEFL course. It was found that the two teachers used different teaching techniques in teaching the same course. However, their teaching techniques did not show much difference when comparing their own teaching in both the TOEFL preparation course and the non-TOEFL course. As a result, the researchers conclude that it is the individual teacher styles and personalities, instead of the TOEFL test, that clearly contribute to the nature of the classes observed. Watanabe (1996) observes how exams influenced two teachers’ teaching methods in a “yobiko” (preparatory school that offers various courses targeting specific university exams in Japan). He points out the general belief that university entrance exams in Japan encourage the use of the grammar-translation method and does find that one teacher used only the grammar-translation method in two different courses targeting two specific university exams even though one of the exams was not “grammar-translation oriented.”. However, the other teacher in the investigation did. use different teaching methods when teaching the two different courses. The possible explanations why exams did have washback effects on one teacher’s teaching methodology but not on the other’s, as proposed by Watanabe, might include (1) teachers’ education background and experiences; (2) teachers’ different beliefs about effective teaching methods and (3) the time related to the exam dates when the 21.

(37) observations were made. According to the above studies, teacher factors, rather than the test itself, play an important role in affecting the washback effects on teaching methodology. A similar result showing washback affects teaching methodology as well as teaching content is also found in Saif’s (1999, 2006) case study of international teaching assistants. This study is different from previous washback studies in that it includes a needs analysis before the test development and the test design in the investigation. The results show that the teacher has changed both the teaching content, such as locating certain areas of language problems and the teaching methodology as well, such as choosing “group instruction” as the main teaching activity. What is even more encouraging is that the ITAs in the experimental group showed significant increases in the scores from pretest to post-test. Saif’s study will be discussed more in the later section on language needs and washback. Chen (2002), in a broader sense, investigates how Taiwanese junior high school English teachers’ perceptions of the impact of the Basic Competence Test (BCT) in English on their curricular planning and instruction were affected by some teacher-related and context-related factors. In her study, teachers’ perceptions of the washback effects of the BCT in English were measured on the following six dimensions: syllabi, contents, teaching materials, classroom activities, teaching methods and classroom assessment, which altogether indicated the extent to which teachers perceived the washback effect of the BCT in English on their curricular planning and instruction. The major finding of her study shows that perceived students’ learning attitudes and teachers’ perceived awareness of the BCT, contributed significantly to the teachers’ perceptions of the impact of the BCT on teachers’ curricular planning and instruction. However, the results do not show exactly which of the six dimensions of teachers’ perceptions of the washback effects of the BCT in 22.