國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
13
CHAPTER 2
LITERATURE REVIEW
This section presents an overview of the theoretical background of washback research, including the discussion on the nature of washback, and how washback works. Finally, the studies related to the BCET washback effects are reviewed.
The Nature of Washback: Positive vs. Negative
The term washback itself is neutral, which has the potential to become either positive or negative (Alderson & Wall, 1993; Bailey, 1996; Hamp-Lyons, 1997;
Hughes, 2003). For example, if the use of oral interview in a proficiency test encourages learners to practice communication skills, then the effect can be seen as positive washback. However, if the skill of speaking is tested only by reading aloud, and learners overlook other important oral production skill such as conversational exchanges, then its washback effect on learning is limited.
Negative Washback
The undesirable effects of testing on teaching and learning are referred to as negative washback. Negative impact of testing has long been identified as a
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
potential problem in language education. From the literature, a number of harmful washback effects have been observed. They include: narrowing the curriculum and the educational process (Alderson & Hamp-Lyons, 1996; Cheng, 2005; Madaus, 1988;
Shohamy, 2001; Qi, 2005; Spolsky, 1995; Wall & Alderson, 1993), increasing anxiety among teachers and students (Ferman, 2004; Shohamy et al., 1996), additional
test-preparation classes or tutorials (in addition to or in place of other language classes) (Alderson & Hamp-Lyons, 1996; Bailey, 1996; Shohamy et al., 1996), reduced emphasis on skills that require complex thinking or problem-solving (Alderson & Hamp-Lyons, 1996; Qi, 2007), and demeaning teachers’ professional judgments (Madaus, 1988).
Another example of negative washback is that testing may lead students to engage in inappropriate learning processes and fail to bring about expected learning outcomes. It is referred to in the educational measurement literature as test score pollution (Alderson & Hamp-Lyons, 1996; Haladyna, Nolen, & Hass, 1991). Test takers’ scores increase, but their ability being measured in fact does not develop to that extent. Test score gains may be a result of rote memorization of material, instead of acquiring the target construct. This may result from the influences that are irrelevant to the construct of interest, and thus contaminate the interpretations based on test scores.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Considering construct validity, Messick (1996) argued for avoiding two major threats to a test’s validity: construct under-representation and construct-irrelevant
variance, both of which may result in the negative washback effects mentioned above.
The former occurs when the test content does not adequately reflect the construct as defined for the test. For example, if a test claims to assess communicative
competence but does not measure speaking or writing abilities, then it would under-represent the construct of communicative competence. On the other hand, construct-irrelevant variance refers to the variance of test scores that is due to factors other than the construct of interest. For instance, if a communicative language test requires test takers to perform decontextualized grammar analyses, teachers might put emphasis on irrelevant tasks instead of helping them acquire communicative
competence.
Many writers have noted that if the test content and test formats fail to reflect the curriculum goals or the course objectives to which they are supposed to be related, then negative washback effects on students’ learning processes and behaviors are likely to occur. Moreover, a test may be seen as lack of validity due to its negative washback effects on teaching and learning.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Positive Washback
Beneficial impact of a test on teaching and learning is referred to as positive washback. Positive washback has been viewed as an important criterion in the development and evaluation of language tests in recent years, and a number of suggestions have been made for ways to promote positive washback. Among these suggestions, communicative methods of language testing are referred to most
frequently. As Bailey (1996) pointed out, one distinctive feature of communicative language tests is the emphasis on promoting positive washback. Communicative language tests are aimed to establish a closer link with the principles of
communicative language teaching. In other words, communicative language tests reflect the language learning goals of the test takers and they are designed based on precise specifications of the learners’ proficiency. Moreover, test-takers are better informed about test results. Descriptive score reporting, rather than a single score, is offered so that test-takers would better interpret their performance and problem areas, which can potentially promote the occurrence of washback on test-takers’ subsequent language learning.
Swain (1985) proposed “work for washback” as one of the four general principles relevant to the design of communicative language tests (p. 36). Bailey (1996), in a review of relevant literature on washback, suggested four areas that foster
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
beneficial washback. They are “the incorporation of (1) language learning goals; (2)
authenticity; (3) learner autonomy and self-assessment; and (4) detailed score reporting” (p. 268).
Similar advice was given by Shohamy (1992) in her diagnostic feedback model for assessing foreign language learning. In addition to the needs for using direct methods and authentic language tasks, she argued for involving teachers and administrators in the planning and decision making of the testing process because
they are the people who will have to make changes. Shohamy particularly stressed the importance for providing “detailed, innovative, relevant and diagnostic”
assessment information that addresses “a variety of dimensions rather than being collapsed into one general score” (p. 515).
Messick (1996) linked the characteristics of communicative language testing—authenticity and directness—to the properties that most likely induce
positive washback. He considered them as safeguards against two possible threats to a test’s validity: construct under-representation and construct-irrelevant variance.
The former leaves out important features of the measured construct; the latter includes features irrelevant to the interpreted construct. Messick maintained that tests with positive washback are likely to include tasks which are “authentic and direct samples of the communicative behaviors of listening, speaking, reading and
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
writing of the language being learnt” (p. 241). He argued that for a test with
beneficial washback, exercises for learning and testing should match to a great extent.
Also, learning practices for mastering the language and learning practices for preparing for the test should be similar.
Hughes (2003) encouraged test developers, either language teachers in classroom situations or test designers in testing agencies, to work toward achieving positive washback by using the following principles (pp. 53-56):
1. Test the abilities whose development you want to encourage.
2. Sample widely and unpredictably.
3. Use direct testing.
4. Make testing criterion-referenced.
5. Base achievement tests on objectives.
6. Ensure the test is known and understood by students and teachers.
7. Where necessary, provide assistance to teachers.
As one might have noticed, some of the recommendations listed above, such as direct testing of abilities and teacher training, are costly in terms of time and money, and this might make a test impractical. Hughes, however, taking practicality into account, still highly values tests that facilitate beneficial washback. He contends
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
that it is certainly sensible and worthwhile to develop a test having positive washback, given the waste of time and effort in teaching and learning practices that may be inappropriate to the goals of language education.
The discussion about achieving positive washback has centered on the
inclusion of features of communicative testing in test design and test content, such as designing the test to measure what the language program intends to teach, ensuring that important stakeholders understand the purpose of the test, using authentic tasks, employing direct testing, and providing detailed reporting of test results.
The Nature of Washback and Context
Like most studies conducted in educational contexts, a complex interplay of various contextual factors seems to be involved in the process of generating washback.
Alderson and Wall (1993) maintained that the quality of washback, whether classified as positive or negative, might be independent of the quality of a test. Washback is
mediated by complex interrelationships between testing, teaching and learning.
Such complexity was highlighted by Messick (1996) as well: “a poor test may be
associated with positive effects and a good test with negative effects because of other things that are done or not done in the educational system” (p. 242). Whether the effect of testing on teaching and learning is regarded as positive or negative should
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
depend largely on where and how it exists and manifests itself within a particular educational context, as most empirical washback studies have demonstrated. This understanding implies the importance of unraveling these contextual factors when the researcher attempts to describe and explain washback effects.
The Washback Mechanism
In order to investigate washback phenomenon, it is important to turn our
attention to the mechanisms through which washback operates. Three models of examining how washback works will be illustrated: (a) Alderson and Wall’s (1993) Washback Hypothesis; (b) Hughes’ trichotomy (1993, as cited in Bailey, 1996); and (c) Bailey’s (1996) model of washback.
Alderson and Wall’s Washback Hypothesis
Alderson and Wall (1993), based on the findings of their own washback study in Sri Lanka, focused on micro aspects of teaching and learning influenced by tests.
Given the scarcity of empirical research into washback, they put forward a series of washback hypotheses, ranging from the most general to the more specific (pp.
120-121):
1. A test will influence teaching.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
2. A test will influence learning.
3. A test will influence what teachers teach; and
4. A test will influence how teachers teach; and therefore by extension from 2 above, 5. A test will influence what learners learn; and
6. A test will influence how learners learn.
7. A test will influence the rate and sequence of teaching; and 8. A test will influence the rate and sequence of learning.
9. A test will influence the degree and depth of teaching; and 10. A test will influence the degree and depth of learning.
11. A test will influence attitudes to the content, method, etc. of teaching and learning.
12. Tests that have important consequences will have washback; and conversely, 13. Tests that do not have important consequences will have no washback.
14. Tests will have washback on all learners and teachers.
15. Tests will have washback effects for some learners and some teachers, but not for others.
In summary, a test can influence learning as well as teaching in terms of what to learn and teach, how to learn and teach, the rate and sequence of teaching and learning, the quantity and quality of teaching and learning, and attitudes to the content
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
and methodology of teaching and learning.
The Hughes Trichotomy Framework
Hughes (1993, as cited in Bailey, 1996), in an unpublished paper, suggested a model of washback on the basis of a distinction between three elements in an
educational system: participants, processes, and products. Participants include students, teachers, administrators, materials writers and publishers, and process is defined as any actions taken by participants which may contribute to the process of learning. Lastly, product refers to the outcomes and the quality of the learning.
Hughes explained his model as follows:
The nature of a test may first affect the perceptions and attitudes of the participants toward their teaching and learning tasks. These perceptions and attitudes in turn may affect what the participants do in carrying out their work (process), including practicing the kind of items that are to be found in the test, which will affect the learning outcomes, the product of that work. (p. 2, as cited in Bailey, 1996, p. 262)
For the present study on the BCET washback, it may be assumed that students’
perceptions of learning English may first be influenced by the BCET itself, which in
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
turn affects how they are involved in learning activities, and then in turn affects ultimate learning outcomes.
Bailey’s Model of Washback
By connecting Hughes’s (1993) ideas of tripartite distinction and Alderson and Wall’s (1993) list of hypotheses mentioned above, Bailey (1996) proposed a basic
model of washback of three major categories: participants, processes, and products.
Her model illustrates that a test may affect products, i.e., learning, teaching, materials and research findings, through the participants and the processes the participants engage in. Furthermore, these outcomes may in turn provide feedback to the test and bring about changes to the test (see Figure 2.1).
‧
PARTICIPANTS PROCESSES PRODUCTS
Figure 2.1
A Basic Model of Washback (reprinted from Bailey, 1996, p. 264)
The model also allows for possible influences on a test from participants and certain products, as dotted lines indicate. In the context of current study, the BCET may directly affect the students, who, in turn, engage in the processes that will eventually lead to the learning outcomes, including their actual language development in English, and their performance on the BCET.
Bailey also categorized the impact of tests into two dimensions: washback to the learners, and washback to the program. The former refers to the direct impact of the test on learners as test takers, and the latter refers to the impact on teachers, administrators, curriculum developers, and counselors. Bailey notes that this
Students
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
distinction is consistent with Alderson and Wall’s washback hypotheses mentioned above. Washback to the learners refers to hypotheses 2, 5, 6, 8, and 10; while washback to the program addresses those of 1, 3, 4, 7, 9, and 11. Furthermore, she particularly points out that washback to the learners is a relatively under-explored area, according to her review of related literature.
Studies on BCET and BCT Washback
Two washback studies of the BCET in Taiwan EFL context are found to date.
One is the doctoral dissertation by Chen (2002), and the other is the master’s thesis by Huang (2004). They will be discussed below.
Chen (2002), using questionnaires and focus group interviews with junior high school English teachers, investigated how the teachers perceived the washback effects of the revised public examination, the BCET, on their curricular planning and
instruction. It was found that the BCET has considerable effect on teachers, but this impact is quite superficial; namely, the BCET may affect teachers what to teach, but not how to teach. It seemed highly probable that changes to the teaching content were caused by the 1997 revised English mandated textbook based on the Junior High School Curriculum Standards in English education, issued by the MOE in 1994.
The study indicated that most junior high school English teachers lack knowledge of
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
how to adjust their teaching methods in alignment with the revised curriculum which emphasized communicative approaches to teaching and learning. This study offers insights into the BCET washback on teachers and their attitudes at the very beginning of introducing this reformed entrance examination.
The other BCET washback study by Huang (2004) explored the BCET washback effects in terms of teaching materials, teaching methods, classroom
assessment, and student learning. A set of 82 teacher questionnaire and 351 student
questionnaire was distributed across Taiwan. Although the study focused mainly on teachers’ perceptions of the BCET in relation to their beliefs and attitudes, students’
perspectives were also investigated. These data provided rich information for the present study; however, the data gathered might have been more useful if the author
had used four-point Likert scale, rather than five-point scale, for the student questionnaire. The students’ responses indicated that most of them hold neutral opinions, probably due to the inclusion of “undecided” options. Also, the wording
of some items is biased. For instance, one of the questions is stated: “The BCET does not assess my writing ability; therefore, I do less writing exercise” (p. 103).
These areas provide valuable insights into the design of the questionnaire for the present study.
Two relevant BCT studies which may contribute to the understanding of the
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
effects of BCT on teaching and learning are worth mentioning here. Yu, Lai, and Liu’s (2004) national survey of students found that the administration of BCT does
not reduce the pressure of entrance exam, nor does it help mitigating the problem of private coaching. The issue of fairness was raised as well, and most respondents suggested that the BCT tends to favor certain groups of test takers.
Yu, Lai, and Liu (2005) also conducted a nationwide survey study on the BCT from the points of view of high school administrators and senior teachers. It was found that the administrators and teachers doubt that the BCT measures students’ true competencies effectively. They argue that the results of the BCT do not accurately indicate students’ academic competence. Comparing the students’ and the teachers’
views, the researchers found different perceptions of the BCT test format. High school senior teachers and administrators suggest that the BCT should include new item types, like writing and listening, whereas most students prefer to maintain current multiple-choice format.
To conclude, Yu, Lai, and Liu’s (2004, 2005) studies explored the BCT’s effectiveness in terms of its test content and test format, the use of scale scores, and fairness, etc. from both learners’ and teachers’ perspectives. Although the voices of students as test takers were heard in these studies, the effect of testing on learning was addressed to a little extent. As for the BCET washback studies reviewed previously,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chen’s (2002) study emphasized teachers’ views and teaching practices, which have
been primary research attention in washback studies (Bailey, 1996; Cheng, 2007;
Hamp-Lyons, 2000; Wall, 2000). Huang’s (2004) study tended to focus on teachers’
perceptions, rather than students’ voices, since the number of the student sample was slightly small. It would seem, therefore, that further investigations are needed in order to know more about washback effects on students.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
29
CHAPTER 3
METHODOLOGY
The current survey study aimed to investigate the washback effect of the BCET on junior high school students in northern Taiwan. For this purpose, a
questionnaire survey was developed as data collection instrument, since it allowed the researcher to collect a large amount of information from a variety of students and to process the data in a more efficient way. In the following sections, the rationale for selecting participants will be explained first. Then the design of questionnaire and procedures for collecting and analyzing survey data will be illustrated.
Selection of the Participants
The target population is the graduating final-grade junior high school students (i.e., 9th-graders), for they are faced with the immediate BCET. The present study focused on the 9th-graders in northern Taiwan, which covers Keelung City, Taipei City, Taipei County, Taoyuan County, Hsinchu City, and Hsinchu County. They account for approximately 40% of the 9th-graders all over the country (MOE, 2008, 2009).
All the junior high schools and integrated junior-senior high schools in northern Taiwan were further classified according to their locations. Four
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
administrative levels were employed to select sampled classes:
(a) National level (直轄市) : i.e., Taipei City, which is under the direct governance
of the central government of Taiwan;
(b) Municipal level (市) : e.g., Keelung City, Hsinchu City, Tucheng City in Taipei
County, and Taoyuan City in Taoyuan County;
(c) Town level (鎮) : e.g., Jhudong Town in Hsinchu County, and Yangmei Town in
Taoyuan County; and
(d) Village level (鄉) : e.g., Shenkeng Township in Taipei County, and Luchu
(d) Village level (鄉) : e.g., Shenkeng Township in Taipei County, and Luchu