Literature Review
This chapter reviews the literatures of translation processes, translation methods, translation qualities and principles, translation strategies, pros and cons of multiple-choice (MC) and constructed-response (CR) tests, washback impact, and the criteria of assessing translation, all of which provide a better understanding of the evaluation of translation.
The Process and Methods of Translation
During the translating process, the language is first transformed in the brain, then produced in either spoken or written form. There are three processes involved—
understanding, expressing, and verifying (Ko, 1994; Chang, 2005). Understanding is based on the context, which can be a sentence, a paragraph or a whole article. The second phrase is to produce the translation. Expression refers to the target texts generated by the translator after what they have read from the source that has been internalized and
interacted with their mother language. The translators finally double check the translation to prevent mistranslating.
According to Shie’s (2002) study, the concept of ‘degree of freedom’ was one of the most important concepts that a translator had to know. He described four kinds of translation methods differentiated by the degree of freedom they were allowed:
1. Word-for-word translation is a direct translation of every word based on the most
general definition of the word in a bilingual dictionary, preserving the original word order.
2. Literal translation includes a response to the target language’s grammar. That is there is minimum adjustment made to word order and the addition or omission of words, but discourse is still disregarded altogether.
3. Semantic translation refers to that type of translation where the form of the original text is followed, and the intention of the author is maintained, but the translation is made natural and fluent.
4. Communicative translation requires the meaning of texts to be translated without concern for lexical or structural equivalents. Readability is of greater concern than faithfulness.
As noted above, the most basic translation process is comprised of two parts, understanding and expressing. However, only in open-ended (OE) or CR tests are test-takers permitted to give this type of answers. It is extremely rare that a person’s translation ability is tested by the MC tests. The four fixed choices neither allow the test-takers to express their own answers, nor the graders to really comprehend the current level of the examinee’s translation abilities.
Translation Quality and Principles
Quality has always been one of the top priorities in the discussions of translation. It is hard to judge which target text is a ‘good’ translation in comparison to a ‘bad’ or ‘poor’
translation. The criteria differ depending on the theoretical framework adopted to evaluate the target texts (Schäffner, 1997). There is no such a short-cut in translation.
There are mainly two kinds of translation principles, semantic translation and
communicative (Liou, 1993). The former emphasizes accuracy, which means that the translators have to translate the original text into the target language exactly without adding any extra content, and without intensifying or weakening any parts of the meaning. The latter stresses fluency, where the translator has to translate the source language into the target language as fluently as possible. In spite of their differences, they are not
necessarily exclusive. They are two sides of the same coin, because the more you want to maintain the original meaning of the source language, the less you keep of the way it is expressed in the target language.
In his book Evolution and Ethics and Other Essays, Yen Fu, one of the most
influential and authoritative translators as well as translation theorists in China, proposed three basic principles as the criteria for a good translation— ‘faithfulness’, ‘fluency’, and
‘elegance’ (Liou, 1995). These criteria have had a dramatic influence on the development of translation in practice and theory. Faithfulness requires the translator to be responsible
to the original text; the meaning in the target language also had to be faithful and pertinent to the meaning of the original one. Fluency refers to the intelligibility of the target language text; the translated text should be faithful to the language rules of the target language. Elegance, the most difficult principle to apply of the three, refers to the requirement that the translation should be esthetically pleasing. These are deemed the three highest principles for most Chinese translators, and can serve as grading criteria at the same time. Sih (1972) in his book Translation Study makes a clear list of seven crucial points for translation:
a. Translation should not be ordered and disciplined, and no extra content should be added. A good translation is like salt dissolved in water. You cannot see it, but it is there.
b. Do not simply try to translate words, but meanings.
c. An English word may have more than one meaning in Chinese. If you translate a word that you know into Chinese, but it seems inconsistent with the meaning of what you have translated so far, you may have misunderstood the meaning of the word.
d. When translating, consult more than one dictionary. No dictionary is perfect.
e. Double-check the translation to make sure there you do not miss any parts.
f. When faced with a difficult topic, translate it carefully. Ask experts or specialists advice.
g. Do not adopt words or idioms that you are not familiar with. (p.5)
In MC translation tests, the qualities are built in by the test designers. The examinee need simply to recognize the four choices. The highest assessment criteria are useless in this type of test.
Translations Strategies
Learning strategy use can help students learn foreign language skills including
listening, speaking, reading, writing, and translating more successfully. Oxford (1990), at the very beginning of her book, Language Learning Strategies, gives a brief definition of learning strategies:
Learning strategies are steps taken by students to enhance their own learning.
Strategies are especially important for language learning because they are tools for active, self-directed involvement, which is essential for developing communicative competence. Appropriate language learning strategies result in improved
proficiency and greater self-confidence. (p. 1)
Test-taking strategies, on the other hand, as defined by McDonough (1995) are “what candidates report as they try to solve various types of test item, what they pay attention to, and how they arrive at the right answer” (p.109). Cohen (1998) also gives a definition of test-taking strategies so as to determine the effects of the test input upon the test-takers, and the processes that the test-takers make use of in order to produce acceptable answers to questions and tasks, before, during, and after responding to them. They discuss an
individual’s approach to complete a test or, more specifically, the individual's way of
effectively and efficiently. Thus, the more strategies use during a test, the higher the achievement she or he would attain. While translating, people must first read and understand the meanings of the source language, then translate them into the target language. Each step includes strategies to help a translator translate the text more accurately.
Learning and assessing in language learning are not necessarily two separate
processes, but should be considered complementary processes (Cohen, 1996). Learning strategy has similarities to test-taking strategies in terms of language learning achievement.
Both strategies provide language learners with a great deal of confidence to continue their learning. If the notion that being a good language learner is to solve the problems coming from different learning situations, both inside and outside the classroom, is acceptable, then all people need to know is how to adopt appropriate either learning or test-taking strategies while confronting the problems. Over the past few decades, a gradual but noticeable shift in the focus of language research and instruction has taken place in the field of second language education. There is less emphasis on the teachers’ teaching and more on students’ self-learning. This change has been reflected in increasing numbers of studies undertaken on the learners’ perspective, particularly in research on language learning strategies. More and more foreign language educators have now recognized that effective learning strategies can enhance students’ efforts to reach their learning goals. Thus,
students are encouraged to ‘learn how to learn’, rather than to depend too heavily on their teachers’ instruction. These strategies encompass a wide range of learning behaviors that are designed to help learners become more autonomous, self-regulated, and goal-oriented, resulting in improved progress in developing language skills. Learning strategies are especially important for Taiwan’s English learners; most of them lack exposure to an authentic English learning environment in school. It is impossible for an English teacher to follow the learning path of each of their students inside and outside the classroom.
One possible way to turn this situation around is to help students develop effective learning strategies and become self-directed learners.
Many translation textbooks now include translation strategies designed to assist the translator complete translation tasks, such as adding, eliminating, omitting, or repeating words, changing the parts of the speech of words and so forth (Ko,1994; Liou, 2005;
Chang, 2005; Baker, 2006). Such methods empower students who are low achievers in terms of language learning. Teaching students strategies to pass tests is very useful since it is so important to pass school entrance exams. Kobayashi and Rinnert (1992) examined translation strategies used by Japanese college students to improve their English writing ability. Their results revealed that less proficient learners equipped with translation strategies showed significant improvement in their writing skills. On the contrary, more proficient learners only improved a little in terms of developing ideas, benefiting little from
the use of translation strategies. Thus, teachers should encourage less proficient learners to use translation strategies to improve writing or translation skills and for more proficient learners to refine their skills. Cohen and Brooks-Carson (2001) compared the French writing performance of 39 American university students in both direct writing and translation writing tests. One interesting finding was that the students’ direct writing performance was not as direct as it might be expected. Students claimed that they wrote by translating the ideas from the first language (L1) into the second language (L2)
knowledge. This suggests that translation strategies might, to some degree, be beneficial to the learners’ L2 writing. In another study, Wang and Wen (2002) analyzed the L1 use of 16 English majors in argumentative writing in China. Proficient writers tended to produce text directly in English, while less proficient learners constructed sentences through Chinese to English translation. Their findings imply that beginning writers could use translation strategies to improve their writing from the L1 to L2 translation pattern to direct L2 construction patterns. However, all these strategies require translators to elaborate with the chance of expression in a translation task. Obviously a test with four pre-given choices for descriptions of a translation does not offer opportunities for
examinees to use these strategies.
Pros and Cons of MC and CR Tests
The basic form of an MC question has two main parts, stem and options. The
question is stated in the stem, and options offer answers for the test-takers to choose from.
Generally speaking, an option has four items, one of which is correct, the others being distracters. Perhaps the most obvious advantage of MC testing is that the scoring is perfectly reliable, rapid, and economical. It is a good test technique aimed at eliciting the performance from the learners. From Hughes’s (2003) point of view, there are four techniques that:
1. will elicit behavior which is a reliable and valid indicator of the ability in which we are interested;
2. will elicit behavior which can be reliably scored;
3. are as economical of time and effort as possible;
4. will have a beneficial backwash effect, where this is relevant. (p.75)
The popularity of MC tests can be attributed to their rapid and reliable scoring. MC tests allow a large amount of material to be tested in a small amount of time, whereas CR tests often require a large amount of time for the test-taker to write out the responses. The grading of MC is also quicker; most tests are now graded by machine. Examinees are graded purely on the selection of one of the pre-determined responses; the test does not allow the grader to introduce personal bias or misinterpretation. Buckles and Siegfried
(2006) pointed out that the advantages of good MC tests include low grading costs, perceived objectivity, and availability of comparative statistical analysis. However, well-designed CR tests can assess the student’s level of understanding better than MC tests.
Another reason for criticizing the adoption of MC testing items can be ascribed to the possibility of guessing on the test. Although the advantages of MC tests are great, there are also weaknesses and deficiencies (Walstad & Becker, 1994; Yunker,1999; Yu, 2002;
Hughes, 2003; Brown, 2004). Hughes (2003) summarizes the following six weaknesses of MC tests:
1. The technique tests only recognition knowledge
2. Guessing may gave a considerable but unknowable effect on test scores 3. The technique severely restricts what can be tested
4. It is very difficult to write successful items 5. Backwash may be harmful
6. Cheating may be facilitated. (pp.76-78)
Yu (2002) has also pointed out several problems intrinsic to the adoption of MC tests:
1. Before being tested, a test-taker may try hard to memorize the answers of some difficult questions so that he or she can easily answer the questions, which perhaps go far beyond his or her current understanding.
2. The scores may not reveal the authentic ability of a test-taker with high test-taking
strategies but low language ability.
3. Some low language achievers guess correctly than what their real abilities are. (p.138) Yunker (1999) has also stated clearly the drawbacks of using MC testing:
Multiple choice test items tend to measure only a very narrow sample of content at a specific point in time and usually require only superficial recognition of information to answer correctly. Items are often constructed to measure factual rather than applied knowledge of objectives, and they are limited to having only one correct answer. The rigid objectivity of multiple choice test items is a flaw when divergent thinking skills are activated and when students’ perceptions differ from those of the instructors. (p. 82)
It should also be noted that the key to correctly designed MC tests is determined not by the stem, but by the ‘distracters’ (Yu, 2002). If a distracter can be too easily
distinguished from the correct answer (owing to its infeasibility), then the function of discrimination is lost; the test-takers can easily rule out the distracters, selecting the correct answer without actually knowing the subject matter. Hughes (2003) contended that only recognized knowledge can be tested in MC test. That is to say, a gap still exists between knowledge and practice. There are tips especially for taking MC language tests. The main purpose of these tips is to make it easier for the examinees to select the correct answer if they fail to figure out the answer based on their own knowledge. For example, guessing is useful if there is no penalty for wrong answers. The elimination of clearly wrong answers improves their chances. There is a 25% chance of guessing the right answer in an item with only four choices. If a test-taker can cross out one as clearly
certain items simply by focusing on one word or idiom they know in the stem. If equivalent counterparts of those words or idioms do not appear in the choices, then they can be easily recognized as wrong answers. Clearly, it is not the examinees’ translation abilities that are being tested, but merely their recognition of words or idioms.
These kinds of problems can be avoided however by testing translation ability with CR tests. The test-taker has to take into account aesthetics, rhetoric, semantics, syntax, and socio-cultural differences. CR tests require them to integrate all these factors when translating not just choose from pre-given items, nor rule out the distracters. The outcome is closer to the candidate’s genuine translation abilities. In addition, when an MC translation test is assessed, neither teachers nor students receive information from the wrong answers on which to base future improvements.
In one study, Chan and. Kennedy (2002) made a comparison between equivalent MC and CR exam questions on economics. They found out that for some certain types of MC tests, students did indeed score better than on ‘equivalent’ CR tests. Similarities can be seen in O’Neill’s (2001) research. He explored the impact of test type (i.e.,
constructed-response versus fixed-response) on student achievement and attitudes within a Principles of Macroeconomics course. The results of the analysis indicate that test type also impacted achievement. Bridgeman (1992) compared quantitative MC and OE test questions, finding that the presentation of quantitative items in an OE response format
offered at least three major advantages over the MC format. First, the measurement error could be decreased measurement because of the eliminating of random guessing. This was particularly valuable in an adaptive testing situation where various thoughts could be formulated in relation to the topics. Second, the unintended corrective feedback inherent with MC items was reduced. If the examinees guessed that the answer was not among the choices, they might realize they were making an error and try a different strategy to find the correct answer. The third advantage of OE quantitative items is that the problems cannot be solved simply by working backwards from the answer choices. This last advantage makes the test more like the problems the students have to solve in their academic studies. In his study, Bridgeman concluded that there are striking differences between the OE and MC formats, even at the level of the individual item. Some items that are relatively easy in the MC format are relatively difficult in the OE format. If the test is intended to describe specific skills that the students possess, the OE format clearly seems to be superior. Cheng (2004) made a comparison of MC and OE response formats for the assessment of listening proficiency in English. In her study, she pointed out that different types of responses have a significant effect on the subjects’ performance.
Subjects performed best on the MC tests and scored lowest on the OE tests. The reason that the majority of the subjects preferred the MC format was that the answer alternatives given facilitated the comprehension of spoken stimuli and it was easier to guess. Many
examinees were unsure how to choose a correct answer when the correct answers were not clearly certain in MC tests. If they were not sure of the correct answer but had enough knowledge to be able to identify one or more choices as incorrect, their chance of getting the right answer was improved. Nevertheless, the higher the probability of guessing the right answer, the less is the examinees’ translation ability verified. Hence, if we want to
examinees were unsure how to choose a correct answer when the correct answers were not clearly certain in MC tests. If they were not sure of the correct answer but had enough knowledge to be able to identify one or more choices as incorrect, their chance of getting the right answer was improved. Nevertheless, the higher the probability of guessing the right answer, the less is the examinees’ translation ability verified. Hence, if we want to