Chapter 2 Literature Review
2.2 Measuring Interpreting Quality
18
interpreting quality ranges from accurate rendition of source, adequate target language
expression, equivalent intended effect, to successful communicative interaction. This
model summarizes the fact that good interpretation has multiple-level of definitions,
from the “lexico-semantic core” at textual output level to the “socio-pragmatic sphere of
interaction” (p. 413).
2.2 Measuring Interpreting Quality
2.2.1 Different Quality Criteria
To assess interpreting quality empirically and to actually measure quality standards,
there have been many endeavors and debates about research methods. One of the most
common empirical studies on interpreting quality has been survey research, as
compared to experimental studies, corpora analysis or case studies (Pöchhacker, 2002).
One of the pioneering survey-based research works started from Büler (1986). Since
then researchers have developed and applied a range of quality criteria, and asked the
respondents to rate the criteria by order of significance. These criteria reflected
researchers’ endeavor to operationalize the construct of quality by breaking it into more
tangible components or objective attributes.
There are basically two main categories of quality criteria that have been widely
19
used. The first category is related to accuracy and focuses on linguistic, semantic,
product-oriented, or content-related criteria. These are the core criteria of interpreting
quality. This product-oriented perspective reflects the reasoning that good interpretation
means that the target-text is a “faithful image” (Gile, 1991, p. 198) or “exact, faithful
reproduction” (Jones, 1998, p. 5) of the source text. Some quality criteria that fall under
this category include fluency of delivery, logical cohesion of utterance, sense
consistency with original message, completeness, correct grammar, and correct
terminology. The second category emphasizes on style and looks at extra-linguistic,
pragmatic, service-orientated, or form-related standards. These criteria are more related
to “listener-orientation” and “target-text comprehensibility” (Pöchhacker, 2001, p. 413).
Take the following criteria for example, pleasant voice, lively intonation, poise, and
pleasant appearance. Interestingly, some criteria, such as fluency of delivery,
synchronicity, or native accent can be categorized as either content- or form-related,
depending on the researchers’ interpretation.
In addition, there can be a sub- or third category, which refers to personality traits
(Büler, 1986). These criteria include those such as thorough preparation, endurance,
reliability, and ability to work in a team. Note that these criteria cannot be directly and
20
explicitly observed or experienced by the audience.
2.2.2 Perceived Importance of Different Quality Criteria
There has been a constant debate about which category of quality criteria are
considered more important. In many studies (e.g. Büler, 1986; Marrone, 1993;
Kopczyński, 1994; Moser, 1996) there seemed to be a general consensus that
content-related criteria are deemed more important than form-related ones. The
linguistic criterion sense consistency with the original has constantly been given the
highest ratings, as well as the accuracy criterion. Meanwhile, form-related, delivery or
extra-linguistic criteria have been considered desirable, but not essential.
However, there are also some studies that found that users might consider more
important “superficial” criteria such as native accent, pleasant voice, and fluency of
delivery, as opposed to linguistic criteria. For instance, Kurz (1993) mentioned that
extra-linguistic criteria might be considered more important in certain situations or
contexts, such as media interpretation or in conferences filled with lively discussion and
spontaneous exchange. There are also some other possible explanations. Users do not
normally understand the source language and its linguistic quality, so they are more
likely to judge interpreting quality based on criteria that they can directly feel or
21
experience (Büler, 1986). Another explanation is that some users have less tolerance
listening to a halting interpretation, even if it is logically coherence and correct.
Although formal, delivery related criteria are not considered essential, they still affect
how the users evaluate the real performance of the interpreters (Garzone, 2002).
Looking at each specific quality criterion, one observes an even more complicated
picture. According to user comments in a study by Pöchhacker and Zwischenberger
(2010), the criterion synchronicity is more important in certain situations or in speeches
with certain features. Take for example speeches with jokes, punch lines, or lots of
numerical data. The criterion native accent is often considered less important or
unimportant. But in the same research by Pöchhacker and Zwischenberger, some survey
respondents expressed the viewpoint that the importance of native accent depends on
the target language. Another user in the same study pointed out that accent might be
related to prosodic quality (e.g. native intonation), which elevates the importance of
accent. These are only observational comments given by individual survey respondents.
However, empirical evidence can be found in a recent experimental study by Cheung
(2013). When judging three versions of an English-to-Cantonese SI differing only in
accent, participants rated the native accent SI quality more favorably than the two other
22
SI that had Mandarin and English accent respectively. The researcher cannot review and
discuss every single criterion in existing studies but the isolated examples listed here are
meant to highlight how and why some form-related criterion were attached significant
importance.
Whether content- or form-related, these quality criteria studies reflect researchers’
efforts to operationalize the construct of quality by breaking it into more tangible
components or attributes. The goal of many of these studies is not necessarily to reach
an absolute dichotomic conclusion about which category of quality criteria is more
important, but to highlight the fact that the discussion of interpreting quality should also
consider the type of interpreting event or assignment, who the respondents are, the
expectations of the respondents, how the respondents perceive the interpreter’s role, and
many other contextual factors.
2.2.3 Challenges in Measuring Quality
Despite the fact that the topic of interpreting quality criteria has been extensively
studied, there still exist quite a number of research challenges. As Moser-Mercer (2009)
pointed out in her article “Construct-ing Quality,” many existing studies were “overly
ambitious”, trying to cover too many attributes of quality in a non-specific way (p. 156).
23
When a study eliminates the situational context, it would create “a gap between ideal
(academic) quality an situated (real-world) quality” (Straniero-Sergio, 2003, p. 135). In
addition, the many quality attributes are often “suggested by researchers and not by
uses”—users mostly likely do not think in terms of quality (e.g. this is a good/bad
interpretation), but more in terms of other constructs such as comprehension (e.g. I
don’t understand what the speaker is trying to say) (p. 157). This causes a construct
validity problem. In addition, many quality criteria are not only subjective, but can also
be interpreted into different meanings. For example the criterion fluency of delivery can
be further characterized by frequent hesitations, pauses, or irregular, unnatural rhythms.
So when a respondent rates fluency of delivery as important, it might be due to a general
feeling or a certain characteristic that he/she thought about. To address this issue,
Macías (2006) used silent pauses as a sub-parameter of fluency to design an experiment.
Respondents were asked to rate the interpreting quality of three simulated simultaneous
interpretation videos, including the control video without additional silent pauses.
Findings indicated that silent pauses have a negative effect on the quality parameter of
fluency. Still, much research design and methodological work remains to be done to
ensure that the construct of quality is clearly measured.
24