CHAPTER THREE
In this research, an expe licit audience’s feedbacks to
diffe
.1 Pilot Survey
determine whether there is a consensus among established
ter
METHODOLOGY riment is designed to so
rent lengths of interpretation in a consecutive setting. In this chapter, the
pilot survey, research subjects, text design, experiment process, questionnaire
design, and data analysis will be discussed.
3
In order to
in preters that interpretation, in a consecutive setting, should take less time than
the original speech, a pilot survey was conducted. A questionnaire (see Appendix
B) was administered to nine instructors at three interpreters training institutes in
Taiwan--- Graduate Institute of Translation and Interpretation of National
Taiwan Normal University, Graduate Institute of Translation and Interpretation
Studies of Fu Jen Catholic University, and Graduate Institute of Interpreting and
Translation of National Kaohsiung First University of Science and Technology.
The questionnaire contains three parts. In the first part, the interviewees fill out
their basic information, such as where they receive their trainings, years of
experience in the industry, and years of teaching interpretations. In the second
part, the interviewees are asked about their perspectives on the length of
interpretation relative to that of the original in a consecutive interpreting setting.
In the third part, they are asked if they time students’ production in class.
3.2 Experiment
find out audience’s expectation of time ratio between
er
speeches with consecutive interpretations: one
in Ja
ersions of corresponding
cons
ation
proc
3.2.1 Framework
In order to
int pretation and its original, and to test it against interpreters’ perspective, an
experiment is designed to simulate real consecutive interpreting settings to seek
feedbacks from the participants.
The experiment includes two
panese, and the other in French. Both original speeches are divided into two
segments and each segment is around three minutes.
Each segment of the original speeches has two v
ecutive interpretations, one being thirty seconds shorter than the original
speech (around 80% of the time of the original speech), and the other thirty
seconds longer than the original (116% of the time of the original speech).
As a result, there are four possible combinations of implement
edures (as shown in Figure 3.1): (a) two French speeches with longer
interpretations and two Japanese speeches with shorter interpretations, (b) two
Japanese speeches with longer interpretations and two French speeches with
shorter interpretations, (c) two French speeches with shorter interpretations and
two Japanese speeches with longer interpretations, and (d) two Japanese speeches
with shorter interpretations and two French speeches with longer interpretations.
Each of the implementation procedure takes around 25 minutes, and such
com
Beginning
Figure 3.1 Experiment Design
Short CI : 2 minutes and 30 seconds of consecutive interpretation
Long CI : 3 minute pretation
binations eliminate the effect of different orders of speeches presented.
Therefore, the effect of the length of interpretation on the audience’s satisfaction
level can be better analyzed. In addition, the length of interpretation is limited to
25 minutes in order to obtain their responses before attentional lows occur.
End
s and 30 seconds of consecutive inter Source: Compiled by this research
FR Speech Long CI JP Speech Short CI
JP Speech Long CI FR Speech Short CI
FR Speech Short CI JP Speech Long CI
JP Speech Short CI FR Speech Long CI
Group One Group Two Group Three Group Four
The choice of language also helps to eliminate the possible effects of
languag
3.2.2 Sp ext Selection
l speeches are “Global Warming” for the
panese presentation and “L
. The researcher first e proficiency on quality assessment. One of the characteristics of
consecutive interpreting is that audience can listen to both the original speech and
the interpretation. However, the result of interpretation quality assessment is also
subject to one’s language proficiency; namely, the audience that understand the
source language may not share the same perspective of interpretation quality as
those who do not understand the source language at all. In order to remove the
effect of varying degrees of language proficiency and focus on the feedbacks of
the audience that has to 100% rely on interpretation for comprehension, two less
familiar languages among the participants, compared to English, Japanese and
French are selected for preparing the two original speeches.
eech T
The topics of the two origina
Ja ifelong Learning” for the French. The reason for
selecting these two topics is to avoid confusion and anxiety that could be incurred
by technical terms and insufficient background knowledge and to simplify the
analysis by eliminating the variable of domain knowledge.
Both speech texts were prepared by reverse-engineering
collected data from the Internet and composed two “speech texts” in Chinese, the
target language. The benefit of reverse-engineering is that the researcher could
control the number of words and information density of the Chinese texts. Then,
the two texts were translated into Japanese and French. The quality of
interpretation is not much of an issue in this study because the participants only
include those who do not speak French and Japanese.
As mentioned previously, there are two versions of Chinese interpretation
able 3.1 Word Count in Different Versions of Interpretation
from each speech, one being two minutes and 30 seconds long, and the other
three minutes 30 seconds long. The numbers of words in each version of
interpretation are similar, as shown in Table 3.3.
T
Length of Interpretation
2 Mins 30 Secs 3 Mins 30 Secs 595
( rds
(
words 769 wo
238 words/min) 219 words/min) C inh ese
Interpretation of Original
Japanese 608 words
(243 words/min) 793 words (226 words/min) 596 words
(238 words/min) 775 words (221 words/min) Chinese
Interpretation of
French Original 629 words
(251 words/min) 813 words (232 words/min) Source: Compiled by this research
As mentioned above, both the Japanese and French original speeches are
arou
3.2.3 Questionnaire Design
o parts: background information and research
t question of the background information section asks about the
part
uestions.
. P ale
2. a scale
nd three minutes long. Therefore, the length of the shorter consecutive
interpretation is around 80% of the original speech, and that of the longer
interpretation is around 120% of the original.
The questionnaire includes tw
questions. In background information, the first three questions are gender, degree
of education, and occupation. The fourth question requires research participants
to specify their language proficiency in French and Japanese. This helps to remove
unqualified samples, namely those who speak and understand either French or
Japanese.
The las
icipant’s experience of using consecutive interpreting service.
In the second part of the questionnaire, there are six research q
1 lease rate the overall performance of interpretation from Japanese on a sc
from 1 to 7, 1 being the lowest quality and 7 being the highest quality.
Please rate the overall performance of interpretation from French on
from 1 to 7, 1 being the lowest quality and 7 being the highest quality.
3. ed: slips
4.
n quality subject to the length of
6. ent of interpretation quality is subject to the length of
nderstanding of these questions, all participants are given an
ora
erall interpretation quality, the audience in fact
bas
Overall quality evaluation = language aspect valuation + time aspect evaluation (+ other factors)
If the factors gathered are ignored wh tation quality, then
it i
Which interpretation is better if the following factors are not consider
of tongue, expressions, number of pauses, fillers, and fluency.
Which interpretation takes more time?
5. Is your assessment of interpretatio
interpretation?
If your assessm
interpretation, which interpretation do you think is better considering the
length of interpretation the only evaluation criterion? (See Appendix C for
questionnaire)
To reduce misu
l explanation of consecutive interpreting before the experiment begins. (See
Appendix D for the introduction)
Presumably, when assessing ov
e their assessment on factors including the length of interpretation and those
gathered from the pretest. This can be conceptualized as
en assessing interpre
s assumed that the participants’ assessment is made based on the length of
interpretation only. The participants are also expected to make their assessment on
the basis of the length of interpretation, but the question is phrased with positive
listing. Such a design aims to:
1. compare the overall evaluation (Question One and Two) with an
2. rspective evaluation
3. entioned two differences in order to see whether
The abo
evaluation excluding the language aspect criteria (Question Three), and
the difference will reveal how the length of interpretation interacts other
quality assessment criteria through negative listing;
compare the overall evaluation with the time-pe
(Question Six), and the difference will also reveal how the length of
interpretation interacts with other quality assessment criteria through
positive listing; and
compare the aforem
there are additional unknown factors affecting interpretation quality.
ve three steps can be depicted in Figure 3.2:
Step 1:
tep 2:
tep 3
Figure 3.2 Cross-comp ix for Assessment Results Source: Compiled by this research
3.2.4 Pretest
experiment, a pretest is conducted in order to solicit comments on S
S
arison Matr
Before the
the interpretations from the audience’s perspectives. The comments are listed in overall quality evaluation
Q.1 / Q.2
language aspect Q.3
time
(other)
+
negative listing
overall quality evaluation Q.1 / Q.2
time Q.6
language
aspect (other)
+
positive listing
language aspect Q.3
time
(other)
+
other
language
(other) time
Q.6
aspect
+
the third question of the modified questionnaire as factors not to be considered.
In the pretest, 19 participants were invited to evaluate interpreters’
perf
al test questionnaire are ormance. They suggested that the Japanese-Chinese interpreter “paused too
frequently,” “stuttered a few times,” and “had a few awkward Chinese expressions
and sentence structures.” For the French-Chinese interpreter, they suggested he
“had a few slips of tongue,” “backtracked,” “had more fillers and pauses than the
Japanese-Chinese interpreter,” “paused at inappropriate places,” and “had some
unclear pronunciations.” These negative comments were categorized into five
general factors of “slips of tongue”, “expressions”, “number of pauses”, “fillers”,
and “fluency”, and listed in Question Three of the modified questionnaire, which
goes as follows: “Which interpreter’s performance is better if all the following
factors are not considered: slips of tongue, expressions, fillers, number of pauses,
and fluency?” The experiment participants have to exclude the aforementioned
factors when evaluating the performance of both interpreters. By doing so, the
participants would evaluate both interpreters’ performance based on one single
factor---time ratio, and exclude all other possible factors.
Except for the third question, other parts of the form
identical to those in the pretest questionnaire.
3.3 The Sample
enty participants are invited to participate in the
.4 Implementation Process
re are four possible combinations of different
leng
One hundred and tw
experiment, 30 for each version of experiment. All the participants are either
college students or holders of a higher education degree. In each version, 20
participants are students, and the other 10 are not. The purpose of such an
arrangement is to ensure an identical pattern of participant profile across versions.
3
As mentioned earlier, the
ths of interpretations from Japanese and French speeches. Therefore, there
are four types of videos and four versions of the questionnaire, as in Table 3.4.
Table 3.2 Four Versions of the Questionnaire
Combination of Speeches and Interpretations Version of
Questionnaire Part One Part Two
Group 1 3 min 30 sec Fr-into-Ch CI 2 min 30 sec Jp-into-Ch CI
Group 2 3 min 30 sec Jp-into-Ch CI 2 min 30 sec Fr-into-Ch CI
Group 3 2 min 30 sec Fr-into-Ch CI 3 min 30 sec Jp-into-Ch CI
Group 4 2 min 30 sec Jp-into-Ch CI 3 min 30 sec Fr-into-Ch CI Note: Fr = F
Source: Compile
renc c
d b
h, Jp = Japanese, CI = Conse y this research
utive Interpreting;
Most of the experiment takes place at the SI rooms at the Graduate Institute
of T
.5 Questionnaire Coding Assignment
s, each questionnaire is assigned a code
n, and whose
valu
length of interpretation, and ranslation and Interpretation of National Taiwan Normal University, though
some participants complete the experiment by using their desktop/laptop
computers or with AV device. Before experiment, each of the participants is given
a brief Chinese introduction (see Appendix D for the introduction and English
translation) to the nature, process, and purposes of the experiment. After the first
part of video, the participants are asked to rate the interpretation from
French/Japanese before continuing with the rest of the video. After finishing
watching all four clips of the video, the participants are given sufficient time to
finish the questionnaire.
3
Before analyzing the experiment result
according to two key parameters: correct/incorrect identification of length of
interpretation, and the participants’ evaluation subject/not subject to the length
of interpretation. Together, there are four types of possible results:
“A”: participants correctly identifying the length of interpretatio
e ation subject to the length of interpretation,
“B”: participants incorrectly identifying the
who
rpretation,
length of
inter
ing table presents four combinations of possible results which are
repre
able 3.3 Possible Combinations of Results
Correctly ID Length Incorrectly ID Length Interpretation
Unable to ID Length of Interpretation
se evaluation subject to the length of interpretation,
“C”: participants unable to identify the length of inte
“D”: participants whose evaluation is not subject to the
pretation.
The follow
sented as A, B, C, and D.
T
of Interpretation of Evaluation Subject to
Length of Interpretation A B C
Evaluation Not Subject to
Length of Interpretation D D D
Source: Compiled by this research
Based on the aforementioned possible results, each questionnaire is
m
nu bered and assigned a code. Take Group One for example. All the
questionnaires are numbered from 01 to 30. If the result of questionnaire 01 is
“A”, the questionnaire will be coded as illustrated in Figure 3.3 (For an example of
code list of each group, please see Appendix F).
a 01 a
.6 Data Analysis
ionnaires are coded, the response of each questionnaire is
recor
ion, a statistical analysis will be conducted in Excel to see if the
cont
Group One No. 01 Type “A “
Figure 3.3 Coding Assignment Source: Compiled by this research
3
After the quest
ded in an Excel spreadsheet. The number of each type of participants and
their preferences are calculated. The majority’s preference is compared with that
of interpreters to see if their preference of length of interpretation agrees with
each other.
In addit
ent of interpretations and the order of video presentations affect the
audience’s perspective on interpretation quality.
