3.6 Data scoring and analysis
3.6.2 Analysis of omissions and substitutions
Four categories of errors were employed in this study: omissions of words (OW),
omissions of segments (OS), substitutions of words (SW), and substitutions of segments (SS).
An omission is defined as a word or a segment that has not been translated in the target
68
speech, which consequently alters the meaning of the source speech. A substitution is defined as a word or a segment that has been inaccurately translated in the target speech, which consequently alters the meaning of the source speech. Under these two general categories of error, a distinction is made between the unit of “word” and “segment.” Due to linguistic and cultural differences between two languages, word-for-word translation is not feasible in
interpreting. Therefore, it would not be appropriate to use a single word as the unit of analysis.
For the purpose of this study, a noun with a determiner (a country; the country), a short
phrase such as prepositional phrases (in addition to; in front of), adverbial phrases (as a result) or verb phrases (is entitled to) and proper nouns (the United Nation) were all treated as
“words” instead of “segments” in the analysis. A “segment,” on the other hand, consists of at least two content words (nouns, verbs, adjectives, and adverbs). Examples of each category of error are presented as follows:
Omissions of words (OW)
Speech A S004A100
We can expect another 2.5 billion people /7 我們可以預期二十五億的人 (We can expect 2.5 billion people) to move from the countryside(可不譯) into
the cities by 2050(OW). /8
會從鄉間移到城市當中
(would move from the countryside into the
cities.)
Speech B S006B100
I am extremely concerned about the many children and young people who
smoke(OW)./35
我非常的擔心許多的青少年還有孩童 (I am extremely concerned about many
young people and children.)
In the first example, the adverbial phrase “by 2050” was omitted in the target output. Since the phrase “by 2050” does not consist of two content words, “OW” was assigned. In the second example, the relative clause “who smoke” was also omitted in the target output. Since the word “who” is a pronoun and a function word, “OW” was also given.
69
Omission of segments (OS)
Speech C
And while we are yet to experience a full-scale cyber incident(OS)/54 like we’ve seen in other countries, /55
New Zealand is not immune to them(OS)./56
S025C160
而我們在其他國家也看到這種網路攻擊事 件(54) (In other countries we have seen this
kind of cyber incidents)
For unit 54, the segment “And while we are yet to experience a full-scale” was missing in the target output, so “OS” was assigned. There was no corresponding target segment for unit 56 of speech C, so it was a complete omission.
Substitution of words (SW)
because you’re passionate about(SW) New Zealand’s economy /5
S013B130
那其實呃吸菸每年造成非常多的歐洲人死 亡 (In fact, smoking every year kills many
Europeans.)
S006C130
那麼各位會來參加 (You come here)
是因為對紐西蘭的經濟有信心 (because in New Zealand’s economy you
have confidence.)
In the first example, “非常多” (many) was produced in the target speech instead of the precise number of “650,000.” In the second example, the verb phrase “passionate about” was incorrectly translated as “有信心” (have confidence in). The word “about” is a function word.
70
So “SW” was given in both examples.
Substitution of segments (SS)
Speech A
At least 6% of national healthcare spending(SS)/17
is eaten up by tobacco-related diseases(SS),/18
S019B130
而且有百分之六的經費 (6% of the fund)
是要花在防制菸害上
(will be spent on tobacco control)
For unit 17 in speech A, “national healthcare spending” was simplified as “經費” (fund) in the target speech, so it was regarded as an error of SS. For unit 18, the interpreter first
inaccurately translated a factual statement into an action that is going to be taken in the future by adding the Chinese word “要” (will) and then misinterpreted “tobacco-related diseases” as
“防制菸害” (tobacco control). Therefore, “SS” was given to the whole unit.
The scoring of omissions and substitutions was conducted by comparing the source speech with the Chinese output transcripts unit by unit. An error was marked on the source speech transcript by highlighting the words or segments in gray and putting the error category in parentheses, as shown in the above-mentioned examples. Each meaning unit in the source speech was given only one error category. When there was more than one “word” that has been omitted or substituted, “OS” or “SS” was assigned. When there was a combination of
“OW” and “SW” in the meaning unit, “SS” was given.
The researcher served as the first rater. A professional interpreter with working
languages of English and Chinese was invited to serve as the second rater. The second rater is also an instructor of SI from English into Chinese at one of the graduate institutes of
translation and interpreting in Taiwan. The rater training session was conducted via Skype.
The researcher first explained the scoring rubrics (See Appendix E) to the second rater,
71
including the definition of each error category, the demarcation of words and segments as wells as the scoring procedure. The first participant’s Chinese output for speech A
(S001A100P) in the pilot study was used in the training session for scoring practices. Then the scoring sheet of S003C130P was provided to the second rater for independent scoring. We met online again to discuss the results. For S001 to S012 in the formal experiment, the
researcher and the second rater scored independently. The scoring sheets and recordings of the participants’ SI performance were provided to the second rater. For S001 to S004, we discussed differences and achieved consensus after marking all scoring sheets for speech A and after marking those for speech B and C. Then, we started the individual scoring for the output of S005 to S012 (24 scoring sheets for each rater). After the scoring was completed, the researcher checked the scoring sheets to see if there was any error mark without an error category and if there was any inconsistency of scoring within a rater (e.g., a word in the output was considered an error in one participant but not in others). The second rater was asked to add those missing error categories and made scoring adjustments for areas with intra-rater inconsistency or editing problems (such as misalignment between the source unit and the target segment). The researcher then went on to finish the scoring for S013 to S028.
The inter-rater reliability was assessed by conducting correlation analysis on the scoring from S005 to S012 (24 scores for each rater). Pearson correlation analysis was performed in SPSS 24 for Windows in order to measure the degree of agreement between the two raters on four error categories: OW, OS, SW and SS. Table 3.11 presents the descriptive statistics of scores from the two raters. Table 3.12 presents the results of correlation analysis. The correlation coefficient ranges from .54 to .96, all reaching .01 significant level.
72
Table 3.11 Means and Standard Deviations for Each Error Category
OW OS SW SS
M SD N M SD N M SD N M SD N
Rater1 4.41 2.32 24 23.50 9.38 24 5.25 2.47 24 18.25 6.83 24 Rater2 3.95 2.51 24 24.54 9.53 24 5.87 2.77 24 20.00 7.10 24
Table 3.12 Correlation Coefficient of Two Raters on Four Error Categories
OW OS SW SS
correlation .66** .96** .54** .89**
Note. ** p <.01
The correlation of OW and SW was comparatively lower than that of OS and SS. The inconsistency of ratings on OW and SW probably stems from the following two reasons. First, there is often times more than one way to translate the same meaning in the source text.
Besides, since English and Chinese are two typologically distant languages, there is less mapping in meaning between words in these two languages compared with languages that belong to the same family (e.g., English and German) (Tseng, Chang, & Tokowicz, 2014).The following example illustrates how the two raters scored differently for two meaning units in the source text.
Speech A S005A130
Rater 1 We can expect another(OW) 2.5 billion people /7
to move from the countryside(可不譯) into the cities by 2050. /8
未來更將有二十五億人 (In the future there will be 2.5
billion people)
在 2050 年的時候都會住在城 市裡
(in the year 2050 all living in
cities.)
73
Rater 2 We can expect another 2.5 billion people /7
to move from the countryside(可不譯) into the cities by 2050. /8 (SW)
未來更將有二十五億人 (In the future there will be 2.5
billion people)
在 2050 年的時候都會住在城 市裡
(in the year 2050 all living in
cities.)
Both raters agreed that the meaning of “another” in the source text is an important concept that cannot be omitted in the target text. Since “another” was not translated in the output of S005, it was considered an error of OW by the first rater. The second rater, on the other hand, thought that the verb “to move” in the source text was inaccurately translated as
“living” in the output of S005, so it was an error of SW. Either way of scoring was based on the meaning of the original message that the population of cities will increase by 2.5 billion people by 2050. Therefore, both methods are acceptable.
Second, the two raters were not equally severe for the scoring of certain words in the source text, as shown in the following example.
Speech A S012A160
Rater 1 I am also reflecting on possible ways of regulating access to tobacco in a more stringent manner /55
to limit the exposure of minors(SW) to tobacco products./56
又或者我們是不是要更仔細 來控管香菸的販售
(Or maybe we should more
carefully regulate the sales of tobacco)
讓年輕人不要那麼容易能夠 接觸到香菸
(so that young people will not
have easy access to tobacco)
Rater 2 I am also reflecting on possible ways of regulating access to tobacco in a more stringent manner /55
又或者我們是不是要更仔細 來控管香菸的販售
(Or maybe we should more
74
to limit the exposure of minors to tobacco products./56
carefully regulate the sales of tobacco)
讓年輕人不要那麼容易能夠 接觸到香菸
(so that young people will not
have easy access to tobacco)
For the first rater, “minors” in the source text was inaccurately translated as “young people”
in the Chinese output. However, for the second rater, it was an acceptable translation. Here is another example.
Speech A S012A160
Rater 1 In the meantime, the European Commission is working with the Member States of EU/59 to update the health warnings that appear on tobacco packages./60
how to design on the tobacco package the health warnings.)
Rater 2 In the meantime, the European Commissionis working with the Member States of EU/59 to update(SW) the health warnings that appear on tobacco packages./60
Commission has already been with the Member States
brainstorming and discussing how to design on the tobacco package the health warnings.)
In this example, to translate the verb “to update” in the source text as “to design” in the output was considered an inaccurate translation for the second rater. However, it was
75
acceptable for the first rater because the following context had specified how the design of health warnings would change, which was correctly translated by this participant.
The results in Table 3.12 indicate a high correlation for OS and SS and a moderate correlation for OW and SW, according to the interpretation of Guilford (1956).4 These results and the above-mentioned examples suggest that it is much more difficult to judge the
accuracy at the word-for-word level than at the segment or clause level, which has been found to be the unit of interpreting (Goldman-Eisler, 1972; Kirchhoff, 1976/2002). To be able to compare findings with previous studies involving the error categories at the lexical level (e.g., Gerver, 1969), the results of OW and SW were not excluded from the present study. For subsequent statistical analysis, the means of OW, OS, SW, and SS of two raters for S005 to S012 were calculated and adopted.