46
Chapter 3 Methodology
This chapter explains key research methods applied in this study, starting from an
explanation about data gathering methods for this research, followed by descriptions
about how the researcher analyzes the two main data components of this study, namely
the lecture transcription and YouTube comment data, to answer the main research
question and to discuss the issue of interpreting quality in this case study.
This naturalistic study analyzed authentic data on the web to collect users’
responses and preferences about two professional interpreters’ live interpreting session.
On December 11, 2013, Harvard Professor Michael Sandel was invited by Taiwan’s
Ministry of Culture to give a lecture about his new book, What Money Can’t Buy: The
Moral Limits of Markets. Two interpreters provided service for this SI session. The
rendition occurred in both directions, Chinese to English and English to Chinese. The
total length was 116 minutes. The lecture was held at a stadium at National Taiwan
University with an audience of 6,000 people. According to the conference organization
agency, about 5,200 people borrowed interpreting headphones, representing a large
47
body of interpreting service users.5 At the same time, the lecture was broadcasted live
on YouTube, so simultaneously there was a group of online users listening to the lecture.
As of December 18, 2013, one week after the lecture when the research data were
collected, as many as 11,648 YouTube viewers browsed the lecture web page, almost
twice the size of the live audience at the stadium. Importantly, because the YouTube
streaming did not provide dual channels, the online audience had no choice but to listen
to the simultaneous interpretation, while the original source language text was
broadcasted at the backdrop at a lower volume. In other words, online YouTube
audience became default absent users of the SI service.
Most important of all, this group of YouTube audience not only watched Sandel’s
lecture and listened to the interpreters online. Many of these listeners also left online
comments simultaneously. These comments revealed a rich set of user responses,
especially regarding their quality perceptions and judgment, their preferences or dislikes
about certain aspects of the interpretation, and also their understanding of the role and
work of interpreters. These comments thus comprised the basis of this research. In
addition to the comment data, the researcher also developed a full transcription of the
5 The author contacted the conference organization agency immediately after the event to inquire about the approximate number of audience members who borrowed headsets.
48
lecture. These are the two main data components of this research. In the following, the
researcher explains in length how these two pieces of data are analyzed, processed and
pieced together to answer the two main research questions.
First, the researcher developed a full lecture transcription data to establish the
typology and context of this unique interpreting event (Appendix A). This is one of the
unique advantages of studying a media interpreting event—because the content is
broadcasted to the public, researchers can have access to authentic performances by
interpreters and conduct corpus-based empirical research (Pöchhacker, 2011). In this
study, the lecture was broadcasted via YouTube. The entire lecture transcription lasted
116 minutes, including Sandel’s talk, the audience members’ answers and comments,
and the interpreters’ rendition. In the transcription, the source and target text were
placed alongside each other for source-target comparison. This was made possible by
the fact the online broadcast included both the source speech and the interpreters’
rendition, instead of broadcasting in voice-over mode in a single audio channel.
Yet just the transcription alone was not enough to describe clearly the interpreting
context. The researcher also developed detailed parameters to better analyze the
transcription data. The entire lecture had a very interesting structure that was unlike a
49
typical speech or conference. It was comprised of numerous back-and-forth,
multi-directional dialogues between Sandel and the different audience members. To
honestly present this very interesting structure, the researcher developed a complete log
about how the entire interpreting event took place, including the time point and time
length of every segment of the lecture, language used by the interpreter, as well as the
speaker (either Sandel or an audience member) whom the interpreter was interpreting
for in each segment.
To ensure that the analysis is valid and rigorous, the research carefully defined key
variables that were used throughout the study. First of all, “speakers” include both
Sandel and all the audience members who stood up and voiced their opinions during the
lecture. There were a total of 24 audience members, and each audience member is
numbered with an ID according to its order of speech, gender, and the language they
used (Appendix B). Secondly, each lecture “segment” is defined as one interpretation
session by one interpreter and for one source language speaker. “Segment” is a basic
unit of analysis in this study, which is used repeatedly in many of the following
quantitative analysis, such as when the researcher compares the interpreting loading of
the male and female interpreters. When there is a change of interpreter or source text
50
speaker, the researcher counts it as another new segment. For better illustration, the
following Table 1 contains three segments. From Segment #77 to Segment #78, both the
speaker and interpreter changed, and the speaker changed again in Segment #79.
Table 1: Example of transcription segments
Segment# Starting time Time length Speaker Interpreter
Source text Target text
77 0:45:32 0:00:44 Sandel F Now there seems to be a shift in opinions. Now I see more orange than I did before. Now I would like to hear from someone who said it was fair to scalp tickets for the Mayday concert but who said it was unfair to scalp train tickets on the Chinese New Year. Someone who raised white for the first question and orange for the second. Who can explain the moral difference between
I believe the key to these two questions, is that these are two things, the nature of these two things. If I don't go to see a mayday concert, I, or, maybe there's, quality of my life will change a little bit, not essential.
But the train tickets is different.
That's a traditional Chinese tradition. We have to get home on Chinese new years, so that will affect my life and I will consider it essential, and that will affect the entire country, transportation, has a broader ramification. So the country need to get involved to control that, to gauge the price of train tickets, that's unfair.
79 0:46:59 0:00:52 Sandel M Alright. Stay there. Thank you for that. So you draw the distinction between the concert…stand up, stay there…the concert is a luxury. It's not a necessity. Well, maybe for some people, Mayday concert is a necessity. But the train ticket on the Chinese New Year has a more significant, more important meaning
51
in life. And that's why it would be unfair to allow market principles ticket scalping to allocate tickets. Do I understand? Okay, stay there.
Someone disagrees. Someone thinks even scalping train tickets is fair. Tell us why.
In addition to the lecture transcription, the YouTube comment data is another
essential data component (see Appendix C for a full list of comments). The researcher
collected the YouTube comments on December 18, one week after the lecture. Although
the researcher did not collect the comments immediately after the lecture, judging from
the content of all the comments, the researcher can likely postulate that the majority of
comments occurred during or right after the lecture.6 Moreover, by December 18, an
original English version dubbed with Chinese subtitle was still not available.7 This
means that the Internet viewers had no other options but to watch this dual-channel
version and thus their comments were based on this version. Therefore the comments
analyzed were as close to the live event as possible.
After collecting all the comments, the researcher then grouped and categorized
these responses. All the comments were labeled with matching quality criteria used in
previous studies if applied, otherwise, the researcher established new categories to
6 In fact, comment #198 seemed to be a cut-off point between comments that occurred simultaneously during the lecture and those that occurred after the event. Post-event comments were by YouTube viewers who watched the rerun of the lecture.
7 The Chinese subtitle version of the lecture was broadcasted on the PBS television program on Dec. 22, 2012, and on the YouTube PBS channel from Jan. 17, 2013.
52
describe the comments. In this way, the quality attributes were developed bottom-up
from the users’ spontaneous comments in a naturalistic setting. This novel approach
allows the researcher to bring the observation and findings close to a real
communication context. In the many previous survey-based research works, quality
attributes were suggested by researchers and not by users (Moser-Mercer, 2009). But
this study takes an opposite approach--quality attributes were suggested and defined
bottom-up from users, allowing a better understanding of the complex and
multi-dimensional concept of quality. Note that some comments fell under more than
one category, thus are discussed in more than one section in this thesis.
After the transcription and YouTube comment data were ready, the researcher
combined and cross-referenced the lecture transcription and YouTube comments. This
allowed the researcher to use certain parts of the transcription as evidence or
explanation to gaze the intention behind YouTube user comments. The combination of
these two data components helped the researcher understand why certain parts of the
interpreters’ rendition elicited certain user reaction or feedbacks. The combination of the
transcription and YouTube comment is the foundation of this case study, offering a more
complete, holistic and in-depth analysis of this unique interpreting event.
53