國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Micro-Discourse: Discourse between individual users; the most likely kind.
As well as these ten types of interaction, we can measure the level of exchange at the CBC website. Tracking the degree to which comments are responses to others and are themselves responded to will help us to understand the nature of interactivity at cbc.ca. The next step is to explore more closely specific research questions and what sort of actual research methods we can use to measure the UGC at cbc.ca.
Research Questions
The above discussion raises a number of important questions. As we saw, the CBC‟s role in the current media environment is unclear. While tiptoeing into the Web 2.0 world the CBC has hesitated to move from its traditional role of content provider. At the same time, the CBC characterizes the interactivity available at its website as desirous of conversation and discourse. It is still unclear to what degree this desire has been satisfied, and more generally what function cbc.ca‟s interactivity is serving for the CBC‟s audience. Thus this study poses the following questions:
RQ1a: How much interactivity at cbc.ca is interpersonal and how much is expressive? What proportion is made up of monologue, responsive dialogue, mutual discourse, and responsive monologue?
RQ1b: How deep is the interpersonal interactivity taking place at cbc.ca? How many exchanges take place between two or more users in the context of a single conversation?
RQ2: Who are the users at cbc.ca addressing their content to? Do they tend towards macro-scopic, mezzo-macro-scopic, or micro-scopic addresses? How does address type vary by interactivity type?
Because of its importance as a national entity and its mandated mission to contribute to shared national consciousness and identity, it is fitting to examine how the user interactivity at cbc.ca may or may not be relevant to the issue of nation building. Research question three aims to explore this further:
RQ3:To what extent do the comment threads at the CBC website contribute to shared national consciousness and identity?
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Method
In order to address the research questions posed above, this study will perform both a content analysis of the UGC at cbc.ca and a discourse analysis of discussions involving national consciousness and identity. Because they are the most active element of the CBC‟s set of interactivity features, the sample will be drawn from comment threads at cbc.ca. As there are no clear theoretical reasons to believe that interaction will vary systematically for historical or topical reasons, the sampling plan is somewhat arbitrary. That said, in order to get a better understanding of what kind of commenting activity is taking place I conducted pilot research over the course of three weeks – one each from March, April and May of 2009 - that randomly sampled comments from 504 news stories. The average number of comments per story divided by national and international news topics is presented in Table 3.2.
Table 3.2: Number of Comments
Origin Mean N Std.
Deviation Domestic 146.92 248 243.382 International 89.82 256 96.558 Total 117.92 504 186.093
We can see that domestic stories tend to draw more comments and have a higher variance of total comments submitted. The range of the number of comments is large with some stories garnering no, or only a few comments, while others end up with well over 1000 comments.
The pilot study also measured comment valence on a 5-point Likert scale – ranging from 1 very negative to 5 very positive.5 The results of these findings, divided by story issue are presented in Table 3.3.
5 Very negative was defined as a comment with a negative tone that used strong language, all capital letters, or strong punctuation such as exclamation marks. Negative comments were those with a negative tone, but lacking the stronger attributes of very negative comments. Neutral comments were those which did not express positive or negative tone. Positive comments were those which used a positive tone but no strong attributes. Very positive comments were those which used a positive tone and strong language, all capital letters, or strong punctuation marks such as exclamation marks.
‧
Immigration/Ethnic Issues 2.87 24 1.035
Science/Technology 3.63 8 1.408
Other 3.38 68 1.282
Total 3.30 504 1.199
In addition to rating comment opinions, the pilot study measured the number of comments per story. These findings are presented, divided by story issue in Table 3.4:
Table 3.4: Number of Comments x Story Issue
Issue Mean N Std. Deviation
Crime 117.64 180 187.949
Health 89.33 48 76.704
Defence 66.98 60 57.600
Politics 308.55 44 396.782
Economy/Trade 115.69 52 95.294
Environment 91.00 20 94.156
Immigration/Ethnic Issues 231.67 24 140.681
Science/Technology 4.00 8 .000
Other 43.29 68 49.714
Total 117.92 504 186.093
These findings suggest that there is significant variation in the discussions taking place within the comment threads of different story types. It therefore makes sense to sample as diverse a selection of story types as possible in order to include these different types of conversation in our analysis. In addition RQ3‟s analysis of national consciousness and identity related discussions requires a selection of stories that could reasonably lead to such discussions.
Therefore, this study proposes a purposive sample of stories from different topic areas of cbc.ca.The stories posted to the CBC website will be monitored and stories will be selected until there is a comment corpus of sufficient size. Stories will be selected based on how well they fulfill the three criteria: national importance, topic variety, and number of comments.
‧
topics will be selected so as to incorporate as many different types of discussion as possible.The number of comments will only be taken into consideration as a result of practical necessity. Some stories attract thousands of comments and would overwhelm this study‟s capabilities. Therefore, stories attracting a number of comments closer to the average (118) will be focused on. All of the comments from the selected stories will be recorded and subjected to a content and discourse analysis aimed at answering the research questions.
Research Question 1a involves four elements of measurement. Each piece of UGC in the sample will be classified asmonologue, responsive dialogue, mutual discourse, or responsive monologue.
Monologue is defined as any comment not responding to another comment and not asking non-rhetorical questions. For instance, in a recent story about H1N1 vaccinations, user JeffryW wrote: “According to my calculations we will have 40, 000, 000 vaccination doses to throw in the garbage when this is all over.”6 This comment would be coded as monologic because it does not seek to engage in dialogue. It is neither asking or answering questions or responding to another user‟s contribution.
Responsive dialogue is defined as any comment either a) asking non-rhetorical questions or b) responding in some mannerto another piece of UGC. For example, in the same vaccination story, 2Little2L8 wrote: “staceylnewman Stacey I think maybe you missed the separation in my last post, the first half is from girlinalberta, I am the conspiracy theorist.”
Because this post is clearly a response to another user it would be coded as responsive dialogue.
Mutual discourseis dialogic in nature and thus similar to responsive dialogue.
However, it is distinguished from responsive dialogue by its higher exchange level, degree of respect and understanding between the participants and number of conversants. To be distinguished from responsive dialogue mutual discourse must involve more than two participants – and thus more than two exchanges - and must involve more than simple disagreements amongst users. Any comment that fits within this definition will be considered part of a mutual discourse and coded as such.
6 These example comments are taken from the story about H1N1 vaccinations available at:
http://www.cbc.ca/canada/story/2009/11/02/swine-flu-debate002.html
‧
Responsive Monologue is defined as UGC that refers to preceding discussion without referring to any particular comments or answering or asking any questions. These messages tend to be commentary on the overall tone of earlier discussion. For example, user Fewmets wrote in regards to the comments made on the story of an accused Canadian terrorism suspect: “I love stories like this; whether the accusation is true or not, the comments remind me that we‟ve still got a lot of racists among us.”7
Feedbackthat users leave by using the „agree‟ and „disagree‟ functions will be measured by tracking the totals of each for each comment. Because earlier comments have more opportunity to attract feedback, the raw numbers of agreement and disagreement will not be comparable across comments. Thus, feedback totals will be used to create a feedback level allowing us to compare levels of agreement/disagreement.8
Research Question 1b will require following the thread of conversations that take place within the UGC at cbc.ca. By keeping track of the exchanges that take place in each comment thread, we can code comments by their exchange-level. Thus anything that is a response to one other comment is a level 1. So, 2Little2L8‟s comment referred to above would be coded as a number 1 because its response to Stacy Newman is the second comment in a conversation thread. If Stacy Newman were to subsequently respond to 2Little2L8‟s comment her reply would be a level 2 comment. Subsequently, anything responding to a level 2 comment is a level 3 comment, and so on. In the case that a single comment responds to two or more previous comments the average exchange-level value will be used.
Research Question 2 requires that the address mode of each item of UGC within the sample be coded.
Macro-scopic addresses are those messages addressed to the public at large. The above comment by JeffryW would be coded as macro-scopic because it is addressed as a general declarative statement.
7 This story can be found at: http://www.cbc.ca/world/story/2010/01/25/chicago-mumbai-terror-100125.html
8 These will be calculated by dividing the number of „agrees‟ by the number of „disagrees.‟ Thus a comment with 500 agrees and 100 disagrees would have an agreement level of 5, while a comment with 20 agrees and 4 disagrees would also have an agreement level of 5. Meanwhile a comment with 4 agrees and 20 disagrees would have a feedback level of 0.2.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Mezzo-scopic addresses are those directed towards the CBC as an institution. These comments are often critique the CBC‟s journalistic practices or the writing of the particular article commented on.
Micro-user-scopic addresses are those messages aimed at particular users. The above comment left by 2Little2L8 would be coded as micro-scopic because it is clearly addressed to another user: stacylnewman.
Micro-story-component-scopic addresses are those aimed at elements of the story. For instance, a recent story about Don Cherry‟s influence on head injuries in hockey attracted many comments addressed towards both Don Cherry and the medical doctor mentioned in the story. For instance, Iron Man 01 wrote: “Cherry… I for one support you.”9
Research Question 3 requires more detailed exploration than a quantitative content analysis can provide. In order to understand how interactive functionality may or may not contribute to a shared sense of national consciousness and identity we need to look to the content of comments left at the CBC website. This will involve a close reading of the comments to identify those that deal with issues pertaining to national consciousness or identity. Once the pertinent comments are identified they will be examined to determine how they represent Canada and Canadianess. The examination will focus on important issues such as how comments act to define Canadian, the language they use, whether or not they discuss issues of Canadian identity, responsibilities, ideals and subsequently the degree of agreement or disagreement amongst user conceptions of “Canadian.” The extent to which these comments reflect the maintenance and/or generation of a shared national consciousness and identity will determine the answer to RQ3.
With these research questions and methods in hand, this study will now turn to the final two chapters, where the research findings will be presented and discussed.
9 This story can be found at: http://www.cbc.ca/sports/hockey/story/2009/12/19/sp-doncherry.html#socialcomments.
‧
Tracking the stories led to the selection of 11 suitable stories with 1491 total comments. Each story was selected based on its contribution to both topic diversity and the potential for nationally oriented discussion. The stories selected and the number of comments on each story are presented in Table 4.1.
Table 4.1: Stories Comments Percent
Haiti Relief(CBC, 2010c) 290 19.5
Tamils(CBC, 2010a) 122 8.2
UFC Health(CBC, 2010k) 218 14.6
Target(CBC, 2010i) 114 7.6
Polar Bears(CBC, 2010b) 95 6.4
Haiti Adoptees(CBC, 2010g) 244 16.4
Mumbai(CBC, 2010d) 61 4.1
Quebecor(CBC, 2010j) 38 2.5
Asbestos(CBC, 2010e) 60 4.0
Repatriation(CBC, 2010f) 60 4.0
Olympic Flag(CBC, 2010h) 189 12.7
Total 1491 100.0
Following its selection each story‟s comment thread was allowed to run its course and be closed for commenting – usually taking about one week from the posted on date. The comments were then collected and coded according to the scheme attached in Appendix A.
Due to resource restrictions, this study was unable to cross code the entire comment corpus. However, in order to determine the reliability of the coding, approximately 10% of the corpus was coded by a secondary coder and the results were compared. The secondary coder was an American female with an undergraduate degree in communication studies. Her training – by the author and original coder – involved full explanations of all the concepts she was to code for and a brief demonstration of how the original coder would have coded a selection of comments.
Because a number of the measurement levels examined objective traits of the comments – i.e. number of words or number of thumbs up and thumbs down – not all elements were cross measured. The cross coding focused on the three traits most vulnerable to subjective interpretation or coding errors. These were:
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Type of Comment
Exchange Level
Address Level
Determining exchange level and sometimes address level required following entire comment threads. As a result, instead of randomly sampling a selection of comments to cross-code, the entire comment sections from three stories were coded. The Mumbai, Quebecor, and Asbestos stories were selected because their comments sections included 159 comments, totalling 10.7% of the total number of comments coded.
Because the variables being coded were measured at a nominal level and averaged about four nominal options per variable, the possibility of random agreement was somewhat high. To address this concern, in order to compare coding results Krippendorf‟s alpha was used instead of a more basic correlation. The level of agreement between the two coders was quite high, with a Krippendorf‟s alpha level of 0.943. These results are within reasonable thresholds established by the literature. Lombard et al(Lombard, Snyder-Duch, &
Campanella Bracken, 2002) indicate that correlations of 0.90 or above are generally considered to be sufficient for all measures of cross-coder reliability, especially for more conservative measures such as Krippendorf‟s alpha. Before delving into the results of this content analysis, let us look briefly to the content of the stories in question.