行政院國家科學委員會補助專題研究計畫
□
期中進度報告
釐清實驗法研究中媒體刺激物的定義
計畫類別：■個別型計畫
□整合型計畫
計畫編號：NSC 96-2412-H-009-001
執行期間：96 年 1 月 1 日至 96 年 8 月 31 日
計畫主持人：陶振超
共同主持人：
計畫參與人員：李孟潔、顏聖致
成果報告類型(依經費核定清單規定繳交)：■精簡報告 □完整報告
本成果報告包括以下應繳交之附件：
□赴國外出差或研習心得報告一份
□赴大陸地區出差或研習心得報告一份
□出席國際學術會議心得報告及發表之論文各一份
□國際合作研究計畫國外研究報告書一份
處理方式：除產學合作研究計畫、提升產業技術及人才培育研究計畫、
列管計畫及下列情形者外，得立即公開查詢
□涉及專利或其他智慧財產權，□一年□二年後可公開查詢
執行單位：國立交通大學傳播與科技學系
中
華
民
國
9
6
年
1
0
月
3
0
日
Conceptualizing Media Stimuli in Experimental Research: Psychological versus Attribute-Based Definitions
Chen-Chao Tao
National Chiao Tung University, Taiwan &
Erik P. Bucy
Indiana University, Bloomington
Correspondence: Chen-Chao Tao
Department of Communication and Technology 1001 Ta Hsueh Road
National Chiao Tung University Hsin-Chu 30010, Taiwan
Tel: 886-3-513-1540 Fax: 886-3-572-1486 E-mail: [email protected]
Chen-Chao Tao (Ph.D., Indiana University) is an assistant professor in the Department of
Communication and Technology at National Chiao Tung University in Taiwan. His research focuses on cognitive processing of interactivity in networked media, the impact of individual differences on
information processing, and social network models of computer-mediated communication.
Erik P. Bucy (Ph.D., University of Maryland) is an associate professor in the Department of
Telecommunications and adjunct associate professor in the School of Informatics and Department of Political Science at Indiana University, Bloomington. His research focuses on the role of interactivity in communication environments, visual analysis of television news, and nonverbal aspects of political behavior. Bucy is the co-editor of Media Access: Social and Psychology Dimensions of New Technology Use (Erlbaum, 2004).
Authors’Note: An earlier version of this paper was presented at the annual meeting of the International Communication Association, Information Systems Division, June 2006, Dresden, Germany. This research was funded in part by a grant from the National Science Council of Taiwan (96-2412-H-009-001).
Tao, C.-C., & Bucy, E. P. (2007). Conceptualizing media stimuli in experimental research:
Abstract
This paper argues for a clearer conceptualization of media stimuli in experimental research and identifies three issues impeding our understanding of message processing: (1) assumptions bolstered by manipulation checks about homogeneity of response to media stimuli; (2) conflation of two different classes of variables –media attributes and psychological states; and, (3) discrepancies between the conceptual model and operational-level hypotheses used to test research questions. To provide a more comprehensive framework for investigating media effects in experimental research, we argue for a clearer conceptual separation between message attributes and user perceptions and apply a mediation model of information processing to
overcome the limitations of conventional approaches. Subjected to two empirical tests involving the assessment of Web-based media, the model finds an increase in explained variance in each instance.
Key words: media stimuli, mediation models, experimental research, bootstrap distribution test, Google News, Web search, interactivity, credibility
Conceptualizing Media Stimuli in Experimental Research: Psychological versus Attribute-Based Definitions
Since the rise of the information processing approach in communication research (see Geiger & Newhagen, 1993; A. Lang, 2000; Reeves & Nass, 1996), scholars have argued that media stimuli, employed as independent variables in experimental studies, should be defined in terms of psychological states. In a telling colloquy with news researchers, Reeves (1989) suggested that the unit of measurement in television research should be based on cognitive
processes,orpsychological“theory units,”ratherthan industry defined categories,such asnews story, bulletin, or program. Over the past two decades, effect-based (i.e., psychological state) definitions of media stimuli have arguably dominated the experimental investigation of how individuals process information. Along these lines, Geiger and Newhagen (1993) suggested that the information processing approach provides a unique characterization by defining media stimuliin termsof“psychologicaldimensionsand attributes”(p.42).Lang (2000) similarly
stressed thattheinformation processing modelviews“television asapsychologicalstimulus”(p. 51). While advocating for receiver-based investigations of message processing, these authors downplay attribute-based (message oriented) definitions of stimuli, assuming that internal states are synonymous with –or should take precedence over –content characteristics.
To provide a more complete framework for investigating media effects in experimental research, in this paper we argue for a clearer conceptual separation between message attributes and user perceptions and apply a mediation model of information processing to overcome the limitations of conventional approaches.
Competing Conceptualizations Effect-labeled media attribute definitions
In the communication and technology effects literature, two effect-oriented definitions of experimental stimuli have developed: the effect-labeled media attribute and the effect-based psychological state (see O'Keefe, 2003). Effect-labeled media attribute definitions assume that a set of intrinsic message or medium properties, including both message content and structural features, reliably vary along specific psychological dimensions. This approach classifies media stimuli into different groups according to their media attributes but identifies these groups in terms conceptual categories, such as user perceptions or viewer emotions, which are evoked by attributes. In other words, media attributes, like ingredients for wine in bottles, are used to distinguish one group of media stimuli from another, while the effects of media attributes, like labels on bottles of wine, are used to name different groups of media stimuli.
To evaluate the appropriateness of testing particular media attributes, scholars routinely employ manipulation checks as a demonstration of the psychological differences that distinct groups of media attributes elicit. In a study that examined whether the effects of television violenceand hypermasculinity on young males’aggression was contingent on personality characteristics, for example, Scharrer (2005) manipulated violence with three television dramas that differed in message content. Conceptual categories were assigned to each drama with The Sopranos representing violence and hypermasculinity, Buffy the Vampire Slayer representing violence without hypermasculinity, and 7th Heaven serving as the control condition, featuring no violenceorhypermasculinity.To provideevidenceofthemanipulation’sefficacy,scalesfor violence and hypermasculinity were administered and, in accordance with the design, a
manipulation check showed that participants rated The Sopranos as having the highest level of violence and hypermasculinity, followed by Buffy the Vampire Slayer and 7th Heaven.
Although perceptions of systematic message differences are confirmed by such manipulation checks, studies adopting effect-labeled media attribute definitions of stimuli
generally omit psychological states from explicit consideration when it comes to hypothesis testing. At the conceptual level, attribute-based approaches focus on the relationship between the cognitive label assigned to a message or medium and the resulting media effects, whereas at the operational level they consider the relationship between distinct groups of media attributes and media effects (see Figure 1a).1Two assumptions underlie effect-labeled approaches: (a)
cognitions and emotions of interest are interchangeable with media content or attributes (the conflation assumption) and, (b) subjects exposed to the same content or media attributes will likely experience identical or highly similar psychological states (the homogeneity of response assumption). Both of these assumptions occlude thorough examination of important statistical relationships between variables and, as we discuss below, are easily violated.
Despite these limitations, media stimuli defined in terms of their effects prevail in communication research. In studies of television, film, or other audiovisual media, researchers commonly assign different levels of emotional arousal and valence to selected clips to examine whether varying levels of affect will significantly influence memory, attention, subjective evaluations, or other outcomes of interest (see Bucy & Newhagen, 1999; Reeves & Nass, 1996). In a study of news images and basic emotions, for example, Newhagen (1998) associated anger, fear, and disgust with different television news stories and examined the effects of negative compelling footageon viewers’approach-avoidance responses. Bolls and Lang (2003) assigned high or low imagery to a set of radio advertisements and argued that high-imagery ads would mobilize more cognitive resources. In Internet research, Sundar, Kalyanaraman, and Brown (2003) operationalized low, medium, and high interactivity as different hierarchical hyperlink structures to investigate whether the relationship between interactivity and impression formation was linear. Although often justified with a pre-test of mean perceived differences between levels
of the independent variable, such attributions actually conflate intrinsic media properties with conceptual categories assigned by the researcher.
Effect-based psychological state definitions
By contrast, definitions reflecting effect-based psychological states hold that the impact of media attributes is indexed by the variation in viewer or user perceptions. Accordingly, this approach rarely generates concern about what precise content characteristic or structural feature is actually being manipulated, favoring instead the effect on the user or receiver. In terms of hypothesis testing, studies employing an effect-based psychological state definition focus at the conceptual level on the relationship between media attributes and media effects, whereas at the operational level they examine the relationship between psychological states and media effects (see Figure 1b). Contrary to attribute-based definitions, the effect-based psychological state approach accommodates the likelihood that even the same media message or attribute will generate different perceptions for different people. A similar situation is observed in the uses and gratifications literature, which has found that media audiences derive different gratifications from the same message (Rubin, 1994).
A study conducted by Stephenson and Palmgreen (2001) in the effect-based tradition explored the influence of perceived message sensation value on viewer processing of anti-marijuana public service announcements (PSAs). Without specifying any systematic differences in message content or features, the authors selected a series of 30-second PSAs to represent a range of creative approaches. Stephenson and Palmgreen (2001, p. 51) argued that message sensation value is best understood as viewer responses to message attributes rather than more objective operationalizations of the messages themselvessincetheformeris“probably more strongly and directly related to persuasiveoutcomes”than thelatter.Accordingly,ascaleof perceived message sensation value was employed to measure viewer evaluations and in
hypothesis testing was found to facilitate pro-message cognitive processing. Although the design yielded positive results, not all hypotheses were fully supported. As with other studies conducted in this tradition, distinct message attributes were not included as a factor in the data analysis.
Media stimuli defined in terms of their effect-based psychological states, such as vividness, fear appeals, or argument quality, ignore effect-independent message features that exist in media content regardless of how they are interpreted (O'Keefe, 2003). Claims about strong or weak argument quality, for instance, are often based on how audiences perceive message content rather than actual logical consistency or the amount of factual evidence
included in a message (Mongeau & Stiff, 1993). Although perceptions in experimental research arefrequently “farmoreinfluentialthan reality”(Reeves & Nass, 1996, p. 253), studies
conducted in the effect-based psychological states tradition typically omit the actual message manipulation from statistical consideration. In a study of message features and subjective evaluations of anti-drug PSAs, Morgan, Palmgreen, and colleagues (2003) observed that message sensation value should be viewed as an integral component or intrinsic property of mediacontentbutinstead “usually hasbeen operationalized asperceived message sensation value”(p.515;italicsadded).They suggest,aswearehere,thatmessagefeatures–and user responses to those features –should be conceptually and operationally separated and the
relationship between them carefully theorized. Perceptual approaches are also gaining ground in Internet research. Indeed, several authors have examined interactivity in terms of user
perceptions, which have been found to impact attitudes and other evaluations (see Bucy & Tao, 2007; McMillan & Hwang, 2002; Wu, 2005).
Challenges to Effect-Oriented Definitions
Recently, effect-oriented definitions have encountered serious challenges, even among researchers who have worked within this tradition. In separate analyses of persuasive
communication, Newhagen (2002) and O’Keefe(2003) argue that conflating emotional
responses and cognitive categories with intrinsic message features, and failing to understand the role of mediating states has thwarted progress in understanding message effects. On the first point,O’Keefe(2003) notesthat“when messagevariablesaredefined in termsofeffectsrather than intrinsic properties, researchers forfeit the ability to speak to questions of the relationship between message properties and persuasive outcomes”(p.268).On thesecond point,Holbert and Stephenson (2003) observe that effect-oriented definitions lead to theoretical incompleteness because there are implicit assumptions about mediation effects that are excluded from hypothesis testing. Potter and Tomasello (2003) demonstrate that separating media attributes and
psychological states and including both classes of variables in hypothesis testing can
substantially increase the amount of explained variance in statistical tests. Such methodological refinement is important, given that the majority of media effects research explains less than 10% of the variance in message influence (Sherry, 2004).
The specification of independent variables in experimental research largely determines the degree to which scholars can ascertain the source of media effects. In a classic explication of the research process, McLeod and Reeves (1980) emphasized that an effective media stimulus should use natural stimulus units, specify the manipulation of the stimulus strength, and maintain the independence of media stimuli (pp. 258-261). Effect-oriented stimulus definitions fail to meet these criteria.
Although Reeves (1989) urged a“radicalseparation”between newsand cognition,his suggestion that media stimuli should be defined in terms of viewer cognitions rather than by industry-imposed message units actually blurs the line between media attributes and
psychological states. Operationally, the stimulus message is an antecedent condition necessary for the psychological state to be induced (see Figure 1c). In mediation models, cognition
represents the processing of a media stimulus but should not stand in for the stimulus itself (O'Keefe, 2003). Reeves (1989) argued that industry-imposed message categories are unlikely to serve as fundamental mental units, similar to how cognitive processes such as attention, memory, and emotionalresponsesareunlikely to “exactly overlap with theboundariesofnewsmessages” (p. 193); indeed, the term news “mightnotexistifcognition weretheonly consideration”(p. 193). However, media effects research concerns not only how people process messages but also what media attributes or intrinsic message features produce effects. The important
methodological consideration is how to accommodate the simultaneous inclusion of media stimuli and cognitive processing into a single statistical model.
Complications arising from the relationship between media attributes and psychological states raise some interesting issues pertaining to internal and external validity. With attribute-based definitions, the assumption that a given stimulus or media attribute produces the same response across individuals is seldom examined. In an analysis of receiver interpretations in media violence research, Potter and Tomasello (2003) edited three versions of a primetime drama according to the number of violent acts. The researchers specified a low, medium, and high violence condition. A manipulation check was conducted and showed that participants perceived the televised violence as intended, but the check itself accounted for just 7% of the variancein participants’perceptionsofviolence.Moreover,the distribution ofresponseswithin each treatment group exhibited a large range (the difference between the maximum and
minimum)and variance,which implied thatparticipants’opinionswereactually widely spread. Perceptions of violence within treatment groups were internally inconsistent yet, at the same time, differences across groups were statistically significant. Such findings illustrate how the
homogeneity of response assumption can be easily violated. Inconsistency within each treatment group also results in low explanatory power. Consequently, Potter and Tomasello rightfully
wonder whether the statistically significant differences between their three treatment groups may be spurious.
The varying relationship between media attributes and psychological states also erodes external validity. O’Keefe(2003) asserts that replicating the results of studies that define persuasive stimuli in terms of psychological states is difficult because the capacity of message properties to reliably elicit specific responses across different users is unclear. Relying on manipulation checks that report overall differences in response to varying treatment conditions doesnotobviatetheneed to accountformessagecharacteristics.“Assessmentsofpsychological states, reported as message manipulation checks, are no substitute for a careful description of message properties, and effect-defined message variations obviously evade the task of describing messageproperties”(O'Keefe, 2003, p. 269). Indeed, an over-reliance on message manipulation checksto justify thatexperimentaltreatmentsareexperienced asintended “disguisesthe undertheorized characterofmessages”themselves (O'Keefe, 2003, p. 272; see also Sigall & Mills, 1998).
In summary, there are three issues associated with effect-oriented definitions of media stimuli impeding our understanding of message processing: (a) assumptions bolstered by manipulation checks about homogeneity of response to media stimuli; (b) conflation of two different classes of variables –media attributes and psychological states; and, (c) discrepancies between the conceptual model and operational-level hypotheses used to test research questions. To address this situation, we suggest a clearer conceptual separation between message attributes (or properties) and psychological responses in experimental research and apply a mediation model to provide a more complete framework for investigating media effects. Subjected to two empirical tests involving the assessment of Web-based media, the model finds an increase in explained variance in each instance.
Although mediation models have been successfully demonstrated in survey based effects studies (see Eveland, 2002; Eveland, Shah, & Kwak, 2003; Slater & Rasinski, 2005), which have argued that media use is inherently a mediating or endogenous process, their application is much less common in experimental research. Thus, pertinent methodological issues for using
mediation in experimental designs are addressed below.
A Mediation Model of Experimental Research
To overcome some limitations of the often assumed but untested relationships outlined above, we propose a mediation model of experimental research (see Figure 1c).2The model has three basic tenets: (a) media stimuli, serving as a independent variable, should be defined in terms of media attributes or intrinsic message properties rather than psychological states; (b) psychological states, taking the form of emotions, perceptions, evaluations, or other cognitive responses elicited by media stimuli, should serve as a mediator variable; and, (c) hypothesis testing should include both media attributes and psychological states in statistical analysis to capture a more complete picture of media influence and increase explanatory power. Examining the relationships between message properties and psychological outcomes avoids conflating two different classes of variables and allows the analysis to proceed without erroneously assuming uniform responses to messages or other media stimuli.
The mediation process originally specified by Baron and Kenny (1986) represents the coreofthemodeland explains“how external physical events take on internal psychological significance”(p.1176).A mediator variable, such as perceived interactivity, aroused emotion, or another psychological state, serves as a pathway through which the independent variable
influences the dependent variable. Studies employing attribute-based definitions provide
evidence for the independent effect of message properties, while those that conflate stimuli with psychological states provide partial support for the indirect effect of media attributes on
measures of interest. Both approaches may be valid under certain circumstances. Consequently, departing from Reeves’(1989) argument that media stimuli should be defined in terms of psychological states, we suggest that media attributes and psychological states should be treated as separate concepts but included in the same general model (see Bucy & Tao, 2007).
The widespread use of analysis of variance (ANOVA) for statistical tests in experimental research turns out to be a major obstacle hindering the investigation of mediating variables. ANOVA is simply not applicable to testing indirect effects (Baron & Kenny, 1986; Mongeau & Stiff, 1993; O'Keefe, 2003). Applied to designs involving one or more experimental factors or independent variables, ANOVA has no room for mediators because they must serve as
independent and dependent variables simultaneously (Baron & Kenny, 1986); Hays, 1994). However, recently developed data analysis strategies using the bootstrap distribution of product test may facilitate the direct assessment of mediation in experimental designs.
Data Analysis Strategies
Despite the above-mentioned limitations, data analysis for studies employing attribute-based definitions of media stimuli mainly involves testing a series of ANOVA models. First, ANOVA is used to compare the effects of different media stimulus conditions on participant perceptions to check the adequacy of the manipulation. The relationship of interest is that between the media stimulus condition and the evoked psychological state. Second, once the manipulation check is confirmed, ANOVA is again employed to examine the relationship between message properties and ensuing media effects (the hypothesis at the operational level). However, the relationship between the relevant psychological state and resulting media effects (the hypothesis at the conceptual level) is seldom tested because mediator variables are rarely specified in experimental designs (see Figures 1a).
Data analysis for studies employing psychological state-based definitions of media stimuli mainly involves testing a simple regression equation. Perceptions elicited by different stimulus conditions are positioned as independent variables predicting media effects (e.g., McMillan & Hwang, 2002). Some studies test the regression equation for each media stimulus condition to show that the pattern is replicated in each instance (e.g., Wu, 2005). However, these analyses neither examine the relationship between media attributes and media effects nor the relationship between media attributes and the mediating psychological state; instead, emotions and perceptions evoked by stimuli are employed as direct predictors of the dependent variable without regard to the message or medium itself (see Figures 1b).
Although mediation is an important concept long recognized by communication scholars (McLeod & Reeves, 1980), the tools and techniques of mediation analysis are not widely
understood or readily available (Holbert & Stephenson, 2003). The absence of features to test mediation effects in the most popular statistical packages, including SPSS and SAS, compounds the problem. Moreover, the classic procedure for testing mediation effects in between-subjects experimental designs,3described by Baron and Kenny (1986), suffers from two critical
shortcomings (see Figure 1c). First, the indirect (evaluating c, c', a, and b respectively) rather than direct (evaluating c - c' or equivalently ab) assessment of mediated effects leads the Baron and Kenny procedure to produce low statistical power; hence, the Baron and Kenny procedure is “likely to miss real effects [but] very unlikely to commitaTypeIerror”(MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002, p. 96). Second, the insistence of a statistically significant total effect (c) as a requisite criterion restricts the Baron and Kenny procedure to experimental
conditions involving just one mediator variable (Collins, Graham, & Flaherty, 1998; MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002; Shrout & Bolger, 2002).
To overcome these constraints, the nonsymmetric bootstrap distribution of product test proposed by Efron and Tibshirani (1993) may be used to directly assess mediation in
experimental research. There are two advantages associated with this technique. First, the mediated effect is the product of two normal random variables (a and b), which is not normally distributed but skewed (Bollen & Stine, 1990; Shrout & Bolger, 2002). Second, a direct test of the null hypothesis –that the mediated effect is zero –improves statistical power and avoids the need to conduct a series of regression equations when testing for mediation (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002; Preacher & Hayes, 2004). Therefore, the
nonsymmetric bootstrap distribution test seems suitable as a statistical technique for examining mediation effects in experimental research.4
To test the applicability of the mediation model in an experimental setting, data from two studies that assessed the effects of interactive media interfaces on user responses were analyzed. The purpose of the first experiment was to examine the effects of search term specificity and search result density on affective experience, namely, feelings of dominance or user control. The second experiment was an investigation into the effects of interactivity on the credibility
assessments of online news. For both studies, conventional statistical tests employing ANOVA and regression were initially used for effect-oriented hypothesis testing; then, to capture
unexplained variance, we employed a mediation model including both media stimuli and evoked psychological states as independent and mediating variables in the same test.
Experiment 1: Searching Google News
Despite the inability of any search engine to index all Web pages, the use of search engines such as Google has emerged as the primary information seeking activity on the Internet (Rainie & Shermak, 2005). Moreover, Web searchers trust search engines. More than two thirds (68%) of Web searchers view search engines as a reliable source of information (Fallows, 2005).
Accordingly, Web search has begun to attract research attention, with special issues of communication and technology journals devoted to the topic (see Hargittai, 2007).
However, current Web search studies, which primarily focus on the usability dimension of search, are marked by two recurring tendencies. First, empirical studies are for the most part based on search engine transaction log analysis (e.g., Catledge & Pitkow, 1995; Chau, Fang, & Sheng, 2005; Jansen & Spink, 2006; Spink, Park, Jansen, & Pedersen, 2006). Other methods of data collection, including survey (e.g., Spink, Bateman, & Jansen, 1999), experimental (e.g., Ford, Miller, & Moss, 2005), and qualitative (e.g., Romand, Donovan, Chen, & Nunamaker, 2003) approaches are seldom employed. Second, search accuracy and efficiency typically serve as the main dependent variables studied in the Web search literature. Emotional or psychological responses experienced during search are rarely measured, although they are considered possible mediating states affecting search accuracy and efficiency. Recently, communication and
technology scholarshave urged thefield to expand into “thesocial,political,economic,and culturaldimensions”ofsearch behavior(Hargittai, 2007). Therefore, adopting an experimental approach to test hypothesized relationships between Web search tasks and psychological responses to search engine use should provide insights into aspects of information seeking that have not been widely explored.
The purpose of the first experiment was to examine whether the relevance of search results to structured search queries affects feelings of user control, or dominance. Emotional responses to media stimuli are an important consideration in communication and technology research, particularly since they may influence other variables of interest, including memory, attention, and user evaluations (Newhagen & Reeves, 1992). Emotional responses also play an important role in Web search behavior (Mastro, Eastin, & Tamborini, 2002), especially feelings of control (Kayany & Wotring, 1996; McMillan, 2002). In this study, we hypothesized that
search term specificity (the number of relevant terms used in a Google search) would be
positively correlated with feelings of dominance, an emotional state characterized by feelings of self-assurance and activation. In particular, high-quality search results generated by a high level of specificity (multiple search terms) should elicit a high degree of perceived relevance,
enhancing feelings of control over the search task. Thisexpectation isconsistentwith Rafaeli’s (1988) suggestion that increased interactivity should impart a sense of mastery, which in turn is expected to encourage cognitive processing. For the study, a high (3 search terms) or low (1 search term) specificity condition was presented to participants and the relationship between search-term specificity and self-reported dominance assessed.5
From an attributes-based perspective, participants exposed to the same media stimulus conditions are assumed to have similar cognitive and emotional responses. A manipulation check is used to confirm the difference in responses between conditions and conceptual labels are assigned to represent the experimental factors and treatment levels. In this experiment, two stimulus conditions were created: (a) search result pages based on one key term or phrase, intended to elicitalow amountofperceived relevance(e.g.,“obesity”), and, (b) search result pages based on three or four key phrases, intended to elicit a much more focused set of results – and ahigherdegreeofperceived relevance(e.g.,“obesity,fastfood,and television”;see
Appendix 1). Although conceptual labels are used to represent different conditions (i.e., “specificity”and “relevance”),itisimportantto notethatthe study actually examined the relationship between two message properties –the number of relevant search terms and search results –and measures of subsequent media effects. It can thus be predicted that:
H1: Self-reported dominance will be greater for search result pages generated by 3 search
terms (high specificity) than search result pages generated by 1 search term (low specificity).
Functionally, Roseman and Smith (2001) propose that emotions serve as appropriate response guides for coping. During computer use, emotions may be evoked by content that is novel, compelling, or surprising, navigation that is difficult or unsuccessful, or other instances of human-computer interaction that are either rewarding or particularly problematic, including Web search (see Bucy, 2004b). Multiple search terms, representing a high level of specificity, should produce more pronounced feelings of control than a single search term, representing a low level of specificity. The logic here is that search results generated by specific queries are more relevant to the information users are looking for and expect to find than general queries; this precision should, in turn, promote feelings of mastery over an assigned search task. Accordingly, the following hypothesis is proposed:
H2: Perceived relevance will be positively related to perceptions of dominance.
From the perspective of mediation, feelings of dominance may be influenced not just directly by the number of search terms but also indirectly through the perception of relevance elicited by varying degrees of search term specificity. Therefore, the media stimulus condition serving as the independent variable and the relevant psychological state acting as the mediator should both be included in statistical testing. The influence of the mediator, representing a pathway through which the independent variable influences the dependent variable, warrants explicit consideration. Thus, it is hypothesized that:
H3: Perceived relevance will mediate the relationship between search-term specificity and
self-reported dominance. Method
Design. The experiment took the form of a between-subjects 2 (search-term specificity) 3 (search-result volume) factorial design. The first factor, search-term specificity, had two levels: 1 versus 3 search terms (representing low and high relevance in the effect-labeled attributes test).
The second factor, search-result volume, had three levels: 10, 20, or 30 results per search. Only the results for search-term specificity are reported here because the second factor involves a different mediator.
Stimulus materials. Stimuli for this study took the form of 24 search result pages edited from actual Google News searches, assigned to six conditions: 10, 20, or 30 search results based on one search term, and 10, 20, or 30 results based on three search terms.6Each condition engaged users in four search queries on political and health-related topics, including obesity, a politically “divided America,”faith and thepresidency,and MichaelMoore’sFahrenheit 9/11 documentary. Original search result pages were obtained by selecting the desired number of search resultsthrough Google’s“Advanced NewsSearch”option and typing search termsinto the search box.7Search terms for each query were specified in advance.
A complete listing of the search terms and questions is presented in Appendix 1. For the low specificity condition, search terms with a single word, name, or phrase mentioned the topic only; for the high specificity condition, search terms with multiple words or phrases mentioned the topic (e.g., 2004 presidential election), contextual focus (e.g., abortion), and presumed causal basis (e.g., faith).
Independent and mediator variables. For the effect-labeled media attributes test, the number of search terms used to produce search result pages served as the independent variable and was labeled high or low relevance. The applicability of these labels was confirmed with a manipulation check.
For the effect-based psychological states test, user perceptions of search relevance, measured by a 5-point Likert-type scale (1 = not at all and 5 = very), served as the independent variable. Different media stimulus conditions (1 versus 3 search terms) were employed to generate variance in user perceptions. No manipulation check was required because participant
perceptions served as the independent variable of interest. Moreover, variation in the media attribute (the number of search terms) was objectively verifiable independent of participant perceptions (O'Keefe, 2003, p. 254).
For the mediation test, the number of search terms served as the independent variable and the 5-point perceived relevance measure was included in the model as a mediator. Again, no manipulation check was required for the mediation test because variation of the media attribute was independent of participant perceptions.
Dependent variables. The Self-Assessment Manikin (P. J. Lang, Greenwald, Bradley, & Hamm, 1993) was used to measure emotional responses. The SAM instrument includes three 9-point pictorial scales that index emotional arousal, valence, and dominance. For this study, only dominance responses (ranging from 1 = in control to 9 = not in control) were examined. Most research involving dominance has not been conducted in interactive environments, where user control is a central aspect of the media experience. In addition to control, the semantic
differentialscalesunderlying dominanceincludesuch termsas“influential,”“important,”and “autonomous”(P. J. Lang, 1998; Mehrabian, 1972).
Participants. Participants consisted of 133 undergraduate students enrolled in
communication courses at a large Midwestern university, including 77 females and 49 males. Participants ranged in age from 18 to 27 (M = 21) and received extra credit for their participation. Overall Web use was high; for a typical weekday, the median amount of time spent online was 3-4 hours and for a typical weekend day 2-3 hours.
Procedure. Participants were first asked to answer a pre-stimulus questionnaire about their Internet usage. Next, they were randomly assigned to a search-term condition and engaged in four search tasks guided by questions concerning the above-mentioned political and health-related topics. Each search task began with a webpage showing a search query and the term(s)
used to generate an associated search result page. To maintain experimental control over the search queries and results, participants were not allowed to type in their own terms. After pressing a“Continue”button,theGooglesearch resultpagewas displayed. Participants were instructed to freely browse the list of results and complete a designated activity of selecting the best news story to match the search question. They were then asked to perform a distraction task, which consisted of answering ten simple math questions. Lastly, they completed a post-test questionnaire that included the emotional self-report measures.
Manipulation check. To verify that participants perceived the manipulation as intended, a 5-point relevance scale (1 = not at all and 5 = very) was included as the manipulation check and one-way analysis of variance performed to examine the main effect. The check was performed to ensure that the two stimulus conditions, representing effect-labeled media attributes, reliably elicited varying levels of perceived relevance. On average, participants rated search result pages generated by multiple search terms higher in perceived relevance than search result pages generated by a single search term or phrase, F(1, 125) = 5.24, p < .05,η2= .04.
However, the distribution of responses within each condition revealed a much more dispersed pattern of individual evaluations than implied by an analysis of mean differences (see Figure 2). Notably, there was a range of responses even among participants exposed to the same search result pages: in the high-relevance condition, evaluative ratings ranged from 2 to 5; in the low relevance condition, ratings ranged from 1 to 5. Moreover, comparing the distributions of the two conditions revealed that 51 participants in the high- and low-relevance conditions (38.3% total) reported the same responses, even though they were exposed to a varying number of search terms and browsed different search result pages. Such tendencies highlight the importance of subjective responses to experimental stimuli and the role that individual differences might play
in combination with stimulus conditions in influencing outcomes of interest (see Bucy & Tao, 2007).
Results
Hypothesis 1 predicted that search result pages with high relevance (generated by 3 search terms or phrases) would produce greater levels of self-reported dominance than those with low relevance (generated by 1 search term or phrase). Hypothesis 1 was tested with a regression equation in which self-reported dominance served as the dependent variable and search-term specificity the independent variable. Testing the model produced a nonsignificant result. The degree of specificity, and therefore relevance, did not generate significant variance in user perceptions of control or mastery; therefore, the hypothesis was not supported. The traditional approach would stop at this point and conclude that search-term specificity did not influence the dependent variable. For Hypothesis 1, the unstandardized regression equation was:
Dominance = 6.06 + (0.39) Specificity (F(1, 125) = 1.45, p = .23, R2= .011)
For the effect-based psychological states analysis, Hypothesis 2 predicted a positive effect of perceived relevance on self-reported dominance. A linear regression without the search term (media attributes) factor was run and the hypothesis was supported. For Hypothesis 2, the unstandardized regression equation was:
Dominance = 3.68 + (0.75) Perceived relevance (F(1, 125) = 27.89, p < .001, R2= .182) Hypothesis 3 predicted that perceived relevance would mediate the relationship between the number of search terms and self-reported dominance. For the mediation analysis based on the Baron and Kenny procedure, three regression equations were run. The first regression was the same as that used for statistical testing of Hypothesis 1, and the third regression equation was the same as that used for testing the manipulation check. The second regression equation was similar to that used for testing of Hypothesis 2 but media attributes (search term specificity) were added
as an independent variable. The results showed that the total effect (c) in the first equation was not significant, c = 0.39, t(125) = 1.21, p = .23, suggesting that mediation did not occur. The path diagram for this model is depicted in Figure 3.
Dominance = 6.06 + (0.39) Specificity (F(1, 125) = 1.45, p = .23)
Dominance = 3.66 + (0.08) Specificity + (0.75) Perceived relevance (F(2, 124) = 13.88, p < .001)
Perceived relevance = 3.22 + (0.41) Specificity (F(1, 125) = 5.24, p < .05)
Next, mediation analysis based on the bootstrap test was examined. The test showed that the mediated effect significantly differed from zero at p < .05, supporting the hypothesis. The point estimate of the mediated effect, ab, the mean of the bootstrap distribution, was 0.30 (see Table 1). The comparison between the Baron and Kenny procedure and the bootstrap test confirmed that the former indeed suffers from low statistical power, failing to detect mediation when the bootstrap test revealed it.
The procedure for computing effect sizes for indirect effects of bootstrap tests remains contested. Preacher and Hayes (in press) suggest using the product of the standardized regression coefficient a~ and standardized regression coefficient b~ as an estimate of effect size (see
Mackinnon & Dwyer, 1993 for other measures of effect size). In Experiment 1, a~ = 0.20 and b~
= 0.42; hence, the effect size was equal to .084. That is, 8.4% of the variance in the dependent variable (Dominance) was explained by the indirect effect, a substantially higher proportion of the variance explained by the independent variable (Specificity) than in Hypothesis 1 (1.1%).
Experiment 2: Interactivity and Media Credibility
The second experiment examined the impact of interactivity on credibility evaluations of television news sites. Media credibility is theorized to be a cognitive structure (Kosicki & McLeod, 1990) activated by exposure to news coverage of particular events or engagement with
interface features that invite user involvement with news content. Credibility assessments vary across different media channels (Newhagen & Nass, 1989; Sundar, 1999; Wathen & Burkell, 2002), suggesting that the packaging and presentation of information plays an important role in the evaluation of news. Owing to technological features that allow users to customize
information delivery, selectively access multimedia content, and engage in message exchange, theInternetallows“individualsto controltheirdefinition ofnewsinstead ofdepending on what producersoreditorshave predefined asnews”(Murrie, 2001). By facilitating personalized news experiences with greater individual relevance, interactive features may thus cultivate impressions of media credibility (Bucy, 2004c).
From an attributes-based perspective, the relevant question is whether variations in interactive behaviors will influence media credibility. For the study, two stimulus conditions, labeled interactive and noninteractive, were created and a manipulation check was run to confirm perceptual differences between conditions. The interactive condition instructed
participants to actively utilize features of the news interface, while the noninteractive condition simply asked participants to read three online news stories. The assumption underlying this operationalization is that perceived interactivity will increase the more that participants actually use interactive features, such as online polls, e-mail, and slide shows. Thus, even though the interactive condition was represented by different applications, the use of each application fell into the same class of online behavior. The reading condition, on the other hand, did not require participants to use any interactive features and constituted a different class of online behavior, one characterized by less active engagement with the site content.8Conceptual labels were again used to represent different stimulus conditions; however, the analysis actually tested the
relationship between user engagement with media attributes (interactive features) and subsequent evaluations. Accordingly, it is predicted that:
H1: Credibility ratings of online news sites will be higher for interactive conditions than
noninteractive conditions.
From a psychological-states perspective, the relevant question is whether the variation of perceived interactivity (i.e., perceptions arising from participatory online behaviors) will
influence evaluations of online news credibility. The same stimulus conditions were used to generate variance in user ratings of perceived interactivity, and then user ratings were employed as the independent variable. Although variations in media attributes were included in the
experimental design, what is actually being examined is the relationship between the relevant psychological state –perceived interactivity –and the dependent variable, media credibility. Previous research has found perceived interactivity, as a variable subjective perception, to be positively associated with a range of online evaluations, including attitude towards the website (Wu, 2005), attitude towards the ad, attitude towards the brand, and purchase intention (Cho & Leckenby, 1999). Similarly, in a news context, perceived interactivity should serve as evidence of subjective involvement with the medium and result in positive evaluative outcomes (Bucy, 2004a, 2004b, 2004c). Accordingly, the following hypothesis is proposed:
H2: Perceived interactivity will be positively associated with assessments of media
credibility.
From the perspective of the mediation model, credibility assessments may be influenced not just directly by the level of interactivity but also indirectly through the perception of
interactivity elicited by these tasks. Therefore, the media stimulus condition serving as the independent variable and the relevant psychological state acting as the mediator should both be included in statistical testing. Research in interactive advertising (e.g., Wu, 2005) has confirmed the mediating role that perceived interactivity plays when considered in conjunction with media
attributes (interface features). We would expect a similar dynamic to explain evaluations of online news. Accordingly, it can be hypothesized that:
H3: Perceived interactivity will mediate the relationship between interactive tasks and
evaluations of media credibility. Method
Design. To test these hypotheses, a single factor (website interactivity) between-subjects experiment was conducted. To maximize ecological validity, participants were assigned to visit actual news sites rather than a researcher-designed mock news page. Before starting, participants were randomly assigned to a single broadcast network news site (ABC, CBS, or NBC).
Depending on assignment, they either performed a series of interactive tasks or engaged in a noninteractive (reading) task, guided by a set of printed instructions.9
The interactive task instructed participants to engage in three online activities, including voting in a poll of the day, viewing a slide show of their favorite candidate, e-mailing the news organization about their election coverage, or a similar activity. The noninteractive task asked subjects to read the lead story on the news home page, plus two other stories of interest. The intent of these tasks was not to saturate subjects with political information but to immerse them in a directed way in the online news environment long enough to cultivate meaningful
impressions of the site.
Independent and mediator variables. For the effect-labeled media attributes test, online activities designated as interactive or noninteractive served as the independent variable. The applicability of these labels was confirmed with the manipulation check.
For the effect-based psychological states test, perceived interactivity, measured by a 5-point perceived interactivity scale (1 = not at all and 5 = very), served as the independent variable. Previous studies have demonstrated that a single-item measure of perceived
interactivity can accurately reflect the extent to which users subjectively experience interactivity (e.g., Sundar, Kalyanaraman, & Brown, 2003). Different media stimulus conditions were
employed to generate variance in user perceptions.
For the mediation test, interface tasks served as the independent variable and the 5-point perceived interactivity measure was included in the model as a mediator. As with Experiment 1, no check was required to justify the applicability of the message manipulation, as participant perceptions again served as the independent variable of interest. Again, variation in the media attribute (specified online activities) was objectively verifiable independent of participant perceptions
Dependent variables. Media credibility was measured with a series of five Likert-type items found to reliably tap credibility in previous research (see Bucy, 2003). The items consisted of believable, reliable, fair, accurate, and credible (1 = not at all to 5 = very). Reliability analysis was performed on the scale, Cronbach’salpha= .88.
Participants. A total of 74 undergraduate students from a large Midwestern university, including 39 females and 35 males, participated in the experiment in exchange for extra credit. Participants ranged in age from 18 to 30 (M = 20 years). Most were Caucasian (90.5%, n = 67); other participants were either Asian, African-American, or Hispanic. As with Experiment 1, overall Web use was high, as might be expected with a student subject pool. More than four in five (86.5%) reported daily Web use; the mean number of days spent online per week was 6.77.
Procedure. Participants were first asked to answer a pre-stimulus questionnaire about their media use and demographics. They were then randomly assigned to a broadcast network news site and asked to either engage in a set of interactive tasks or take part in a noninteractive reading task. As a distractor, participants also visited the campaign site of either leading
candidate, George W. Bush or Al Gore, and watched a television network news story about the presidential debates.
After spending 5 minutes on each site, participants completed a series of emotional and evaluative measures assessing their affective reactions to and evaluative perceptions of the site. Only the results for credibility are reported here. The study, conducted in the two weeks before and after the contentious 2000 presidential election, was completed before the final results of the election were known, so interest in the news was consistent throughout.
Manipulation check. To verify that participants perceived the interactive condition as intended, a one-way analysis of variance was run to examine the main effect. As expected, participants reported significantly higher levels of perceived interactivity after performing a series of interactive activities than after the noninteractive reading task, F(1, 72) = 16.66, p < .001,η2= .188.
However, as with Study 1, the distribution of responses within each condition revealed a dispersed pattern (see Figure 4). Again, there was a considerable range of responses even among participants exposed to the same conditions: for both interactive activities and the noninteractive reading task, the responses ranged from 1 to 5. Moreover, comparing the distributions of the two conditions revealed that 23 participants in both task groups (31.1% total) reported the same responses, although they performed different online activities. Again, such findings highlight the importance of subjective responses to experimental stimuli and the role that individual
differences might play in determining outcomes. Results
Hypothesis 1 predicted that the interactive condition would produce higher credibility ratings than the noninteractive condition. As with Experiment 1, the hypothesis was tested with a regression equation, which again was not significant. The hypothesis therefore was not supported.
The traditional approach would stop at this point and conclude that there was no relationship between interactivity and credibility. The unstandardized regression equation was:
Credibility = 3.96 + (0.09) Interactivity (F(1, 72) = .37, p = .55, R2= .005)
For the effect-based psychological states analysis, Hypothesis 2 predicted a positive effect of perceived interactivity on evaluations of news site credibility. A linear regression was run without considering the interactivity manipulation and the hypothesis was supported. For Hypothesis 2, the unstandardized regression equation was:
Credibility = 3.50 + (0.15) Perceived interactivity (F(1, 72) = 6.94, p < .05, R2= .088) Hypothesis 3 predicted that perceived interactivity would mediate the relationship between levels of interactivity on television news sites and evaluations of news site credibility. For the mediation analysis based on the Baron and Kenny procedure, the following three
regression equations were tested, similar to Experiment 1. The results showed that the total effect (c) in the first regression equation was not significant, c = 0.09, t(72) = .61, p = .55, pointing to a lack of mediation. The path diagram is shown in Figure 5.
Credibility = 3.96 + (0.09) Interactivity (F(1, 72) = .37, p = .55)
Credibility = 3.49 + (-0.09) Interactivity + (0.17) Perceived interactivity (F(2, 71) = 3.60, p < .05)
Perceived interactivity = 2.82 + (1.02) Interactivity (F(1, 72) = 16.66, p < .001) Next, mediation effects based on the bootstrap test were examined. The bootstrap test showed that the mediated effect was significantly different from zero at p < .01, supporting the hypothesis. The point estimate of the mediated effect, ab, the mean of the bootstrap distribution, was 0.175 (see Table 2). Again, the comparison between the Baron and Kenny procedure and the bootstrap test revealed that the former suffers from low statistical power, failing to detect
In Experiment 2, a~ = 0.43 and b~ = 0.33; hence, the effect size was equal to .142. That is,
14.19% of the variance in the dependent variable (Credibility) was explained by the indirect effect, a considerably higher proportion of the variance explained by the independent variable (Interactivity) than in Hypothesis 1 (0.50%).
Discussion
Consistent with recent calls for conceptual and empirical reformation in media effects research (Holbert & Stephenson, 2003; Newhagen, 2002; O'Keefe, 2003; Potter & Tomasello, 2003), this study has demonstrated that the three-variable mediation model can substantially fortify the theoretical framework and research design of experimental studies, uncovering results that would otherwise remain masked. Empirically, two experiments confirmed that the effect-labeled media attribute and effect-based psychological state approaches both test an incomplete causal model: the former fails to consider the relationship between psychological states and outcomes, while the latter only examines the relationship between psychological states and outcomes. Moreover, the effect-labeled media attribute approach assumes homogeneity of response to media stimuli, ignoring variation in user perceptions and leading to non-significant findings or low explanatory power. In short, both the attributes-based and psychological states approaches conflate message properties with user responses and reveal, under close examination, discrepancies between conceptual and operational-level hypotheses.
In this study, testing different conceptual models with perceptual measures alternately positioned as manipulation checks, independent variables, or mediators produced noticeably different outcomes. In the first experiment, situating search-term specificity as the predictor and self-reported dominance as the dependent variable produced a nonsignificant result in tests using simple regression. A similar pattern was observed in the second experiment, which employed levels of interactivity as the independent variable and assessments of media credibility as the
predicted outcome. Yet in each test, by including the perceptual measure as an independent variable, a positive and significant relationship was found. A more complete and intellectually defensible causal explanation was obtained, however, for the tests of mediation, which included both media attributes and psychological states (user perceptions) in the same model. Although not applicable in all empirical tests of media and technology effects, mediation models in the two studies reported here overcame the conceptual limitations and initial nonfindings produced by conventional analytical methods and accounted for the stimulus properties on which
communication and technology research relies.
The mediation model can also be applied to more complex designs, of which two are at least worthy of mention. First, the multiple-mediator model entails designs that involve more than one mediator. While explaining the association between an independent variable and dependent variable, these mediators can work either at the same stage (see Figure 6a) or at a series of different stages (see Figure 6b). Statistical procedures for examining mediated effects for multiple mediators at the same stage are summarized by Preacher and Hayes (in press), while procedures for multiple mediators at a series of stages are addressed by Cheung (2007). Second, mediated moderation and moderated mediation models include designs that involve both
mediation and moderation. Moderation, usually referred to as interaction in communication research, specifies various conditions under which the direction and/or strength of the relationship between the independent and dependent variables occurs. Mediated moderation implies that the interaction effect of the independent and moderator variables on the dependent variable depends on a mediator, while moderated mediation implies that the direction and/or strength of the indirect effect depends on a moderator (Bucy & Tao, 2007).10
Understanding how people process media content, whether interface features or televised violence, is fundamental to advancing the field of communication, since information processing
explains the cognitive operations and psychological mechanisms that produce media effects. The mediation model explores how these psychological mechanisms operate. In this paper we have argued that current conceptualizations of media stimuli –as effect-labeled media attributes or effect-based psychological states –hinder, if not outright preclude, the use of mediation analyses. Although the importance of mediation has been emphasized by many media scholars, the
application of mediation in experimental research is relatively rare. Rethinking analytical strategies to explicitly accommodate media attributes (message properties) as well as user perceptions within the same experimental design may reveal relationships that more commonly employed models and techniques are unable to illuminate. With this approach, experimental effects research could experience significant advancement. Further work in this area should discuss the theoretical and statistical issues associated with mediation in more detail to firmly establish thetechnique’srelevanceand application –and continue to apply the model to other areas of experimental research, particularly to other types of media stimuli and within-subjects designs.
Notes
1
Chaffee (1996) observes that empirical theory can be construed at two different levels, the conceptualand operationallevel.Theory attheconceptuallevel“involvesboth abstractconcepts and arelationship between them,”whiletheory attheoperationallevelinvolves“observable phenomena[and]predicted relationships”(Chaffee, 1996, p. 17). The comparison between abstract concepts and observable phenomena is referred to as concept explication, which examineswhetherthetwo “relateto thesamephenomena”(Chaffee, 1996, p. 17).
2
Mediation models are growing in popularity in communication research (see, for example, Beaudoin & Thorson, 2004; Eveland, 2002; Eveland, Shah, & Kwak, 2003; Holbert & Stephenson, 2003; Potter & Tomasello, 2003) but are still not widely used in either survey or experimentally based work.
3
For testing mediated effects in within-subjects experimental designs, see Judd, Kenny, and McClelland (2001).
4
The bootstrap mediation test involves four steps. First, 1,000 bootstrap samples are created by resampling with replacement from the original sample. Each bootstrap sample has the same sample size as the original sample. Second, the mediated effect (ab) is computed for each bootstrap sample. The 1,000 mediated effects construct the bootstrap distribution. Third, for a two-tailed 95% bootstrap confidence interval, the 25thand 976thscores of the bootstrap distribution serve as the lower and upper bounds, respectively. The null hypothesis –that the mediated effect is zero –is rejected if the bootstrap confidence interval does not include zero (i.e., there is a mediation effect), or is supported if the bootstrap confidence interval includes zero (i.e., there is no mediation effect). Fourth, the estimate of the mediated effect is the average mediated
effect computed over the 1,000 bootstrap samples (i.e., the sum of 1,000 mediated effects divided by 1,000).
Using SPSS and SAS macros provided by Preacher and Hayes (2004), it is possible to execute the command set and specify the dependent variable (y), the independent variable (x), the mediator variable (m), the number of bootstrap samples (boot), and the name of the data file (data). (The name of the data file is only required for SAS.) Instructions and syntax files are available at http://www.comm.ohio-state.edu/ahayes/sobel.htm. The output shows the mediated effect and its p value computed by the Baron and Kenny (1986) procedure, the normal
distribution of product test, and the bootstrap distribution of product test.
5
From research conducted in the U.S., key findings reveal that typical Web queries are short. Two thirds of queries contain no more than two terms (Spink, 2003). Hence, one-term queries were chosen for the low relevance condition and three-term queries were chosen for the high relevance condition. In addition, examination of search logs has shown that advanced features such as Boolean operators or quoted search terms are seldom used, and most sessions consist of only one or two Web queries (Chau, Fang, & Sheng, 2005; Jansen & Spink, 2006; Spink, 2003; Wang, Berry, & Yang, 2003). Interestingly, although the number of users and technical features of Web search engines has grown rapidly in recent years, public Web search behavior has shown no substantial change (Spink, 2003).
6
News images, news descriptions, and repeated news titles were deleted in order to minimize confounding factors. The capacity to perform live searches through Google News was also removed from the stimulus interface to prevent participants from changing the search terms and results, which would have nullified the experimental manipulation.
7
8
As Tremayne (2005) notes, when researchers create high and low interactivity conditions by varying Web structures and then observe significant effects on a dependent variable, it is often unclear whether study participants actually exhibited differential interactive behavior and this led to the effect, or whether they simply perceived one site to be more interactive and that led to the effect. Our operationalization of interactivity insured that participants assigned to the interactive condition, in fact, exhibited interactive behaviors; those assigned to the noninteractive condition were simply asked to read site content and not actively participate beyond this.
9
Since the news sites were subject to frequent updating, this required periodic revision of the instructions to adjust for expired links and other changes. The instructions were written in such a way as to minimize the need for revision. The noninteractive conditions merely asked subjects to read the lead story on the home page, plus two other stories that were of interest, while the interactive conditions required that subjects engage with special interactive features on the sites. Since these features are costly to develop, news organizations have a vested interest in keeping them for a length of time as a consistent fixture of the pages. Revisions to the instructions were therefore limited for the most part to re-linking features that had been moved to another part of the site.
10
For statistical procedures examining mediated moderation and moderated mediation, see Muller, Judd, and Yzerbyt (2005) and Preacher, Rucker, and Hayes (2007).
References
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical consideration. Journal of Personality and Social Psychology, 51(6), 1173-1182.
Beaudoin, C. E., & Thorson, E. (2004). Testing the cognitive mediation model: The roles of news reliance and three gratifications sought. Communication Research, 31(4), 446-471. Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and bootstrap estimates
of variability. Sociological methodology, 20, 115-140.
Bolls, P. D., & Lang, A. (2003). I saw It on the radio: The allocation of attention to high-imagery radio advertisements. Media Psychology, 5(1), 33-55.
Bucy, E. P. (2003). Media credibility reconsidered: Synergy effects between on-air and online news. Journalism & Mass Communication Quarterly, 80(2), 247-264.
Bucy, E. P. (2004a). Interactivity in society: Locating an elusive concept. The Information Society, 20(5), 375-385.
Bucy, E. P. (2004b). The interactivity paradox: Closer to the news but confused. In E. P. Bucy & J. E. Newhagen (Eds.), Media access: Social and psychological dimensions of new technology use (pp. 47-72). Mahwah, NJ: Lawrence Erlbaum Associates.
Bucy, E. P. (2004c). Second generation net news: Interactivity and information accessibility in the online environment. JMM: The International Journal on Media Management, 6(1/2), 102-113.
Bucy, E. P., & Newhagen, J. E. (1999). The emotional appropriateness heuristic: Processing televised presidential reactions to the news. Journal of Communication, 49(4), 59-79. Bucy, E. P., & Tao, C.-C. (2007). The mediated moderation model of interactivity. Media
Psychology, 9(3), 647-672.
Catledge, L. D., & Pitkow, J. E. (1995). Characterizing browsing strategies in the World-Wide web. Computer Networks and ISDN Systems, 27(6), 1065-1073.
Chaffee, S. H. (1996). Thinking about theory. In M. B. Salwen & D. W. Stacks (Eds.), An integrated approach to communication theory and research (pp. 15-32). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Chau, M., Fang, X., & Sheng, O. R. L. (2005). Analysis of the query logs of a Web site search engine. Journal of American Society for Information Science and Technology, 56(13), 1363-1376.
Cheung, M. W. L. (2007). Comparison of approaches to constructing confidence intervals for mediating effects using structural equation models. Structural Equation Modeling, 14(2), 227-246.
Cho, C.-H., & Leckenby, J. D. (1999). Interactivity as a measure of advertising effectiveness. In M. S. Roberts (Ed.), Proceedings of the 1999 Conference of the American Academy of Advertising (pp. 162-179). Gainesville, FL: American Academy of Advertising. Collins, L. M., Graham, J. J., & Flaherty, B. P. (1998). An alternative framework for defining
mediation. Multivariate Behavioral Research, 33(2), 295-312.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
Eveland, W. P., Jr. (2002). News information processing as mediator of the relationship between motivations and political knowledge. Journalism & Mass Communication Quarterly, 79(1), 26-40.
Eveland, W. P., Jr., Shah, D. V., & Kwak, N. (2003). Assessing causality in the cognitive mediation model. Communication Research, 30(4), 359-386.
Fallows, D. (2005). Search engine users. Washington, DC: Pew Internet & American Life Project. Retrieved May 8, 2006 from
http://www.pewinternet.org/pdfs/PIP_Searchengine_users.pdf.
Ford, N., Miller, D., & Moss, N. (2005). Web search strategies and human individual differences: Cognitive and demographic factors, Internet attitudes, and approaches. Journal of the American Society for Information Science & Technology, 56(7), 741-756.
Geiger, S., & Newhagen, J. E. (1993). Revealing the black box: Information processing and media effects. Journal of Communication, 43(4), 42-50.
Hargittai, E. (2007). The Social, Political, Economic, and Cultural Dimensions of Search Engines: An Introduction. Journal of Computer-Mediated Communication, 12(3). Retrieved fromhttp://jcmc.indiana.edu/vol12/issue3/hargittai.html.
Holbert, R. L., & Stephenson, M. T. (2003). The importance of indirect effects in media effects research: Testing for mediation in structural equation modeling. Journal of Broadcasting & Electronic Media, 47(4), 556-572.
Jansen, B. J., & Spink, A. (2006). How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Information Processing & Management, 42(1), 248-263.
Judd, C. M., Kenny, D. A., & McClelland, G. H. (2001). Estimating and testing mediation and moderation in within-subject designs. Psychological Methods, 6(2), 115-134.
Kayany, J. M., & Wotring, C. E. (1996). Relational control and interactive media choice in technology-mediated communications situations. Human Communication Research, 22(3), 399-421.
