PLEASE SCROLL DOWN FOR ARTICLE
On: 7 October 2009
Access details: Access Details: [subscription number 731896418] Publisher Psychology Press
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Visual Cognition
Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713683696
Binding or prioritization: The role of selective attention in visual short-term
memory
Yei-Yu Yeh a; Cheng-Ta Yang a; Yu-Chin Chiu a a National Taiwan University, Taipei, Taiwan. Online Publication Date: 01 July 2005
To cite this Article Yeh, Yei-Yu, Yang, Cheng-Ta and Chiu, Yu-Chin(2005)'Binding or prioritization: The role of selective attention in visual short-term memory',Visual Cognition,12:5,759 — 799
To link to this Article: DOI: 10.1080/13506280444000490 URL: http://dx.doi.org/10.1080/13506280444000490
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Binding or prioritization: The role of selective
attention in visual short-term memory
Yei-Yu Yeh, Cheng-Ta Yang, and Yu-Chin Chiu
National Taiwan University, Taipei, Taiwan
Whether selective attention binds features in visual short-term memory or priori-tizes selection for memory consolidation and decision was investigated with a change detection paradigm. Two types of change were manipulated: Feature or conjunctions of features. Previous work suggests that the allocation of attentional resources affects binding; hence attentional shifts during retention should affect the detection of conjunction changes more than feature changes. The results of Experiments 1 and 2 showed that attention shifts had a similar impact on detecting feature and conjunction changes. Experiment 3 showed a performance benefit with a post-cue occurring 200 or 550 ms after stimulus offset, but no improvement was found when prioritization occurred with a delay of 800 ms. The results of Experiment 4 suggested that signals from both feature changes and conjunction changes contribute to detection. The theoretical implications are discussed.
The visual world is rich with details and yet little information seems to be encoded during each eye fixation when people look at the world (see Henderson & Hollingworth, 1999; Rensink, 2002 for reviews). The processes that con-tribute to the building of a stable visual representation have become an important research issue. With an understanding of these processes, researchers can ultimately unveil what takes place beyond the retinal receptors and link perception to cognition.
Various forms of visual memory must be involved to build a coherent representation of the external visual world. In his seminal work, Sperling (1960) Please address all correspondence to Yei-Yu Yeh, Department of Psychology, National Taiwan University, No. 1, Sec. 4, Roosevelt Road, Taipei, Taiwan 106. Email: [email protected]
This research was supported by a contract from the Asian Office of Aerospace Research & Development (AOARD-02-4011). Dr Terence Lyons was the manager and Dr Brian Tsou was the technical monitor. We thank Kris Chang, Hsuan-Fu Chao, Yi-Jye Chen, Yang-Ming Huang, Bo-Cheng Kuo, and William Gin for their assistance. We thank Dr Olwen Bedford for her technical editing. We are grateful to Dr Humphreys and an anonymous reviewer for their comments and suggestions. Parts of the results were presented in the 10th annual meeting of the Chinese Psychological Association in Tainan, 2002, and parts of the results were presented in the 44th annual meeting of the Psychonomic Society in Vancouver, 2003.
# 2005 Psychology Press Ltd
http://www.tandf.co.uk/journals/pp/13506285.html DOI:10.1080/13506280444000490 VISUAL COGNITION, 2005, 12 (5), 759±799
demonstrated the existence of a very brief memory of high capacity. After viewing a 3 6 4 array of letters, participants in his study were asked to recall all the letters contained in the display. In this whole-report condition, there was recall of only three to four letters. However, participants could accurately report a whole row of four letters in a partial-report condition in which a tone sounded shortly after stimulus offset and indicated which row to report. Because parti-cipants could not predict which row would need to be reported beforehand, this high accuracy was taken as evidence for an initial high-capacity memory con-taining almost all the information in the array, from which participants could report a cued row: iconic memory (Neisser, 1967).
Sperling's (1960) seminal work on iconic memory instigated several decades of empirical studies with theoretical controversies over the number and the nature of the distinct types of persistence underlying performance. Coltheart (1980) proposed three distinct types of persistence: Neural persistence, visible persistence, and informational persistence. Di Lollo and Dixon (1988) suggested two forms: visible persistence and the visual analogue representations that are time locked to the onset and offset of stimulation respectively. Loftus and Irwin (1998) argued that visual stimulation produces a sensory response (visible persistence) from which information is acquired (information persistence) to produce the partial-report performance. Researchers also disputed whether the locus of iconic memory was in the retinal receptors (Long, 1980) or beyond the retina (Adelson & Jonides, 1980) and disagreed about the relative decay rate of location and identity codes that underlie the errors in partial-report performance (e.g., Chow, 1986; Mewhort, Campbell, Marchetti, & Campbell, 1981).
Although the theoretical controversies remain unresolved and the theoretical usefulness of iconic memory has been challenged (Haber, 1983), there is no question that iconic memory as a hypothetical mechanism represents a fleeting visual memory from which phenomenological report is possible. The retention of iconic memory is recognized as short lived and thought to last between 250 and 500 ms (cf. Chow, 1986). Information is lost after the retention interval unless it has been transferred into a more durable form of memory, such as the visual short-term memory (VSTM). VSTM (Pashler, 1988; Phillips, 1974) contains stimulus-specific properties that may last up to 16 s for low-contrast sinusoidal gratings (Tanaka & Sagi, 2000) and up to 30 s for the spatial fre-quency of grating patterns (Magnussen, Greenlee, Asplund, & Dyrnes, 1991). Thus, VSTM is ideal for preserving detailed information over a longer interval, although it does have a limited capacity.
Attention appears to play an important role in transferring stimuli from iconic memory to VSTM. Yet, its exact roles are unclear. Two major theoretical views have been proposed: Prioritized selection and feature binding. The prioritization hypothesis has a long history (e.g., Becker, Pashler, & Anstis, 2000; Bundesen, 1990; Busey & Loftus, 1994; Duncan & Humphreys, 1989, 1992; Gegenfurtner & Sperling, 1993; Palmer, 1991; Potter, 1976; Schmidt, Vogel, Woodman, &
Luck, 2002). By guiding selective attention to a row with a tone in the partial-report method, Sperling (1960) implicated the role of selective attention as ``selective readout'' in sampling information from iconic memory for verbal report. Although a cue may take about 125 ms to be effective in directing attention to an item in iconic memory (Eriksen & Lappin, 1967), a postcue (following the stimulus offset) can improve letter recognition (Dixon, Gordon, Leung, & Di Lollo, 1997). For example, a postcue appearing 50 ms after stimulus display can enhance the detection of a change in a coloured square (Becker et al., 2000) and a postcue occurring 120 ms after stimulus offset can improve length discrimination (Palmer, 1988). Thus, postcueing enhances per-formance as long as the cue appears early during the retention interval after stimulus offset. Dixon and his colleagues further showed that the post-cueing effect depends on attention (Dixon et al., 1997; Giesbrecht, Dixon, & Kingstone, 2001).
The binding hypothesis is more recent, as compared to the prioritization hypothesis. Wheeler and Treisman (2002) suggested that VSTM is feature-based and attention binds features into object representations in VSTM. With the same assumption about the role of visual attention in perception, Irwin (1992) pos-tulated that attention to an object results in a unified bound object description that can be maintained across saccades to build representations of visual scenes. Rensink (2000) also argued that visual attention is necessary to bind features into a coherent object representation and to maintain this representation in VSTM.
Evidence strongly supporting the binding hypothesis comes from Wheeler and Treisman's (2002) study. Using a change detection paradigm, they manipulated the number of elements in the display and the type of change. An initial display of coloured stimuli was first shown for retention until a test display occurred. When change detection operated on a single feature such as colour or shape, two new colours or shapes replaced the old ones. When change detection operated on the combination of two features such as colour and location, the two stimuli exchanged positions so that successful detection required accurate binding of two features. Even though both types of change detection contained two change signals (e.g., two new colours replaced the old ones, or two objects exchanged positions), performance was worse in detecting conjunction changes. More important, the difference in performance occurred only when the test contained the same number of elements as the initial display (whole display condition). No difference was found when only a single probe was presented in the test. A whole display, they argued, requires reallocation of attention and thus disrupts feature binding in VSTM. As a result, performance was worse in detecting conjunction changes.
Though the prioritization and binding hypotheses are not necessarily mutually exclusive, they extend a long-standing debate on the role of selective attention in perception (e.g., Bundesen, 1990; Dosher & Lu, 2000; Duncan & Humphreys, 1989, 1992; Eckstein, Thomas, Palmer, & Shimozaki, 2000; Hill-BINDING OR PRIORITIZATION 761
yard, Vogel, & Luck, 1999; Shaw, 1978; Shore, Spence, & Klein, 2001; Treisman & Gelade, 1980). By extending the debate from the study of per-ception into the domain of VSTM, researchers can link sensory analysis to the retention of visual representations (e.g., Magnussen et al., 1991; Sakai & Inui, 2002). This link opens up the opportunity to bring to bear on VSTM notions about how selective attention influences perceptionÐin particular, is the content of VSTM feature or object based (Luck & Vogel, 1997; Vogel, Woodman, & Luck, 2001; Wheeler & Treisman, 2002; Xu, 2002).
The first objective of this study was to clarify how selective attention influences VSTM. Because researchers proposing both the binding hypothesis (Wheeler & Treisman, 2002) and the prioritization hypothesis (e.g., Schmidt et al., 2002) have used the change detection paradigm to address the issue, we adopted the same paradigm to address a series of questions. First, is the influ-ence of an attentional shift on detecting conjunction changes greater than its impact on detecting feature changes? When attention shifts away from the stored representations during retention to another stimulus processing, attention real-location ought to occur. If the detection of conjunction changes across multiple elements of a whole display is worse because of the reallocation of attention, the detection of conjunction changes should be worse with an attentional shift. We investigated this question in Experiment 1.
Second, does spatial cueing during retention have the same benefit for detecting these two types of changes? Selective attention to a location enhances target processing through spatial rehearsal (Awh, Jonides, & Reuter-Lorenz, 1998). If attention acts to bind features in VSTM, spatial cueing on the position where a change could occur in the test display should benefit performance. Moreover, this benefit should be greater for conjunction targets than for feature targets. Wheeler and Treisman (2002) proposed that attentional resources affect only the detection of conjunction changes based on the assumption that attention affects feature binding. In contrast, prioritization in information transfer need not differentially affect conjunction over feature targets. We examined this issue in Experiment 2.
Third, does the prioritized selection process in the change detection paradigm follow the temporal characteristics that Sperling (1960) demonstrated in his seminal work on iconic memory? The binding hypothesis does not address the temporal characteristics of the influence of attention on VSTM. On the other hand, the prioritization hypothesis would predict that early transfer of infor-mation into VSTM should allow more time for memory consolidation before a test display occurs. As the result, a spatial cue that occurs before iconic memory decays should benefit detection. In contrast, a late cue that appears after iconic memory decays should not influence change detection. Experiment 3 addressed this question.
The second objective of this study was to address a critical finding that distinguishes the detection of conjunction changes from the detection of feature
changes in different test contexts. Wheeler and Treisman (2002) interpreted the worse performance in detecting conjunction changes with a whole display as the consequence of reallocating or disrupting attention when a multiple-element test display was used. We conceptualized the difference in performance between a whole display versus a single probe as reflecting the difference in signal detection (Eckstein et al., 2000; Palmer, Verghese, & Pavel, 2000) across many elements versus from one stimulus respectively.
Change detection is fundamentally a visual search task (Zelinsky, 2001), except that the target is not defined until a comparison is made between the test display and VSTM of the initial display. Given that a decision integration hypothesis or a signal detection model (Palmer, 1990; Palmer et al., 2000; Shaw, 1978, 1982) can fit visual search even for conjunction targets (Eckstein et al., 2000), decision integration can underlie performance in change detection. According to the decision integration hypothesis, the effect of attention is on decision rather than on perception. Each element in a display represents a noisy source of information. When the number of display elements increases, the integration of information for response selection declines because each new source represents a chance to err in signal detection. With this perspective, two factors are critical in influencing performance: signal strength and the number of noisy information sources in the decision process.
The finding that performance with a single probe was equally accurate in detection between conjunction and feature changes (Wheeler & Triesman, 2002) suggests that a full representation of the initial display exists for both types of changes. Performance may deteriorate for detecting conjunction targets with a whole display because of the effects of a lower signal-to-noise ratio (S/N) in detection. Wheeler and Treisman rejected the possibility that the difference in performance between a whole display and a single probe resulted from the difference in the number of decisions that must be made in each condition. To support this statement, they showed that a simultaneous cue in a whole display, to limit the decision to one element, failed to benefit the detection of conjunction changes. However, we argue that a simultaneous cue cannot instantly and completely exclude noisy information sources in the decision process because it takes time to process a cue to initiate selection. While the selection mechanism is activated, signals from irrelevant elements are processed in the decision process. Thus, the number of comparisons in the decision matters for change detection. Experiment 4A investigated how the number of comparisons influ-ences change detection.
A whole display consists of individual items at the local level and the schematic pattern composed of multiple elements at a global level. The con-junction-change condition and the feature-change condition in Wheeler and Treisman's (2002) study were similar at the local level for each changed object. The only difference was that two new colours replaced the old ones in the feature-change condition, whereas two old colours exchanged locations or BINDING OR PRIORITIZATION 763
recombined with old shapes in the conjunction-change condition. A signal of difference existed at the local level for each changed object in both conditions. No difference in performance should be observed between the two conditions if the decision is made at the local level. We tested this hypothesis in Experiment 4B.
EXPERIMENT 1
The purpose of this experiment was to test whether an attentional shift from stored representations has the same effect on detecting feature changes as it has on detecting conjunction changes. If selective attention acts to bind features in VSTM, an attentional shift should influence the detection of conjunction changes more than it affects the detection of feature changes due to attentional reallocation during retention.
We manipulated two types of attentional shift. We presented a letter during the retention interval in the central area (central shift) or in a corner of a larger area that embodied the memory display area (peripheral shift). Participants had to accurately and quickly judge the letter while maintaining the initial array. Thus, the second task should not compete for VSTM resources because visual perception should not interfere with VSTM when the task does not require formation of another VSTM representation (Phillips & Christie, 1977). Parti-cipants, however, must shift their attention away from the initial display area in the peripheral shift condition. Detection performance should deteriorate only with the peripheral shift, although the task demand is the same as in the central shift condition. The critical issue is whether the deterioration is of equal mag-nitude for detecting feature and conjunction changes.
Method
Participants. Four undergraduate and eight graduate students at the National Taiwan University volunteered for this experiment. One undergraduate received a bonus credit in an introductory psychology course and the others received a monetary reward of US$5 for their participation. Their ages ranged from 18 to 27 years old. All participants had normal or corrected-to-normal vision.
Equipment. A PC with a 667 MHz Intel Pentium III processor was used to run the experiment. The display monitor was a 21-inch Hitachi colour monitor with a vertical refresh rate of 75 Hz. The software E-Prime (Psychology Software Tool Inc.) was used to run the experiment.
Design and stimuli. Six blocks of trials were run for the 2 (judgement type: Same/different) 6 2 (type of change: Colour/conjunction) 6 3 (type of attentional shift: Control/central/periphery) experimental design. Following
Wheeler and Treisman's (2002) method, the display elements were composed of six single-coloured squares on a grey background (x = .289, y = .342; luminance = 9.7 cd/m2). Each square was randomly assigned with a unique colour from a
set of eight: Red (x = .621, y = .338; 15.6 cd/m2), green (x = .298, y = .593; 21.8
cd/m2), orange (x = .430, y = .377; luminance = 32.7 cd/m2), yellow (x = .436, y
= .486; luminance = 34.1 cd/m2), blue (x = .150, y = .070; luminance = 8.8 cd/
m2), cyan (x = .216, y = .318; luminance = 29.4 cd/m2), magenta (x = .262, y =
.132; luminance = 21.5 cd/m2), and black. Each square subtended a visual angle
of 0.738 (horizontal) 6 0.708 (vertical) at the viewing distance of 60 cm and was placed in one of eight possible locations equally spaced in an invisible square grid that subtended 8.68 (horizontal) 6 8.38 (vertical) around the screen centre. Each trial consisted of two stimulus displays for the primary task: memory and test. Changes occurred in the test display by either replacing two squares with two new colours that did not occur in the memory display (colour change) or by swapping two coloured squares such that detection requires accurate memory of the combination of colour and location (conjunction change). During the retention interval between the two displays, a letter from a set of 10 (A, E, I, O, U, B, G, K, P, X) was shown either in the centre of the screen or in one of the four corners of a 12.458 6 12.028 area. Each letter subtended a visual angle of 0.738 6 0.708. The corner location was randomly selected on each trial.
Procedure. Prior to the experimental session, each participant practiced a block of 48 trials, half in judging a letter presented in the centre and the other half in judging a letter presented in the periphery. Six experimental blocks were then run in an order determined by a Latin-square design. Each block began with eight practice trials, and then 40 experimental trials. The test display was identical to the memory display in half of the trials, and contained two changes in the other half. As shown in Figure 1, each trial began with a white fixation cross for 1500 ms. A memory display was then presented for approximately 180 ms. After a brief delay of 100 ms, a letter was shown for the participants to press the ``1'' key of the keyboard if it was a vowel and the ``2'' key if it was not. The letter remained on screen for a total duration of 1900 ms even after participants had completed the judgement. From the stimulus onset until the response ended in a trial, participants were asked to vocally repeat names of three favourite fruits or the names of seven basic colours, as they preferred. This was done to eliminate verbal coding. A whole display was then presented until participants judged whether there was any change in the test as compared with the original memory display or until 3000 ms had elapsed. The intertrial interval was 1500 ms. Participants were instructed to press the left button of the mouse if there were no changes and to press the right button if any change occurred. They were told to make the judgement accurately. Response speed was not emphasized.
BINDING OR PRIORITIZATION 765
Figure 1. A diagram of the procedure used in Experiment 1. Different fill patterns are used to represent different colours. The left side shows an example of a colour-change condition with a central shift in retention, and the right side shows an example of a conjunction-change condition with a peripheral shift during retention.
Results and discussion
Vowel judgementA 2 (type of change: Colour/conjunction) 6 2 (central/peripheral shift) repeated measures analysis of variance (ANOVA) was used to analyse perfor-mance on this task. The only significant effect was the main effect of type of attentional shift, F(1, 11) = 3946.35, MSE = 0.001 , p < .0001. Judging a letter in the centre was more accurate (M = 0.95, SD = 0.04) than judging a letter in the periphery (M = 0.44, SD = 0.04).
Change detection
Whether participants made a correct response in vowel judgement did not significantly influence detection performance nor did it interact with the other variables in a preliminary 2 (vowel judgement: Correct/incorrect) 6 2 (judge-ment type: Same/different) 6 2 (type of change: Colour/conjunction) 6 3 (attentional shift: No shift/central shift/peripheral shift) repeated measures ANOVA. Provided that participants made a correct judgement of the letter, their accuracy data in judging the test display were analysed by a 2 6 2 6 3 repeated measures ANOVA. Because Wheeler and Treisman (2002) used accuracy as the dependent measure, we focused on this measure in graphical presentation but also present the nonparametric sensitivity measure A' (Grier, 1971) in Table 1. The three-way interaction in the accuracy data was not sig-nificant, but the two-way interaction between judgement type and type of change was significant, F(1, 11) = 7.41, MSE = 0.014, p < .05. An analysis
TABLE 1
Mean proportion correct and sensitivity in judgement with and without attentional shift in Experiment 1
Attentional shift
No shift Central shift Peripheral shift
M SD M SD M SD Same judgement Colour .85 .08 .82 .10 .83 .14 Conjunction .85 .08 .81 .14 .75 .07 Different judgment Colour .82 .11 .69 .15 .64 .15 Conjunction .65 .16 .61 .15 .48 .18 A' Colour .90 .04 .84 .07 .82 .12 Conjunction .83 .08 .80 .09 .67 .13 BINDING OR PRIORITIZATION 767
was then performed on the accuracy data for same and different judgements separately by a 2 (change type) 6 3 (shift condition) repeated measures ANOVA.
Same judgement. None of the effects reached significance (p > .1), although there was a trend for worse performance in the peripheral shift condition.
Different judgement. The main effect of change type was significant, F(1, 11) = 20.95, MSE = 0.016, p < .005, so as the effect of attentional shift, F(2, 22) = 7.89, MSE = 0.024, p < .001. The results from the Tukey test showed that detection was less accurate in the peripheral shift condition than the no-shift condition, and the latter was not significantly different from the central shift condition. The critical comparison, the interaction between attentional shift and type of change, was not significant (p > .3). An analysis of the A' data with a 2 (change type) 6 3 (shift condition) repeated measures ANOVA showed the same pattern with the two-way interaction being non significant (p > .2).
Replicating Wheeler and Treisman's (2002) finding with a whole display, per-formance was more accurate in detecting colour changes than in judging con-junction changes. As shown in Figure 2, the results further showed that a second task that does not compete for memory resources can severely disrupt change detection when this task shifts attention away from VSTM during the retention interval. More important, shifting attention away from VSTM interfered with the detection of both types of changes equally. If attention binds features in VSTM,
Figure 2. Mean hit rates of change detection as a function of type of change and shift condition in Experiment 1.
shifts should interrupt memory of conjunctions more than it disrupts memory of features. The results suggest otherwise.
EXPERIMENT 2
The goal of this experiment was to investigate whether a high priority set, provided by cueing target locations, benefits detection of feature and conjunc-tion changes equally. By assuming that selective attenconjunc-tion prioritizes the transfer of information to a durable visual memory, we expected to find an equal magnitude of any benefit. In contrast, if attention is to bind features in VSTM, we predicted a greater benefit for detecting conjunction changes. Wheeler and Treisman (2002) argued that reallocation of attentional resources affected feature binding, suggesting that the effect of attentional resources is greater for detecting conjunction changes than for feature changes.
We presented two spatial cues during the retention interval to indicate the locations where changes could occur. The two cues occurred 200 ms (early) or 800 ms (late) after the termination of the initial memory display. The cues remained on the screen during the retention interval and were also presented with the test display. Given that partial-report superiority to whole report diminishes at cue delays of 500±800 ms (Gegenfurtner & Sperling, 1993), we expected to see benefits only with early cuing on both types of change detection because early cueing allows a longer consolidation process than late cueing prior to the appearance of the test display.
Method
Participants. Twenty-seven undergraduate students at the National Taiwan University volunteered in this experiment to receive a bonus credit in an introductory psychology class. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
Design, stimuli, and procedure. This experiment used a 2 (judgement type: Same/different) 6 2 (type of change: Colour/conjunction) 6 3 (cue condition: No cue/early/late) factorial design. The same stimuli used in Experiment 1 were used in this experiment. The procedure changed in three aspects: Each block consisted of 32 experimental trials, the retention interval was 900 ms, and two spatial cues in white were presented with a cue delay (interstimulus interval; ISI) of 200 ms or 800 ms after the termination of the memory display. The spatial cues pointed to the upper left corner of the two locations where changes could occur. The cues remained on the screen until participants responded to the test display or until 3 s had elapsed after the presentation of the test display. Participants were told that when changes took BINDING OR PRIORITIZATION 769
place, the changes always appeared at the cued locations. Figure 3 shows a schematic diagram of a trial.
Results and discussion
Mean performance data are presented in Table 2 and analysed with a 2 (judgement type) 6 2 (type of change) 6 3 (cue condition) repeated measures ANOVA. Judgement type significantly interacted with type of change, F(1, 26) = 75.53, MSE = 0.016, p < .0001, and with cue condition, F(2, 52) = 23.15, MSE = 0.014, p < .0001. The three-way interaction did not reach significance (p > .2). Two separate analyses were conducted for same and different judgements.
Same judgement. The main effect of cue condition was significant, F(2, 52) = 7.07, MSE = 0.006, p < .001, showing a better judgement in the condition without the cues (M = 0.93, SD = 0.09) than in the other two conditions (early cue: M = 0.89, SD = 0.09; late cue: M = 0.89, SD = 0.13). No other effects were significant.
Different judgement. The main effect of cue condition was significant, F(2, 52) = 29.39, MSE = 0.017, p < .0001, showing a better judgement in the condition with early cues (M = 0.83, SD = 0.18) than in the other two conditions (no cue: M = 0.67, SD = 0.21; late cue: M = 0.65, SD = 0.22). The main effect of change type was also significant, F(1, 26) = 88.71, MSE = 0.025, p < .0001, showing a better performance in detecting colour changes (M = 0.83, SD = 0.15) than in detecting conjunction changes (M = 0.60, SD = 0.22). The interaction
TABLE 2
Mean proportion correct and sensitivity in judgement with and without spatial cues in Experiment 2
Cueing condition
No cue Early cue Late cue
M SD M SD M SD Same judgement Colour .93 .08 .89 .09 .87 .16 Conjunction .93 .09 .88 .10 .90 .09 Different judgement Colour .79 .15 .93 .10 .78 .13 Conjunction .55 .18 .73 .19 .52 .21 A' Colour .92 .07 .95 .04 .89 .09 Conjunction .85 .07 .88 .11 .81 .11
Figure 3. A diagram of the procedure used in Experiment 2. The left side shows an example of a colour-change condition with early cueing, and the right side shows an example of a conjunction-change condition with late cueing.
771
was not significant (p > .5). Figure 4 shows the hit rates of change detection. The analysis of A' data with a 2 (type of change) 6 3 (cue condition) repeated measures ANOVA showed a similar pattern, with a significantly higher sensitivity for detecting colour than conjunction changes and a significant benefit with early cueing. Late cueing significantly degraded sensitivity, as compared to the no-cue condition. The interaction was not significant (p > .8). Spatial cueing interfered with same judgements, suggesting that participants could not completely rule out the signal of difference created by the spatial cues when no change occurred between the memory and test displays. Thus, they were biased toward a different judgement with two cues in the test display.
As expected, early cues benefited accuracy in change detection. More important, the cues influenced detection equally for both types of changes. If attention is specific for binding in VSTM, while the consolidation of feature information does not require attention, selective attention should benefit con-junction targets more than colour targets. In contrast, the spatial cues appear to function as a priority setting in the process of consolidating information in the VSTM. Gegenfurtner and Sperling (1993) suggested this strategy of ``selective transfer'' with an early cue in partial report. Thus, any changes in the cued items can be better detected.
Late cues did not benefit change detection. The null effect may have resulted from a ``nonselective transfer'' strategy with a late cue when the cue delay was manipulated as a block variable (Gegenfurtner & Sperling, 1993). With this strategy, subjects transfer nonselectively about three to four items in the first 100 ms after the stimulus onset. When the late cues occur, they may point either to
Figure 4. Mean hit rates of change detection as a function of type of change and attentional cueing in Experiment 2.
the transferred or to the nontransferred items. If cues point to the transferred items, late cues may interrupt the ongoing consolidation process. If the late cues point to nontransferred items, VSTM has already been filled with other items and the iconic traces of the cued items are already too weak to gain benefit from cueing. Furthermore, an interval of 100 ms may be insufficient to consolidate memory traces given that the consolidation process for a symbol could be as long as 1 s (Jolicoeur & Dell'Acqua, 1998).
EXPERIMENT 3A
The results from the first two experiments suggest that attention influences the detection of feature changes as much as it affects the detection of conjunction changes. Furthermore, late spatial cues do not benefit detection. The null result of late cues is of importance because, in prior work, a null result produced by a simultaneous cue was interpreted as supporting the hypothesis that the number of decision comparisons did not account for the difference in detection perfor-mance between a whole display and a single probe (Wheeler & Triesman, 2002). A simultaneous cue also failed to benefit detection performance when a test display was shown shortly (16 or 281 ms) after stimulus offset, when the iconic trace ought to be strong (Becker et al., 2000). Because it takes time for a cue to shift attention selectively (Eriksen & Collins, 1969; Eriksen & Lappin, 1967), a simultaneous cue may not completely eliminate competition between inputs for access to VSTM and decision.
For a postcue after the memory display to be effective, it must occur prior to the test display rather than simultaneously. Becker et al. (2000) showed a detection benefit when a spatial cue occurred 66 ms prior to the test display. The results of Experiment 2 of our study, however, showed no effect of spatial cues even when they occurred 100 ms prior to the test display, although an early cue (7700 ms) benefited performance. Methodological differences exist between the Becker et al.'s study and our Experiment 2: Two cues and a longer retention interval were used in Experiment 2, whereas Becker et al. used only one cue with a shorter retention interval of 281 ms.
We attempted to replicate the finding of Experiment 2 with a single spatial cue in Experiment 3A. We postulated that the finding with the 766 ms cue in their study was partly due to the fact that their test display occurred when the iconic trace was strong so that memory consolidation did not play a crucial role. However, when iconic memory decays with a longer retention interval of 900 ms, a late cue at 7100 ms does not allow sufficient time to consolidate memory and should not benefit performance. In addition to the no cue and late cue conditions for detecting conjunction changes, we also manipulated set size and included two feature-change conditions (colour, location) to replicate Wheeler and Treisman's (2002) study. We were interested in investigating the relation of the set-size effect to change detection when the number of noisy percepts increases in decision.
BINDING OR PRIORITIZATION 773
Method
Participants. Eighteen undergraduate students at the National Taiwan University volunteered in this experiment and received a monetary reward of US$5 for their participation. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
Design, stimuli, and procedure. This experiment used a 2 (judgement type: Same/different) 6 2 (set size: 3/6) 6 4 (condition: Colour/location/ conjunctionÐno cue, CNC/conjunctionÐone cue; C1C) factorial design. The same stimuli and procedure used in Experiment 2 were adopted, except that the fixation time was reduced to 1000 ms. Because the position of each stimulus item was randomly selected from eight possible positions, the average interitem spacing was larger when the display set size was three than when set size was six. DMDX (Forster & Forster, 2003) was used in this series of experiments. In the location-change condition, two elements changed to new locations that were not occupied by any stimuli in the memory display (see Figure 5 for an example). In the C1C condition, the cued location was randomly selected from the two changed items (Figure 5). An experimental session consisted of three blocks, with location- and colour-change conditions randomized in the same block. Each block began with 12 practice trials and 96 experimental trials. Block order was counterbalanced by a Latin-square design.
Results and discussion
Table 3 shows the mean performance in each condition. A 2 (judgement type) 6 2 (set size) 6 4 (condition) repeated measures ANOVA was used to analyse accuracy data. Because the three-way interaction was significant, F(3, 51) = 3.97, MSE = 0.009, p < .05, two separate analyses were conducted for same and different judgements.
Same judgement. The main effect of set size was significant, F(1, 17) = 9.27, MSE = 0.005, p < .01, showing a better judgement with a small display set size (size 3: M = 0.94, SD = 0.07; size 6: M = 0.91, SD = 0.08). No other effects were significant.
Different judgement. Figure 6 shows the hit rates of change detection. The main effect of set size was significant, F(1, 17) = 137.51, MSE = 0.008, p < .0001, showing a better judgement with a smaller display set size (M = 0.88, SD = 0.14) than with a large display set size (M = 0.71, SD = 0.23). The main effect of condition was significant, F(3, 51) = 46.19, MSE = 0.016, p < .0001. The Set size 6 Condition interaction was also significant, F(3, 51) = 8.08, MSE = 0.013, p < .0005. The simple main effect of condition was significant for both set sizes: Small, F(3, 102) = 13.16, MSE = 0.014, p < .0001, and large, F(3, 102) = 43.99,
Figure 5. The left side shows an example of a location-change condition, and the right side shows the conjunction-change condition with one cue used in Experiment 3A.
775
MSE = 0.014, p < .0001. Analysis of A' by a 2 (set size) 6 4 (condition) repeated measures ANOVA showed the same pattern.
Both accuracy and sensitivity data showed the same effects of set size across the conditions. The detection of location changes was not influenced by set size, whereas performance in all the other conditions was degraded as the set size
TABLE 3
Mean proportion correct and sensitivity of change detection in Experiment 3A with a cue delay of 800 ms
Test condition Colour Location CNC C1C M SD M SD M SD M SD Same judgement Set-size 3 .97 .04 .92 .11 .95 .06 .94 .05 Set-size 6 .91 .10 .92 .09 .92 .06 .88 .07 Different judgement Set-size 3 .96 .06 .98 .04 .82 .12 .77 .16 Set-size 6 .74 .21 .97 .04 .56 .16 .59 .19 A' Set-size 3 .98 .02 .98 .03 .94 .04 .91 .07 Set-size 6 .88 .14 .97 .03 .84 .07 .83 .08 CNC: Conjunction changes without a cue; C1C: Conjunction changes with a cue.
Figure 6. Mean hit rates of change detection in different conditions of Experiment 3A. CNC is the conjunction-change condition without a cue, and C1C is the conjunction-change condition with one cue.
increased. For a small set size, Tukey post-hoc comparisons from both the accuracy and sensitivity data showed equivalent performance between the location- and colour-change conditions, and both were significantly higher than the CNC and C1C conditions. When the display contained six stimuli, accuracy data showed a significantly better performance in the location-change condition than the colour-change condition, which in turn was better than the other two conditions. Sensitivity data showed a significantly better performance in the location-change condition than in the other three conditions. No difference in sensitivity was found among the colour-change, CNC, and C1C conditions for a large set size. Neither data set showed a significant difference in performance between the CNC and C1C conditions. A late cue (7100 ms) did not benefit the detection of conjunction changes.
The set size in the display has no effect on the detection of location changes because participants could base their judgement on a configurative change (Wheeler & Treisman, 2002). When the display layout altered, the participants could accurately and quickly detect the change. In contrast, increasing the set size significantly degraded performance in detecting both colour and conjunc-tion changes. As predicted, late cueing (±100 ms) had no influence on detecting conjunction changes.
EXPERIMENT 3B
The objective of this experiment was to replicate the effect of early cues. Because participants of Experiment 2 showed worse performance in making a same judgement with two cues than without any cues, the number of cues was manipulated. One or three cues occurred on the screen during the reten-tion interval. The interference in same judgements found in Experiment 2 could be an anomaly that needs to be verified. Another reason motivates the control of the number of cues. One cue is typically used in studies of spatial cueing (e.g., Becker et al., 2000; Palmer, 1988, 1991; Schmidt et al., 2002). Yet VSTM has been confirmed to contain about four items (see Cowan, 2001; but see Landman, Spekreijse, & Lamme, 2003 for a different view). Thus, the ``selective readout'' process (Gegenfurtner & Sperling, 1993; Sper-ling, 1960) should be capable of prioritizing three items to fill the VSTM. That is, three cues should also benefit performance provided there is suffi-cient time for consolidation.
Method
Participants. Eleven undergraduate students at the National Taiwan University volunteered in this experiment and received a monetary reward of US$5 for their participation. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
BINDING OR PRIORITIZATION 777
Design, stimuli, and procedure. This experiment used a 2 (judgement type: Same/different) 6 2 (set size: 3/6) 6 5 (condition: Colour/location/CNC/C1C/ conjunctionÐthree cues; C3C) factorial design. The same stimuli and procedure used in Experiment 3A were adopted except that four blocks were run in random order. In the C3C condition, all the items were cued when set size was three (see Figure 7). When display set size was six, the third cued item was randomly selected from the other four unchanged items. That is, one of the cued items in C3C was unchanged.
Results and discussion
A 2 (judgement type) 6 2 (set size) 6 5 (condition) repeated measures ANOVA analysed accuracy data (see Table 4 for mean performance). Because the three-way interaction among these variables was significant, F(4, 40) = 8.86, MSE = 0.006, p < .0001, two separate analyses were conducted for same and different judgements.
Same judgement. The main effect of set size was marginally significant, F(1, 10) = 3.59, MSE = 0.002, p < .1, showing a trend for better judgement with a smaller display set size (size 3: M = 0.97, SD = 0.05; size 6: M = 0.95, SD = 0.06). No other effects were significant. Showing three cues during the retention interval did not disrupt same judgements in the conjunction-change conditions.
TABLE 4
Mean proportion correct and sensitivity of change detection in Experiment 3B with a cue delay of 200 ms Test condition Colour Location CNC C1C C3C M SD M SD M SD M SD M SD Same judgement Set-size 3 .96 .07 .96 .04 .98 .03 .97 .03 .98 .04 Set-size 6 .95 .06 .96 .04 .95 .07 .94 .06 .94 .06 Different judgement Set-size 3 .98 .04 .95 .17 .87 .07 .96 .06 .90 .09 Set-size 6 .70 .19 .98 .05 .58 .13 .87 .08 .75 .09 A' Set-size 3 .98 .03 .98 .04 .96 .02 .98 .02 .97 .03 Set-size 6 .91 .04 .99 .02 .86 .06 .95 .03 .91 .03 CNC: Conjunction changes without a cue; C1C: Conjunction changes with a cue; C3C: Conjunction changes with three cues.
Figure 7. An example of the three-cue condition in Experiment 3B, with the left side showing a display of three stimuli and the right side showing a display of six elements.
779
Thus, the worse performance caused by the presence of two cues in Experiment 2 was not replicated.
Different judgement. Figure 8 shows the hit rates of change detection. The main effect of set size was significant, F(1, 10) = 55.72, MSE = 0.008, p < .0001, showing a better judgement with a small display set size (M = 0.93, SD = 0.10) than with a large display set size (M = 0.78, SD = 0.18). The main effect of condition was significant, F(4, 40) = 11.13, MSE = 0.016, p < .0001. The Set size 6 Condition interaction was also significant, F(4, 40) = 12.68, MSE = 0.008, p < .0001. The simple main effect of condition was significant only for display set size of six, F(4, 80) = 21.62, MSE = 0.012, p < .0001, and not for the small set size (p > .1). Tukey post-hoc comparisons revealed that the hit rates in the location-change condition were significantly higher than the other conditions except for the C1C condition. Hit rates in the C1C and C3C conditions were significantly higher than in the CNC condition, with nonsignificant difference in performance between C1C and C3C conditions. Analysis of A' by a 2 (set size) 6 5 (condition) repeated measures ANOVA showed the same results.
When the display set size was six, the detection of location changes was superior to performance in the other conditions except for the C1C condition. As confirmed by the Tukey post hoc comparisons, spatial cueing improved different judgements in detecting conjunction changes. One spatial cue ele-vated the detection of conjunction changes to a performance level equivalent to the detection of location changes, and was better than the detection of col-our changes. Without a cue, the detection of conjunction changes in CNC was significantly worse than all the other conditions when the display contained
Figure 8. Mean hit rates of change detection as a function of display set size and condition in Experiment 3B. C3C is the conjunction-change condition with three cues.
six elements. The most interesting result is the cueing benefit in the C3C con-dition, with performance not significantly different from performance in the C1C condition. That is, three items can be sampled in one transfer in the change detection paradigm as found in Sperling's (1960) work on iconic memory.
EXPERIMENT 3C
The objective of this experiment was to investigate whether the cueing benefit mirrors the temporal characteristics of partial-report superiority. Partial-report superiority diminishes when the tone occurs 500±800 ms after stimulus offset. Palmer (1988, 1991) also showed that a postcue with a delay of 500 ms did not reliably enhance shape or length discrimination. The results of Experiment 3A showed that a spatial cue did not benefit the detection of conjunction changes when this post-cue cue was delayed for 800 ms after the memory display. The results of Experiment 3B showed a cueing benefit when one or three cues occurred early at 200 ms after stimulus offset, allowing a consolidation time of 700 ms.
In Experiment 3C, the delay was controlled at 550 ms, allowing 350 ms to select and consolidate memory prior to the occurrence of the test display (7350 ms). Given that change detection does not require detailed analysis in comparing memory and test items, it is of interest to investigate whether a consolidation time of 350 ms is sufficient to benefit performance. We investigated the cuing effect for the detection of conjunction changes.
Method
Participants. Fourteen undergraduate students at the National Taiwan University volunteered in this experiment and received a monetary reward of US$5 for their participation. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
Design, stimuli, and procedure. This experiment adopted a 2 (judgement type: Same/different) 6 2 (set size: 3/6) 6 3 (condition: CNC/C1C/C3C) factorial design. The same stimuli and procedure used in Experiment 3B were adopted except that the ISI was set at 550 ms for cue delay.
Results and discussion
A 2 (judgement type) 6 2 (set size) 6 3 (condition) repeated measures ANOVA was used to analyse accuracy data (see Table 5) and showed a mar-ginally significant three-way interaction, F(2, 26) = 3.32, MSE = 0.007, p < .051. Two separate analyses were then conducted for same and different judgements.
BINDING OR PRIORITIZATION 781
Same judgement. The main effect of set size was significant, F(1, 13) = 6.44, MSE = 0.004, p < .05, showing a better judgement with a small display set size (M = 0.95, SD = 0.05) than with a large display (M = 0.92, SD = 0.08). No other effects were significant.
Different judgement. The main effect of set size was significant, F(1, 13) = 112.21, MSE = 0.01, p < .0001, showing a better judgement with a small display set size (M = 0.91, SD = 0.09) than with a large display set size (M = 0.67, SD = 0.16). The main effect of condition was significant, F(2, 26) = 6.77, MSE = 0.011, p < .005. The Set size 6 Condition interaction was also significant, F(2, 26) = 4.13, MSE = 0.013, p < .05. The simple main effect of condition was not significant for a small set size (p > .6) and was significant for a large set size, F(2, 52) = 10.49, MSE = 0.012, p < .001. As shown in Figure 9, Tukey post hoc comparisons confirmed that performance was significantly (p < .05) better with one cue in C1C than without a cue in CNC. Hit rates in the C3C condition were marginally higher (p < .1) than in the CNC condition, and were marginally lower (p < .1) than the rates in the C1C condition. Analysis of A' in a 2 (set size) 6 3 (condition) repeated measures ANOVA showed the same pattern except that the two-way interaction was marginally significant (p < .1). Cueing had a significant main effect, F(2, 26) = 5.65, MSE = 0.002, p < .01, with higher sensitivity in the C1C condition than in the CNC condition. Sensitivity in the C3C condition was not significantly higher than in the CNC condition.
TABLE 5
Mean proportion correct and sensitivity of change detection in Experiment 3C with a cue delay of 550 ms
Test condition CNC C1C C3C M SD M SD M SD Same judgement Set-size 3 .95 .05 .96 .04 .95 .07 Set-size 6 .92 .10 .92 .06 .91 .07 Different judgement Set-size 3 .89 .09 .90 .08 .92 .09 Set-size 6 .57 .10 .76 .14 .67 .18 A' Set-size 3 .96 .03 .96 .03 .96 .03 Set-size 6 .84 .09 .91 .04 .88 .05 CNC: Conjunction changes without a cue; C1C: Conjunction changes with a cue; C3C: Conjunction changes with three cues.
The participants could have adopted a wait-then-transfer strategy to hold stimuli in iconic memory until one or three cues appeared on the screen. If this strategy had been adopted, we should find little cueing benefit with three cues because iconic information is degraded by 550 ms. Though the benefit produced by three cues in a display of six elements was not significantly better than the no-cue condition, there was a trend for better performance in the hit rates. Furthermore, the hit rates in C1C and C3C were not significantly different. That is, one cue significantly improved performance, but the benefit from three cues was smaller. The smaller cueing benefit was perhaps due to there being insuf-ficient time (350 ms) to consolidate all three cued items.
EXPERIMENT 4A
Wheeler and Treisman (2002) suggested that the difference in detection per-formance with a whole display and a single probe is not in decision processes, because a simultaneous cue with a whole display did not benefit the detection of conjunction changes. In contrast, the results of Experiments 2 and 3 showed that cueing could benefit performance provided that there was sufficient time for consolidation. However, spatial cueing could influence consolidation, decision, or both. In this experiment, we focused only on the decision stage.
The objective was to demonstrate that the number of decision comparisons does matter in detecting conjunction changes. Furthermore, we investigated how the preservation of a display configuration influences change detection. In addition to the whole display and the single-probe conditions, we included a condition with a test display of three probes. When an initial memory display contained three items, a single test probe may disrupt the configuration. If Figure 9. Mean hit rates of change detection as a function of display set size and condition in Experiment 3C.
BINDING OR PRIORITIZATION 783
participants cannot ignore such a change, their same judgements should be degraded with a single probe. In contrast, their different judgements with a single probe should be better since there was only one information source in decision. Following the same logic, same judgements in the three-probe con-ditions should be worse than in a whole display of six elements when the initial memory display contained six items. Accuracy of different judgements, how-ever, should improve with a decreasing number of comparisons from the whole display, to the three-probe condition and to the single-probe test. When the number of comparisons decreases, the S/N in detection increases because the number of noisy percepts in decision decreases.
Method
Participants. Twelve undergraduate students at the National Taiwan University volunteered in this experiment and received a monetary reward of US$5 for their participation. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
Design. This experiment adopted a 2 (judgement type: Same/different) 6 2 (set size: 3/6) 6 3 (test type: Whole display/one probe/three probes) factorial design. The third probe was randomly selected from the other unchanged items when the display set size was six (see Figure 10 for an example). When the display set size was three, the three-probe condition was the same as the whole display condition. A Latin-square design counterbalanced the block order between subjects, while judgement type and set size were randomized across trials in a block. Other aspects were similar to Experiment 3C.
Results and discussion
A 2 (judgement) 6 2 (set size) 6 3 (test type) repeated measures ANOVA was used to analyse accuracy data and showed a significant three-way interaction, F(2, 22) = 15.68, MSE = 0.011, p < .0005, while the main effect of judgement type was not significant (p > .4). Two separate analyses were then conducted for same and different judgements. Table 6 shows the mean performance data.
Same judgement. The main effect of set size was significant, F(1, 11) = 24.07, MSE = 0.017, p < .001, showing a better judgement with a small display set size (M = 0.92, SD = 0.08) than with a large display set size (M = 0.77, SD = 0.18). The main effect of test type was significant, F(2, 22) = 12.55, MSE = 0.014, p < .0005, showing worse performance with one test probe. The Set size 6 Test type interaction was also significant, F(2, 22) = 8.4, MSE = 0.009, p < .005. The simple main effect of test type was significant for both the small display, F(2, 44) = 4.14, MSE = 0.012, p < .05, and the large display, F(2, 44) = 17.67, MSE = 0.012, p < .0001. As shown in Figure 11a and verified by the
Tukey post hoc comparisons, accuracy was the lowest with a single probe, whereas the other two conditions that consisted of three items did not differ for the small display set size. When the initial display contained six elements, accuracy with one test probe or with three test probes was significantly lower than the accuracy in detecting across six stimuli in a whole display.
Different judgement. Both the main effects of set size, F(1, 11) = 35.25, MSE = 0.008, p < .0005, and of test type, F(2, 22) = 39.26, MSE = 0.005, p < .0001, were significant. The Set size 6 Test type interaction was also significant, Figure 10. An example of a test display that contained one probe or three probes in Experiment 4A.
BINDING OR PRIORITIZATION 785
F(2, 22) = 15.74, MSE = 0.006, p < .0005. The simple main effect of test type was marginally significant for a small set size (p < .07) and was significant for a large set size, F(2, 44) = 49.60, MSE = 0.006, p < .0001. As shown in Figure 11b and confirmed by Tukey post hoc comparisons, performance was significantly (p < .05) better with one probe than with three probes. Performance with three probes was significantly better than detecting changes across six elements in a whole display. Because the number of probes had different effects on same and different judgements, analysis of A' showed similar results, but the Set size 6 Test type interaction was marginally significant (p < .1).
Preservation of a display configuration influences change detection. Disruption of the configuration biased subjects toward the choice of a different judgement when there was no signal of change. When there was no change, correct rejection of a single probe was low for both the small and large displays. When the initial memory display contained six items, correct rejection was low when only one item or three elements were shown in the test. Although this bias could have resulted from a lenient response criterion, the leniency was caused by a change in the display configuration when computing the difference between VSTM of the initial display and the test display.
The number of elements in the test display influenced the detection of con-junction changes. When the initial memory display contained six items, change detection significantly improved as the number of probes decreased from six to three and also from three to one. When the initial memory display contained three stimuli, the improvement from three probes to one probe was only marginal,
TABLE 6
Mean proportion correct and sensitivity of detecting conjunction changes in Experiment 4A
Test condition
Whole display One probe Three probes
M SD M SD M SD Same judgement Set-size 3 .95 .06 .85 .07 .96 .03 Set-size 6 .92 .11 .69 .13 .70 .20 Different judgement Set-size 3 .91 .07 .97 .03 .91 .09 Set-size 6 .63 .16 .93 .08 .85 .09 A' Set-size 3 .96 .04 .95 .02 .96 .03 Set-size 6 .87 .05 .89 .04 .85 .07
perhaps because performance had reached a ceiling (M = 0.97, SD = 0.03). Thus, the number of information sources in the decision influenced change detection.
EXPERIMENT 4B
The results of Experiment 4A suggested that preservation of a display con-figuration influenced same judgements and that the number of comparisons in the decision affects the detection of conjunction changes. Thus, we postulated Figure 11. Mean percentage correct of judgements in Experiment 4A, as a function of display set size and test type for (a) same and (b) different judgements.
BINDING OR PRIORITIZATION 787
that S/N may underlie the difference in change detection between a whole display and a single probe. When a whole display of all the stimuli is shown, difference signals are computed from each stimulus item at a local level and also from the overall composition in a feature map. As shown in Figure 12, differ-ence signals exist at the local level for both the colour- and conjunction-change conditions. In the colour-change condition, there are two difference signals at the local level because two new colours replace the old ones. In the conjunction-change condition, there are also two difference signals at the local level because the colours at two locations change.
According to the signal detection approach, performance should be equal between the two conditions whenever a decision is based on local signals. Thus, Wheeler and Treisman (2002) did not find any difference in performance between the feature- and conjunction-change conditions with single test probes. The purpose of this experiment was to verify whether the null difference can still be
Figure 12. A diagram of signal computation in the colour- and conjunction-change conditions, with R: Red; G: Green; B: Blue; Y: Yellow; C: Cyan; M: Magenta; O: Orange; and K: Black. d1 and d2 represent where a difference signal exists at the local level, and 0 indicates no difference exists at a specific location. At the global level, there exist two signals in the colour-change condition, because of the presence of two new colours in the overall colour composition. In contrast, no signals exist at the global in the conjunction-change condition because the overall colour composition is identical to the one stored in VSTM.
observed after a longer interval. In Experiment 4B, we asked the participants to make two serial comparisons with a single probe shown each time. We expected no difference in detecting colour and conjunction changes, for either test probe. An alternative explanation can account for the difference in detection between feature and conjunction changes in a whole test display that contains multiple elements. As in visual search (Treisman & Gelade, 1980), the detection of conjunction changes requires serial comparison, whereas feature changes can be detected in parallel across the elements. Because memory traces decay with time, the serial comparisons involved in detecting conjunction changes are worse than the parallel comparisons used to detect feature changes. If this is the case, we should observe the detection of conjunction changes as accurately as the detection of feature changes for a first test probe, but worse performance for a second test probe.
Because we presented a single probe twice in the test, four conditions resulted: No change in either probe location (NC), change at both probe loca-tions (BC), change in the first probe location (C1), and change in the second probe location (C2). We asked participants to judge whether a change occurred in each probe location.
Method
Participants. Sixteen undergraduate students at the National Taiwan University volunteered in this experiment to receive a bonus credit in an introductory psychology course. Their ages ranged from 18 to 22 years old. All participants had normal or corrected-to-normal vision.
Design and procedure. The experimental design was a 2 (judgement type: Same/different) 6 2 (type of change: Colour/conjunction) 6 4 (condition: NC/ BC/C1/C2) 6 2 (decision order: First/second) factorial design and run by E-Prime. Block order in detecting colour and conjunction changes was counter-balanced between subjects. Each block began with 16 practice trials and then 96 experimental trials. As shown in Figure 13, each trial began with a fixation cross for 1000 ms. A memory display was first shown for 180 ms. After a retention interval of 900 ms, a test probe was shown for 400 ms for the participants to judge whether the probe was same or different from the original item in the memory display. Participants had 1500 ms to respond after the stimulus presentation. The second test probe then was shown for 400 ms for the participants to make a judgement. A grey background was used during each retention interval.
Results and discussion
A 2 (judgement) 6 2 (type of change) 6 4 (condition) 6 2 (decision order) repeated measures ANOVA was used to analyse judgement accuracy (see Table 7), and showed a significant two-way interaction between condition and decision BINDING OR PRIORITIZATION 789
Figure 13. A diagram of the procedure used in Experiment 4B. This example is a conjunction-change condition.
TABL E 7 Mean perfo rmance in Exp erimen t 4B Same trial s Different trials Prop ortion correct Prop ort ion correc t A' C1 -2nd C2 -1st NC-1st NC-2nd BC -1st BC-2 nd C1-1 st C2-2nd BC-1st BC -2nd C1 -1st C2-2nd MS D M SD M SDM SD M SDM SD M SD M SD M SD M SD M SD M SD Colour .58 .16 .61 .12 .60 .13 .59 .13 .85 .09 .74 .19 .91 .08 .79 .15 .81 .08 .74 .13 .85 .07 .76 .14 Conjunc tion .66 .19 .58 .13 .60 .13 .59 .15 .92 .08 .79 .15 .88 .08 .76 .16 .86 .05 .77 .13 .83 .07 .79 .09 BC: A cha nge occ urred in bot h int ervals; C1: A change occur red in the fir st interva l; C2: A ch ange occurre d in the second interva l; NC: No change occurr ed in eit her inter val; 1st :1st interva l; 2nd: 2nd inter val. 791
order, F(3, 45) = 28.87, MSE = 0.02, p < .0001. This interaction occurred because hits (accurate judgements that a change occurred) were higher than correct rejections (accurately judging that no change occurred), with a larger magnitude in the first decision than in the second decision. Separate analyses with a 2 (change type: Conjunction/colour) 6 2 (test type: Change in both probes/change in one probe) 6 2 (decision order) repeated measures ANOVA were then conducted on same and different judgements.
The analysis of same judgements showed a marginal three-way interaction (p < .06). The results of different judgements showed a significant main effect of change type, F(1, 15) = 20.07, MSE = 0.023, p < .0005, with higher accuracy in detecting the first change. The Change type 6 Decision order interaction was also significant, F(1, 15) = 6.38, MSE = 0.009, p < .05, resulting from a lower accuracy in detecting a colour change in the first decision when changes occurred in both probes. Because a test with a single probe could bias partici-pants toward a different judgement, and the effects of the independent variables on same judgements were not of concern, we considered the sensitivity measure a better indicator for this experiment. The analysis using A' (see Figure 14) showed a significant main effect of decision order, F(1, 15) = 17.19, MSE = 0.01, p < .001. Sensitivity in detecting a change in the first test probe (M = 0.84, SD = 0.07) was significantly higher than that in detecting a change in the second probe (M = 0.77, SD = 0.13). No other effects were significant.
Detection sensitivity significantly deteriorated when participants judged the second probe, presumably due to there being weaker memory traces with a
Figure 14. Sensitivity measure A' in different conditions of Experiment 4B. BC-1st: The first decision when changes occurred in both test probes; BC-2nd: The second decision when changes occurred in both probes; C1: The first decision when a change occurred only in the first probe; C2: The second decision when a change occurred only in the second probe.
longer retention interval. Decision sensitivity was not significantly different for detecting colour and conjunction changes, as expected. Thus, the results do not support the hypothesis that feature detection can be executed in parallel, whereas the detection of conjunction changes must be executed serially to ensure whether each test item shows the correct binding in memory.
GENERAL DISCUSSION
The results of these experiments show the following patterns. Preservation of the configuration of the initial display influences change detection. When the spatial configuration of items was changed, participants accurately detected the changes regardless of display set size (Experiments 3A and 3B). When a single test probe was used, which tended to disrupt the original configuration, decisions were biased toward a different response, such that correct rejection was quite low on trials when no change occurred (Experiments 4A and 4B).
When a multiple element set was presented for the comparison test display (Experiments 1, 2, 3A, and 3B), a significant difference in detection accuracy occurred between colour- and conjunction-change conditions, replicating Wheeler and Treisman's (2002) results. The difference caused by the multiple-element comparison display cannot be attributed to serial search for detecting conjunction changes, because judging a single probe after a long delay of 2.8 s did not show a disadvantage for conjunction changes in comparison to colour changes (Experiment 4B). The disadvantage also cannot be caused primarily by attentional reallocation, because attentional shifts generated by having partici-pants attend to a peripheral stimulus during retention affected feature and conjunction changes equally (Experiment 1).
Spatial cueing also had similar effects on detecting colour and conjunction changes (Experiment 2). With conjunction changes tested, the cueing benefit was reliable with one cue (Experiment 3B), two cues (Experiment 2), and three cues (Experiment 3B) at 200 ms after the termination of the initial memory display, when selection mechanisms could prioritize the transfer of iconic information into VSTM early in the process. When the cue delay was 550 ms (Experiment 3C), one cue led to a significantly better performance, and there was a trend of benefit with three cues in hit rates. No benefit was observed at a delay of 800 ms with either one cue (Experiment 3A) or two cues (Experiment 2).
The results suggest two factors have a role in change detection: Signal strength, and the number of information sources in decision. These two factors affect the signal-to-noise ratio in decision. Signal strength influences change detection. Sensitivity was lower in change detection when detection was made on a second test probe of Experiment 4B, as memory traces were weaker with increasing retention intervals. With an early cue, consolidation can strengthen memory traces so that change detection is enhanced (Experiments 2, 3B, 3C). In BINDING OR PRIORITIZATION 793
contrast, a late cue does not provide sufficient time to consolidate the memory traces such that no benefit can be observed (Experiments 2 and 3A).
The number of information sources in the decision also affects change detection. The detection of conjunction changes with one item was superior to performance with three elements, which in turn was better than change detection across six elements (Experiment 4A). Increasing the display set size from three to six decreased detection accuracy for both colour and conjunction changes (Experiment 3). When one cue designated the location where a change could occur (C1C condition), competition between the items could not be eliminated, as there remained an advantage for small over large display sizes (Experiments 3B and 3C). Eriksen and Collins (1969) also showed that neither a precue nor a postcue completely eliminated the influence of extraneous inputs on target report in multiple element displays.
Binding hypothesis
According to this hypothesis, attention binds features in VSTM. Wheeler and Treisman (2002) proposed that VSTM is feature based and that attention binds features in VSTM. They suggested that a whole display of multiple elements required reallocation of attentional resources across the test items, and the bound representations in VSTM were disrupted. Thus, the detection of conjunction changes degraded. Rensink (2000) in his coherence theory also argued that attention is necessary to bind features into one coherent object representation and to maintain this representation in VSTM. Moreover, this object repre-sentation disintegrates to ``unglued'' preattentive features when attention is withdrawn. In the object file theory of transsaccadic memory, Irwin (1992) postulated the same role of visual attention for binding features into an object file (Kahneman & Treisman, 1984) and for maintaining this object file repre-sentation in VSTM until being replaced by new object files.
If the role of selective attention is specific for feature binding in VSTM, memory of features should not be affected by any change in selective attention. However, we showed that the detection of feature changes is influenced by the manipulation of selective attention. Attentional shifts deteriorate detection; spatial cueing enhances detection. That is, visual attention also influences the maintenance of feature representations in VSTM. The binding hypothesis must consider the influence of attention on feature detection.
To account for the difference between detecting feature and conjunction changes, this hypothesis can incorporate the visual search aspect of change detection.1 When a test display contains multiple elements, attention must be
distributed across the test items for a change to be detected. Due to attention being spread across the items, it is difficult to bind features into conjunctions
1We thank Dr Humphreys for suggesting this modification.
