In Chapter 5, we showed that the matching stage is color-selective, as our chromatic symmetry detection model shown. The symmetry encoder is easier to pair the features of the same color (e.g., red) than those of different colors (e.g., red-green). In this chapter, we further examined the color selectivity of the next stage, pooling.
Similar to the matching stage, we considered three possible ways the symmetry encoder pools the matched pairs to determine the axis of a symmetric pattern. The first one, the symmetry encoder only counts the pairs of the same color for the computation of the symmetry axis (Figure 6.1
a). The pairs whose midpoints form an
axis are regarded as signal while others are noise. In this case, the color selectivity is preserved at the pooling stage. The second one, the symmetry encoder counts the pairs as signal as long as their colors are from the same color opponent channels (Figure 6.1b). For instance, a symmetry encoder may regard both the red-red and the
green-green pairs as signal but not the red-red and the blue-blue pairs. The third one, the symmetry encoder is not color-selective. All the pairs are taken into account regardless of their chromaticity (Figure 6.1c). These three pooling mechanisms
represent different color selectivity of symmetry channels. The first two suggest that there are more than one color selective channels in symmetry processing while the third one suggests that there is only one color-blind symmetry channel.46
To the best of our knowledge, there is no published research systematically examines the color selectivity of the pooling stage in symmetry processing. In this chapter, we investigated the color selectivity of the pooling stage of the symmetry encoder, that is, which color pairs can be pooled to determine the symmetry axis.
Here we used noise masking paradigm to probe this issue. We measured the target density threshold with the presence of noise masks of various colors. If the response of the underlying mechanism is influenced by the noise mask, the mask would interfere with the target mechanism and thus produce a masking effect, measured as threshold increment, on target detection. Otherwise, the presence of the mask would have no effect on target detection (Giulianini & Eskew, 1998; Gegenfurtner & Kiper, 1992; Hansen & Gegenfurther, 2006). Hence, if the symmetry encoder pools only the pairs of the same color, only the mask of the same color would produce a masking
b.
a.
c.
Figure 6.1. Three possible ways the symmetry encoder acts in the pooling stage. (a) The symmetry encoder only counts the pairs of the same color for the computation of the symmetry axis. (b) The symmetry encoder counts the pairs as signal as long as their colors are from the same color opponent channels. (c) The symmetry encoder is not color-selective. All the pairs are taken into account to determine the symmetry axis regardless of their colors.
47
effect. If the symmetry encoder can pool the pairs of the opponent colors, a symmetry target of a color would be masked by masks both of its own color and its opponent color. If the pooling stage is not color-selective, all the noise masks would show masking effect.
This chapter examines the color selectivity of the pooling stage of symmetry processing. Since Chapter 5 showed the difference in pairing image features with different chromaticity and luminance, we also examined the color-selective properties of the pooling stage in both chromaticity and luminance dimensions. We measured the target detection threshold of the red, blue and white target superimposed on the noise mask of different colors to achieve this goal.
6.1. Method
Participant. Three observers attended this experiment: CCW, CPY and HYC.
HYC was naïve to the purpose of the experiment.
Stimuli. In each trial, the stimuli consisted of a vertical symmetric-dot target or a
random-dot noise control superimposed on a random-dot noise mask. The color of the target and the noise control was red and blue in the isoluminance conditions while was white in the luminance conditions (see Table 4.1 in Chapter 4 for the definition of the color). The colors of the noise mask were 0°, 22.5°, 45°, 67.5°, 90°, and 180°
deviated from the target along the isoluminant plane in the isoluminance conditions and along the plane spanned by the red-green cardinal axis and the luminance axis in the luminance condition. The density of the noise mask was fixed at 0.01. To prevent observers from using only the information near axis to make a judgment, no dot was in the region within 1.4° visual angle width. Figure 6.2 shows the examples of our stimuli.
48
Procedure. The 2AFC procedure was the same as the experiment represented in
Chapter 5. The observers’ task was to judge which stimulus interval contained the target. They were informed that the orientation of axis was vertical. The two chromaticity conditions and one luminance condition were run separately. The order of noise color in both conditions was randomized.
6.2. Results
Our results showed a clear color-selective property of the pooling stage. The highest target density threshold was measured when the noise mask and the target were of the same color in both isoluminance and luminance conditions. Figure 6.3 shows the target density threshold in the isoluminance conditions. The red and blue circles represent the density threshold for the red and blue target respectively in the presence of the various noise masks. The pink and cyan open triangles represent the
Figure 6.2. The example of the stimuli: (a) the red target superimposed on the noise mask of its own color (0° deviation) and (b) the red target superimposed on the noise mask of green color (180° deviation). The mask density was 1%.
49
red and blue target density threshold respectively when there is no mask. Each panel represents the data from one observer.
Target density threshold (log)
Figure 6.3. The results of isoluminance conditions. Each panel represents data from one observer. The red and blue symbols denote the target density thresholds for red and blue target superimposed on the noise mask of various colors respectively. The pink and cyan symbols denote the red and blue target density threshold when there was no mask, serving as a baseline.
50
For all observers, the mask of the same color (0° deviation from the target in the isoluminance plane) increased the target detection threshold 1 to 1.37 log unit, which was 10 to 23 fold change. The masking effect decreased as the difference between the target and the mask color increased. The mask of orthogonal color (90° deviation from the target in the isoluminance plane) and opponent color (180° deviation) showed little, if any, masking effect. They increased the target detection threshold about 0 to 0.3 log unit, equivalent to 1 to 2 fold change. This effect was small when compared with the effect of the same color. Hence, our results favored the first hypothesis. That is, the symmetry mechanism pools the pairs of the same color to determine the axis. The visual system has a band of independent color-selective symmetry channels at each orientation axis.
Our data also showed that the color tuning of each color-selective symmetry encoder was not the same. The color tuning of the red symmetry encoder was broader than that of the blue one. The red symmetry encoder responded to a range of colors with different sensitivities (22.5° to 67.5° deviation) while the blue symmetry encoder only responded to a narrower range of colors. The mask of the colors deviated more than 22.5°from the target color produced no masking effect to the target detection.
This might be due to the difference in sensitivity between the red-green and the blue-yellow channels
The results of the luminance conditions were similar to those of the isoluminance conditions. Figure 6.4 shows the data for the luminance conditions. The black and gray symbols represent the target density thresholds in the presence and absence of noise mask respectively. Each panel represents the data from one observer.
51
Similar to the isoluminance conditions, the masking effect was largest when the color of the mask was the same as the target. For all observers, the mask of the same
Target density threshold (log)
Figure 6.4. The results of luminance conditions. Each panel represents data from one observer. The black symbols denote the target density thresholds for white target superimposed on the noise mask of various colors. The gray symbol denotes the target density threshold when where was no mask, serving as a baseline.
52
luminance increased the target detection threshold 0.97 to 1.2 log unit (equivalent to 9 to 16 fold change). The masking effect decreased as the difference between the target and the mask luminance increased. The mask of isoluminance color (90° deviation from the target on the plane spanned by the red-green cardinal axis and the luminance axis) and opposite luminance polarity (180° deviation) showed little, if any, masking effect. They increased the target detection threshold about 0.03 to 0.24 log unit (1.08 to 1.75 fold change), much smaller than that produced by the mask of the same luminance. Our results also showed that the white symmetry encoder only responded to a narrower range of colors, narrower than that of the red one. The mask of larger than 45° deviation from the target color produced no masking effect to the target detection.
6.3. Discussion
In this experiment, we used the noise masking paradigm to investigate the color-selective property of the pooling stage of symmetry processing. Our results showed that the noise mask of the same color with the target produced the largest masking effect on the target detection. The masking effect decreased as the difference in the color between the target and the mask increased. This suggests that the color-selective property is preserved in the pooling stage. The symmetry encoder counts only the pairs of the same color for the computation of the symmetry axis. Hence, the presence of dots in a different color would not interfere with symmetry detection. Hence, there are a band of color-selective symmetry encoders at each orientation.
Notice that, our result shows that the symmetry processing is color selective at the encoding stage. It is by no means implying that the symmetry channel is color selective. It is possible that a later mechanism integrates information across different
53
color selective encoders. However, if this is the case, the integration should occur after a nonlinear operation has been applied to the color specific symmetry channels.
Otherwise, such mechanism cannot distinguish information from different colors and thus inherently color blind. Our result does not support such color blind mechanism.
Some might argue that the masking effect we observed may be unrelated to the color-selective pooling stage but due to the color selective matching stage. The noise mask of the same color as the target can be paired with the dots in the target while that of the different colors cannot. Hence, the dots in the noise mask interfered with the symmetric target detection by matching rather than pooling. However, if this is the case, we should expect that the noise mask that has a different color from the target should also show a masking effect, because the symmetry encoders would take all the pairs in the noise mask into consideration to determine the axis of a symmetric pattern.
Our results however did not support this claim. Except for the luminance condition of the observer CPY (t(1) = 37.75, p = .008), the mask of the orthogonal color and opponent color showed no masking effect (all t(1) < 5.24, p > .06).
Another argument is that the masking effect is unrelated to the color-selective pooling stage but due to selective attention. For example, the observers may just pay their attention to the target color and ignore whatever other colors in the display.
Hence, the dots of the other colors are not encoded in the matching stage and are left out in the pooling stage. Hence, the pooling stage is not necessarily color-selective as we claim but color-blind. To exclude this possibility, we carried out a control experiment. We excluded the effect of selective attention to reexamine the color-selectivity of the pooling stage. In this control experiment, we presented two symmetric patterns of different axes superimposed on each other in one interval and a random dot pattern in another interval. The colors of the two symmetric patterns were
54
the same (i.e., both red or both green, Image A in Figure 6.5a) or different (i.e., one red and one green, Image B in Figure 6.5b). The random dot pattern contained the same number of red and/or green dots as the interval containing the targets. The observer’s task was to detect which interval contained any symmetric pattern. In this experiment, the observers needed to pay attention to all colors in the display to get max information.
In this case, the color-selective and color-blind pooling stage hypotheses make different predictions about the symmetry detection performance in these two images.
The color-blind hypothesis predicts the same performance in the two images while color-selective one does not. If the pooling stage of symmetry encoder is color-blind, there are only two orientation-selective symmetry encoders, one sensitive to the
left-Figure 6.5. The target stimuli of control experiment. Each target image was composed of two symmetric patterns of same or different colors. (a) Image A was a red right-diagonal symmetric pattern superimposed on a red left-right-diagonal symmetric pattern. (b) Image B was a red left-diagonal symmetric pattern superimposed on a green right-diagonal symmetric pattern.
a. Image A b. Image B
55
diagonal and another one, the right-diagonal. For these two symmetry encoders, half of the dots in Image A are signal and another half are noise. So is Image B. Hence, the color-blind hypothesis predicts no difference in symmetry detection performance between these two images. However, if the pooling stage of symmetry encoder is color-selective, there are four symmetry encoders involved in this task: red left-diagonal, red right-left-diagonal, green left-left-diagonal, and green right-diagonal ones. The presence of Image A would produce an excitation in the red left-diagonal and red right-diagonal symmetry encoders. In Image A, half of red dots are symmetric about the left diagonal axis while the other half, the right diagonal axis. Thus, for the red left-diagonal symmetry encoder, the red dots that are symmetric about the left diagonal axis are considered as signal while those dots that are symmetric about the right diagonal have no structure to the left diagonal axis and thus are considered as noise. Hence, for the red left-diagonal symmetry encoder, half of the dots in the Image A are signal while another half dots are noise. The same consideration also applies to the red right-diagonal symmetry encoder. On the other hand, the presence of Image B would produce an excitation in the red left-diagonal and green right-diagonal symmetry encoders. For the red left-right-diagonal symmetry encoder, the red dots that are symmetric about the left diagonal axis are considered as signal. The green dots that are symmetric about the right diagonal have no structure to the left diagonal axis. However, since the symmetry encoder is color-selective, these green dots would not be considered as noise by the red left-diagonal symmetry encoder. Hence, for the red left-diagonal symmetry encoder, half of the dots in the Image B are signal while the other half is ignored. The same argument also applies to the green right-diagonal symmetry encoder. On the other hand, for the green left-diagonal symmetry encoder, there is no signal dots in the image in which the green dots that are symmetric about
56
the right diagonal axis but are structureless for the left diagonal axis. Hence, for this encoder, half of the dots in the pattern are noise while none is signal. Similar argument can be applied to the red right-diagonal symmetry encoder. Hence, the signal to noise ratios for these two images differ in these color-selective symmetry encoders and in turn the symmetry detection performance. Our results showed such difference in detection performance. The averaged data from three observers showed that the target density threshold for the Image A (M = -1.36, SE = 0.12) is much higher than that for the Image B (M = -2.42, SE = 0.22), t(4) = 4.25, p = .007. This supports the color-selective hypothesis. There should be a band of color-selective symmetry encoders in our visual system. This implies the existence of a band of color-selective symmetry channels.
The next step is to characterize how our visual system integrates these color-orientation selective symmetry channels to perceive chromatic symmetry. Recall that our visual system needs to integrate the responses of orientation-selective symmetry channels to detect symmetry when there is only one color in the images (Chen &
Tyler, 2010). The more orientation-selective symmetry channels the visual system monitors, the worse the symmetry detection performance. If the images contain more than one color, our visual system needs to integrate a band of color-orientation selective symmetry channels to perceive chromatic symmetry. Does the integration of the responses of color-selective symmetry channels differ from that of orientation-selective ones? If yes, what is the difference? If no, whether the increment of the number of the colors in an image increases the uncertainty of symmetry detection, leading to worse performance, just like the uncertainty effect of the orientation axis?
Or color facilitates the symmetry detection by other mechanisms? In Chapter 7 to 9, we investigate this issue by the noise masking paradigm mentioned above.
57