General description of the brainwaves. Figure 4 depicts the brain response to standards, deviants and the target, which shows that the ERP responses elicited by targets are much stronger than those by standards and deviants.
29
Figure 4.ERP responses elicited by targets, standards, and deviants across deviant types, dimensions, and language groups.
30
Grand averaged ERPs to standards, within- and between-dimension deviants are plotted in Figures 5 and 6 for Chinese and English speakers,
respectively.
Figure 5. ERP responses to standards, within- and between-dimension deviants in the Chinese group.
31
Figure 6.ERP responses to standards, within- and between-dimension deviants in the English group.
32
As Figure 5 shows, in the Chinese group, the ERP response to standards is similar to that to between-dimension deviants in early latency, while the response to within-dimension deviants diverges from the other two stimulus types, especially between 100-250 ms. In contrast, Figure 6 shows that the ERP responses to all stimulus types in the English group generally merge together between 100-200 ms. It is not until 200 ms that the response to between-dimension deviants starts to diverge from the other two stimulus types.
A quick glance at Figures 5 and 6 reveals that ERP responses collected from the occipital sites (O1, OZ, O2) are different in polarity between 100-200 ms: the brainwaves generally went in the opposite direction with negative-going brainwaves at the anterior sites but positive-going one at the occipital sites. Occipital electrodes were thus excluded from further analysis to resolve the statistical problem that the amplitudes would otherwise cancel each other out because of the inverse relation between ERP components within the same time window.
Figures 7 and 8 show the grand averaged ERPs to standards and deviants in both language groups, with Figures 7 and 8 centering on within-dimension and between-dimension deviants respectively.
33
Figure 7. Grand averaged ERP responses to standards and within-dimension deviants. CS: Chinese Speakers; ES: English Speakers.
P2 effect
34
Figure 8.Grand averaged ERP responses to standards and between-dimension deviants. CS: Chinese Speakers; ES: English Speakers.
35
Figures 7 and 8 above show that the ERP components are more fronto-centrally distributed with relatively small amplitudes observed over the posterior sites. An early component N1 (peaking around 150 ms) can be detected in all conditions, but it seems much smaller for the within-dimension deviants in the Chinese group. The N1 then shifts to a positive-going wave, a P2 which peaks between 200 and 250 ms. This P2 component in the Chinese group is notably greater in amplitude for within-dimension deviants compared with the corresponding standards. Following the P2, a negativity is found with an amplitude generally larger in the English group than in the Chinese group, for both standards and deviants.
All the brainwaves then start to converge between 300 and 400 ms except for the between-dimension deviant in the Chinese group, which slightly diverges to another positive-going wave until 400 ms.
Figures 9 and 10 illustrate the difference waves and the topographic maps of the P2 deviancy effects (deviants – standards) between 150-250 ms, where a sharp distinction between language groups and a more frontally oriented distribution can be visually identified.
36
Figure 9. Difference waves of within-/between-dimension deviants minus standards. CS: Chinese Speakers; ES: English Speakers.
P2 effect
37
As shown in Figure 9, the within-dimension deviancy effect in the Chinese group (black line) is conspicuous in the P2 time window while the between-dimension deviancy brainwaves of Chinese and English groups largely merge together (red line vs. green line), except for the more positive-going brainwave in the Chinese group from 300 to 400 ms. Figure 10 presents the topographic maps, showing that the P2 effect is frontally distributed, and that it is salient in the within-dimension condition in the Chinese group, but not in the between-dimension condition or in the English group.
μV
CS
ES
Figure 10. Topographic maps of the deviancy effects (deviant minus standard) (P2 effect: 150-250 ms). Top: Chinese Speakers (CS). Bottom: English Speakers (ES). Rightmost: channel locations of 32 electrodes. Positivity is painted in red and negativity in blue.
38
The target effect. Mean amplitudes between 150-250 ms of the targets were measured
and submitted to two repeated measures ANOVAs for the midline and the laterality analyses, respectively. The midline analysis was carried out with two within-subject factors of Stimulus Type (standard, target) and Anteriority (front, middle, back) and one between-subject factor of Language (Chinese, English); the laterality analysis was carried out with an additional within-subject factor of Laterality (left, right). Both analyses revealed a main effect of Stimulus Type (F(1,27) = 50.41, p < .0001, ηp2 = .66), showing that the P2 effect was stronger for targets than for standards (D(target-standard) = 2.86, p < .0001)2. However, Stimulus Type did not interact with Language (F(1,27) = .05, p = .82, ηp2 = .002), suggesting that there was no difference between language groups with respect to processing targets (cat) among standards.
The N1 effect. Mean amplitudes between 100-150 ms of the non-target stimuli were
measured and submitted to two repeated measures ANOVAs for the midline and the laterality analysis, respectively. The midline analysis was carried out with three within-subject factors of Stimulus Type (standard, within-dimension deviant, between-dimension deviant) and Anteriority (front, middle, back) and one between-subject factor of Language (Chinese, English); the laterality analysis was carried out with an additional within-subject factor of Laterality (left, right). Neither of the analyses revealed a main effect of Language or an
2 D stands for the microvolt mean differences.
39
interaction with Language, showing that the classifier effect in the Chinese group was not yet significant during the selected N1 window.
The P2 effect. Mean amplitudes between 150-250 ms of the non-target stimuli were
measured and submitted to two repeated measures ANOVAs for the midline analysis and the laterality analysis, respectively. The midline analysis was carried out with three within-subject factors of Stimulus Type (standard, within-dimension deviant, between-dimension deviant), Dimension (one-dimensional, two-dimensional), and Anteriority (front, middle, back) and one between-subject factor of Language (Chinese, English). The results revealed a significant main effect of Stimulus Type (F(2,27) = 23.81, p < .0001, ηp2 = .478), and such effect further interacted with Language (F(2,27) = 5.86, p < .01, ηp2 = .184), showing that the P2 effect was stronger for the within-dimension deviancy, but not for the between-dimension one, in the Chinese than in the English participants (CS: D(within-standard) = 1.57, p < .0001; D(between-standard)
= -.27, p = .72; ES: D(within-standard) = .37, p = .48; D(between-standard) = -.37, p = .35). A significant Anteriority effect was also found (F(2,27) = 28.3, p < .0001, ηp2 = .521). Its interaction with Stimulus Type indicated that the within-dimension deviancy effect was more widespread (but maximal in the fronto-central region, reflected by the mean differences) (F(4,27) = 7.35, p
< .005, ηp2 = .22; front: D(within-standard) = 1.12, p < .0001; middle: D(within-standard) = 1.18, p < .0001;
back: D(within-standard) = .62, p < .01). In addition, its interaction with Dimension showed that the difference between 1D and 2D stimuli was significant in the parietal region (F(2,27) = 7.85, p
40
< .01, ηp2 = .232; front: D(1D-2D) = .1, p = .64; middle: D(1D-2D) = -.08, p < .67; back: D(1D-2D) = -.44, p < .05). No other main effects and interactions were found (Language: F(1,27) = .56, p
= .46, ηp2 = .021; Dimension: F(1,27) = .58, p = .45, ηp2 = .022). See Table 4 for a summary of the ANOVA for the midline analysis.
Table 4. Summary of the degrees of freedom, F values, and p values of repeated measures ANOVAs for the midline and laterality analyses of the P2 effect (150-250 ms)
dfs F p
Language × Stimulus Type × Dimension × Anteriority 4 2.01 0.14 LATERALITY ANALYSIS
41
Stimulus Type × Anteriority 4 27.09 <.0001
Stimulus Type × Laterality 2 0.09 0.80
Language × Stimulus Type × Dimension × Anteriority 4 0.78 0.47 Language × Stimulus Type × Dimension × Laterality 4 0.12 0.82 5-way interaction
Language × Stimulus Type × Dimension × Anteriority ×
Laterality 4 5.60 <.005
Following the midline analysis, the laterality analysis was carried out with an additional within-subject factor of Laterality (left, right). As also shown in Table 3, the analysis was beset with a five-way interaction of Language, Stimulus Type, Dimension, Anteriority, and Laterality (F(4,27) = 5.6, p < .005, ηp2 = .18). Due to this significant five-way interaction, the data were then split by Laterality (right, left) to conduct two four-way repeated measures ANOVAs with three within-subjects factors of Stimulus Type, Dimension, Anteriority, and one between- subject factor of Language.
Similar to the midline analysis, the left analysis showed a significant Stimulus Type effect (F(2,27) = 12.45, p < .0001, ηp2 = .324) and its interaction with Language (F(2,27) = 4.31, p
42
< .05, ηp2 = .142), with the P2 effect stronger for the within-dimension deviancy in the Chinese group (CS: D(within-standard) = 1, p < .0001; D(between-standard) = -.12, p = 1; ES: D(within-standard) = .18, p = 1; D(between-standard) = -.18, p = 1). The main effect of Anteriority was found (F(2,27) = 56.03, p < .0001, ηp2 = .683), and such effect also interacted with Stimulus Type (F(4,27) = 28.11, p
< .0001, ηp2 = .519) and Dimension (F(2,27) = 15.57, p < .0005, ηp2 = .375), respectively. For the interaction of Anteriority and Stimulus Type, follow-up t-tests suggested a fronto-centrally distributed within-dimension deviancy effect (front: D(within-standard) = .98, p < .0001; middle:
D(within-standard) = .78, p < .0001; back: D(within-standard) = .02, p = 1). For the interaction of Anteriority and Dimension, follow-up t-tests suggested a significant difference between 1D and 2D stimuli at the posterior site (front: D(1D-2D) = .09, p = .67; middle: D(1D-2D) = -.16, p = .37;
back: D(1D-2D) = -.67, p < .005). No other main effects and interactions were found (Language:
F(1,27) = .48, p = .5, ηp2 = .018; Dimension: F(1,27) = 2.03, p = .17, ηp2 = .072). See Table 5 for a summary of the ANOVA for the left analysis.
Table 5. Summary of the degrees of freedom, F values, and p values of repeated measures ANOVAs for the left and right analyses of the P2 effect (150-250 ms)
left right
43 Stimulus Type × Anteriority 4 28.11 <.0001 13.98 <.0001 Dimension × Anteriority 2 15.57 <.0005 14.56 <.0005 3-way interaction
Unlike the left analysis, the right analysis revealed a four-way interaction of Language, Stimulus Type, Dimension, Anteriority (F(4,27) = 3.29, p < .05, ηp2 = .112). The data were then split by Language (right CS, right ES) to conduct two three-way repeated measures ANOVAs with only within-subject factors of Stimulus Type, Dimension, and Anteriority.
In the right CS analysis, but not in the right ES analysis, two main effects were found:
Stimulus Type (CS: F(2,27) = 10.23, p < .005, ηp2 = .44; ES: F(2,27) = .34, p = .57, ηp2 = .026) and Dimension (CS: F(1,27) = 8.76, p < .01, ηp2 = .403; ES: F(1,27) = .52, p = .53, ηp2 = .038).
The Stimulus Type effect in the Chinese group further interacted with Anteriority (F(4,27) = 12.05, p < .0005, ηp2 = .481), suggesting an anterior P2 effect for the within-dimension deviancy (front: D(within-standard) = 1.51, p < .0001, D(between-standard) = -.3, p = .91; middle: D
(within-standard) = 1.42, p < .0005; D(between-standard) = -.004, p = 1; back: D(within-standard) = .62, p = .22;
44
D(between-standard) = .33, p = .7). It is noted that the Chinese within-dimension deviancy effect also marginally interacted with Dimension (F(2,27) = 3.11, p = .07, ηp2 = .193), which showed that within-dimension deviancy effect was larger in 2D and which might indicate the sensitivity to dimension in the right hemisphere (1D: D(within-standard) = .60, p = .26; 2D: D(within-standard) = 1.76, p < .0005). Both CS and ES analyses revealed an Anteriority main effect and the interaction of
Anteriority and Dimension; however, follow-up t-tests showed that significant 1D vs. 2D difference in the centro-parietal region was only found in the Chinese group (front: D(1D-2D) = -.23, p = .26; middle: D(1D-2D) = -.47, p < .05; back: D(1D-2D) = -.98, p < .005), not in the English group (front: D(1D-2D) = .14, p = .67; middle: D(1D-2D) = -.05, p = .85; back: D(1D-2D) = -.56, p
= .08). No three-way interaction was found in both language groups. See Table 6 for a summary of the ANOVAs for the right CS and right ES analyses.
Table 6. Summary of the degrees of freedom, F values, and p values of repeated measures ANOVAs for the right CS and right ES analyses of the P2 effect (150-250 ms)
right CS right ES
45
To check if the non-significant results were due to a lack of statistical power, we conducted a post hoc power analyses using G* Power (Faul et al. 2007; Faul et al. 2009), with power (1
− β) = .8 and α = .05. Based on the partial eta-square effect size (ηp2 = .16) from a critical comparison of Boutonnet et al. (2013), where a significant interaction was found for Deviancy and Language Group (F(1, 25) = 4.9, p < .05, ηp2 = 0.16), the analysis suggested a sample size of 10 for each group. Therefore, the non-significant results could not be attributed to the lack of statistical power.
In sum, both the midline and laterality analyses revealed that the classifier effect was indexed by a prominent fronto-central P2 effect during 150-250 ms, which was associated with the within-dimension deviancy in the Chinese group, but not in the English one. However, the P2 effect was absent in the between-dimension deviancy in both language groups.
46
4 Discussion
This study explored the influence of language on cognition by examining whether the linguistic categorization—the classifier system—affects our perception of the world. We manipulated objects categorized by different (types of) classifiers in Mandarin Chinese, and compared perceptual responses to the selected objects between two language groups, native Chinese speakers and native English speakers, using an oddball paradigm. The ERP results revealed a critical difference between language groups when participants perceived the within-dimension deviant against standards: the P2 effect was significantly more potent in the Chinese than in the English group. However, the between-dimension deviants did not show such a perceptual discrepancy between language groups. The following paragraphs aim to establish a connection between the P2 effect and the classifier/language effect, to compare the current findings with previous behavioral studies on ‘classifier relativity,’ and to discuss how the specific classifier effect may enlighten us.
We believe that the observed P2 effect can be associated with high-level perceptual processing within a cognitive matching system that compares sensory inputs with stored memory (Luck & Hillyard 1994, among others). Despite the implicit difference in the use of classifiers (versus the consistent surface difference in shape, for example), Chinese speakers, as habitual users of the language-specific classifier system, seemed to perceive the
within-47
dimension deviant to be physically different from standards, and therefore were able to detect and respond to this extreme subtlety by 200 ms.
Importantly, we failed to observe a P2 effect in response to between-dimension deviancy (between deviants – standards) in the Chinese group. We believe that the absence of the effect was not due to our sample size because the post hoc power analysis excluded such a possibility (see Section 3.2 ERP data). Instead, we believe that its absence might have to do with the manipulation of the materials. As mentioned in Section 2.2 (Materials), between-dimension deviants were always preceded by 1 or 2 blocks of stimuli where they were treated as standards.
Therefore, even though they were deviants in the “new” blocks, the evoked classifier effect might still remain and thus greatly reduced the deviancy effect in the Chinese group, if any, because such deviants were still treated as “standards” by the subjects. Consequently, the responses to between-dimension deviancy were similar between the Chinese and English group, as if classifiers did not play a role in such processing.
We did not find the vMMN (two shades of blue, Thierry et al. 2009) and the DRN/posterior N1 (cup vs. mug, Boutonnet et al. 2013) in our study and we think this is due to the following reasons. First, the vMMN is a component indexing a non-attentional, low-level perceptual process (Clifford et al. 2010; Czigler 2007). However, participants in the current study were instructed to fixate on the center of the screen and to pay attention to the multidimensionally different real world objects, not two objects or two shades of colors as in
48
previous research; therefore, the vMMN deviancy effect was not observed (see Boutonnet et al. (2013:1707) for a similar argument). Second, the visual N1 has been found to manifest a discriminative process within the focus of attention, but it is absent in a detection task (e.g., Vogel & Luck 2000). Although the participants in Boutonnet et al.’s (2013) study were not asked to discriminate explicitly between cups and mugs, they might still contrast the two similar objects implicitly, which was reflected by the N1 modulation. As for the current study, the greater diversity of the stimuli (six different photos in one condition: four objects as the standards, one object as the deviant, and one cat as the target) might have reduced the potential discriminative effect because the participants did not need to discriminate between the objects in a systematic manner. Therefore, no N1 deviancy effect was found. Moreover, the selected N1 window (100-150 ms) in the current study was probably too early a stage for higher-order perceptual processing (regarding the differences between classifier categories) to occur.
Overall, the current design and materials were not identical with or as simple as those of previous studies (classifier system vs. terminology; photos or real-world objects vs. color shades/line-drawing pictures; four items as standards vs. one item as standards), so observing an ERP component different from those found in previous research was not completely unexpected.
It seems that the results of the current ERP study were qualitatively different from previous behavioral literature on linguistic relativity with respect to Mandarin classifiers.
49
Saalbach and Imai (2007) and Imai and Saalbach (2010) found that the greater classifier effect in Chinese speakers (versus German speakers) only held for “unspeeded similarity judgments”
(as well as the blank property induction task) but not for the bacteria property induction task and the semantic priming paradigm. Huang and Chen (2014) found that classifier relations benefited Chinese speakers in the noun-recall task only when explicit labels of classifier categories were presented along. As argued by them, these findings pointed to the inactive role of the classifier system in automatic cognitive processes and to the unlikelihood of the classifier system serving as an organizer of conceptual representations (cf. taxonomic and thematic relations). Contrary to their findings, the current study found that the classifier effect could appear in a nonverbal context where pictorial stimuli were rapidly presented (350 ms for one stimulus) and processed without any verbal cue and with conceivable object naming or postperceptual strategies being obstructed (by silently reading dadada…). More importantly, such an effect emerged in early perceptual processing (150-250 ms), which should be in an automatic and unconscious phase. The observed classifier effect on cognition might be limited in scope (e.g., classifier vs. taxonomy) and in processing depth (e.g., perception vs. induction), but the organizational power of classifier classification system, over early perceptual processing in particular, is definitely worth our attention.
Sufficiently early visual perceptual difference in Chinese speakers (linguistic relativity pertaining to Mandarin classifiers) in this ERP study using nonverbal materials directs our
50
attention to the thinking-after-language proposals: language as spotlight (increased sensitivity) and language as inducer (a given mode) (Wolff & Holmes 2011; see Section 1.1). Despite the fact that it remains unclear which proposal better elucidates the perceptual difference between Chinese and English speakers, the results showed that language effect on cognition occurs even when language is not in use. According to the participants’ self-reports after the ERP experiment, none of them was aware of the relations between stimuli and few of them could distinctly remember or even recognize what they had been presented with. This suggests that the participants focused on the target without intentionally processing the other stimuli, not to mention naming each object repeatedly or drawing on some metacognitive strategies within 350 ms.
We admit that the classifier effect may be fragile because in past research it collapsed easily when visual difference or taxonomic nodes, for example, dominated our cognitive processes (Huang & Chen 2014; Imai & Saalbach 2010; Saalbach & Imai 2007). We also acknowledge that this effect is intricate because the classifier system, on top of all kinds of universal classifications that seem already sufficient and necessary, probably only fine-tunes the way we interact with the world. For non-classifier languages, they might embody the intricate classification in other systems, such as grammatical gender and countability. But is the ‘fragile’ and ‘intricate’ nature of the classifier effect enlightening at all, if the classifier effect can be manifest only in an experimental setting with careful, or strict, variable controls?
51
Or, we might ask broader questions: Why do we classify things at all? Why do we classify things in a sophisticated way? What is the extremity of the sophisticated classification? Why do we need language-specific classification systems in addition to culturally universal ones?
Although we are not able to answer these questions within the scope of this study, these tricky, philosophical questions themselves somehow indirectly signify that the classifier classification is not something trivial and meaningless. Whether the classifier classification is a matter of life-and-death in terms of evolution, this neurolinguistic study showed that the effect is there affecting speakers’ perception; the neurobiological salience gives due weight to the effect, and might lend support to anthropological research, for instance. Since the ‘seemingly tiny’
classifier effect cannot be, also should not be, ignored in the story of human cognitive design, it might illuminate the development of artificial intelligence (AI) (S. L. Yeh, personal communication, November 09, 2018). Machines and robots might be more human when part of their computation includes refined classification like ours, which influences visual perception or something beyond.
To the best of our knowledge, this is the first ERP study on Mandarin classifier relativity, and we selected a group of objects rather than only one object to be the standard, which was to ensure that we were testing the covert classifier difference rather than the difference between two objects. However, the design inevitably caused considerable heterogeneity in many aspects (shape, color, function, picture typicality, taxonomic relation, thematic relation, familiarity,
52
grammatical gender, countability, etc.) which might confound the prediction of linguistic relativity. Even so, we did not identify any consistent (dis)similarity between the standards and
grammatical gender, countability, etc.) which might confound the prediction of linguistic relativity. Even so, we did not identify any consistent (dis)similarity between the standards and