Data Analysis - 句尾非預期效應對語句處理的影響：事件相關腦電位研究

The EEG data was processed with Edit 4.5 software from NeuroScan Incorporated. To begin with, a linear derivation file was imported to convert the four monopolar eye-movement monitoring channels to two bipolar channels (VEOG and HEOG). Next, ocular artifact reduction was conducted. The parameter setting was listed below: VEOG was selected to be the channel that monitored blinking, with positive as the trigger direction and 10% as the threshold value. To determine the blink value, at least 20 sweeps that contained blinking were reviewed. The duration of

blinking was set to be 400 ms. Once the ocular artifact reduction was performed, the blinking noise was corrected.

After the blinking noise was reduced, the continuous EEG file was epoched, and the epoch spanned from -200 to 1000 ms of the stimulus onset. Baseline correction was then applied with the pre-stimulus -200 to 0 ms interval. After baseline correction, artifact rejection was carried out by reviewing the sweeps manually.

Sweeps contaminated by excessive eye movements, body movements, skin potentials, and amplifier saturation were rejected. The overall rejection rate was 33% (range:

27% to 43%). Artifact-free epochs were then sorted and averaged based on the event codes referring to the experimental conditions. Infinite Impulse Response (IIR) filter was applied to each condition, with the band-pass value set to be 0.01 Hz to 30 Hz, 12 dB/oct. Finally, grand average was conducted by averaging all the participants’

averaged ERP data with the same experimental condition(s).

For statistic analysis, mean amplitudes in the 350 to 550 ms time window and in the 750 to 950 ms time window⁷ were exported to examine the N400 and the Post-N400 positivity (P600/LPC) effects, respectively. For theN400 effect, since previous studies did not specify regional differences between cloze-probability-related N400 and language-switching-related N400, two regions of interest in general, the frontal-central scalp sites and the parietal scalp sites, were chosen. Nine channels from the frontal-central sites (i.e., F3, FZ, F4, FC3, FCZ, FC4, C3, CZ, and C4) and nine channels from the parietal sites (i.e., CP3, CPZ, CP4, P3, PZ, P4, O1, OZ and O2) were selected. As for the P600/LPC effect, two other regions of interest were chosen so that we could examine both the anterior positivity and the

“traditional” P600/LPC at parietal sites. Seven channels at the prefrontal and frontal

7A slightly late time window was chosen to avoid overlap with N400.

sites (i.e., FP1, FP2, F7, F3, FZ, F4, and F8) and six channels at the parietal sites (i.e., CP3, CPZ, CP4, P3, PZ, and P4) were selected. The mean amplitudes of each region were measured by averaging the selected electrodes in the particular area.

For each time window, a four-way repeated-measure ANOVA was carried out to examine the factors of cloze probability (high, low), language mode (Mandarin, Taiwanese), switching (non-switched, switched), and anteriority (frontal, parietal).

Follow-up post-hoc paired t-tests would be conducted when interactions were observed. A relatively strict alpha level of .01 was adopted to avoid type I error resulted from multiple comparisons.

Chapter Four

Results

The ERP technique was adopted to investigate 1) how people process sentences terminated by an unexpected but semantically plausible word, 2) how people process sentences with the terminal word switched into another language (i.e., code-mixed sentences), and 3) whether switching directionality (i.e., sentences switched from Mandarin to Taiwanese and sentences switched from Taiwanese to Mandarin) matters. We will first report the behavioral results. Then we will examine brainwaves elicited by the high-cloze and the low-cloze conditions and by the non-switched and the switched conditions. We will also explore the effect of language used in sentence contexts, which could be interpreted as participants’ language mode prior to the target word.

For the behavioral results, participants’ accuracy rate and their response times to the probe words were calculated. The overall accuracy rate of the word recognition task was 98.6%, indicating that participants were paying attention in the experiment.

Response times to probe words that had occurred in the sentence context were significantly shorter than those that had not (present: 1163 ms; absent: 1269 ms;

t(20)=3.288, p <.005).

We first wanted to see if the effects of expectancy (high- vs. low-cloze probability), switching (switched vs. non-switched) and language (Mandarin vs.

Taiwanese) were clear in our sentence processing data, so the grand average waveforms to the targets were computed for each factor and then plotted in Figures 4, 5 and 6, respectively. Figure 4 shows the grand average ERPs to the high-cloze and

the low-cloze conditions. In both conditions, the ERPs were characterized by an initial positivity (P1) peaking at about 110 ms, followed by a negativity (N1) at 160 ms, and a positivity (P2) at around 260 ms. In continuous speech, N1 and P2 were usually reduced due to lack of clear physical boundaries between words (Hagroot &

Brown, 2000; Liu, Shu, & Wei, 2006; Van Petten, Coulson, Rubin, Plante, & Parks, 1999). It is possible that in our experiment, each sentence stimulus was presented word by word, with a 200 ms interval between words, such that these early components could be clearly observed. These responses were followed by a negative-going wave between 300 and 600 ms (N400). By visual inspection, the N400 was larger for low-cloze conditions, especially over central-parietal sites. As time proceeded, responses of the low-cloze condition became more positive over the prefrontal and frontal sites compared with the high-cloze condition.

Figure 4: Grand average ERPs from 21 subjects from all scalp sites, showing

responses to the high-cloze conditions and the low-cloze conditions.

Plotted in Figure 5 are the grand average ERPs for the non-switched and the switched conditions. Early components such as P1, N1 and P2 were observed in both conditions. Following these responses, a slow negative shift was detected in the switched condition between 300 and 600 ms (N400), with an extensive scalp distribution. Such a widespread N400 was fairly different from the N400 elicited by the low-cloze condition, which was larger over central-parietal sites (See Figure 4).

After the N400, visual inspection revealed a greater positivity for the switched condition over the parietal sites.

Figure 5: Grand average ERPs from 21 subjects from all scalp sites, showing responses to the non-switched condition and the switched conditions.

Figure 6 displays the grand average ERPs for conditions of context languages (i.e., Mandarin and Taiwanese), which was the base language from which the target was switched and could be interpreted as participants’ language mode prior to the target word. Both conditions elicited early P1, N1, P2 responses, and these

components were followed by a negative shift from 300 to 600 ms (N400). After the N400 time window, the brainwaves became more positive over parietal sites in both conditions. A sustained negativity, however, was observed over the frontal sites when participants were in Mandarin mode. Although brainwave patterns of the two conditions were fairly similar, responses elicited by Taiwanese mode were more positive throughout the whole epoch. Differences between the two conditions became larger as the time proceeded. All the above information indicated that the two languages were processed differently, and languages mode could influence the process of the upcoming word.

Figure 6: Grand average ERPs from 21 subjects from all scalp sites, showing responses to the Mandarin-based condition and Taiwanese-based conditions.

To statistically test the patterns described so far, four-way repeated-measure ANOVAs were carried out on the mean amplitudes of the N400 and LPC/P600 to

switching (non-switched, switched), and anteriority (frontal, parietal). The time window for the N400 was 350 to 550 ms and that for the LPC/P600 was 750 to 950 ms. Follow-up paired t-tests were performed when interactions were observed. A relatively strict alpha level of .01 was adopted to avoid type I error resulted from multiple comparisons. We will start from statistic analysis of N400 latency window, followed by the analysis of P600/LPC.

At the N400 time window, a four-way repeated-measure ANOVA was carried out to examine the factors of cloze probability (high, low), language mode (Mandarin, Taiwanese), switching (non-switched, switched), and anteriority (frontal-central, parietal). Significant main effects were found in cloze probability (F(1, 20)=9.609, p<.01) and switching (F(1,20)=15.737, p<.005), indicating that, in line with our visual inspection, the N400 effect was stronger in the low-cloze than in the high-cloze condition and stronger in the switched than the non-switched condition. There were no main effects for language mode (F(1,20)=3.785, p=.066) and anteriority (F(1,20)=.692, p=.415). A two-way interaction was observed between cloze probability and anteriority (F(1,20)=21.610, p<.001). Post-hoc paired t-test showed that over the parietal sites, the N400 amplitude elicited by the low-cloze condition was significantly larger than the high-cloze condition (t(20)=3.851, p<.005). Such a significant result, however, was not observed over frontal-central sites (t(20)= 2.099, p=.049).

There was also an interaction between language mode and switching (F(1,20)=

9.723, p<.01). Follow-up paired t-tests revealed that in the Mandarin mode, the N400 amplitude was significantly larger in the switched condition than in the non-switched condition (t(20)=4.172, p<.001). In other words, compared with Mandarin non-mixed sentences, Mandarin-Taiwanese code-mixed sentences elicited a larger N400 effect.

However, such an effect was absent while comparing Taiwanese code-mixed sentences with Taiwanese non-mixed sentences. That is, in Taiwanese mode, the N400 amplitude did not differ between the switched and the non-switched conditions (t(20)=-.449, p=.658). See Figure 7 for the averaged brainwaves for the switched and the non-switched conditions in the Mandarin mode and Figure 8 in the Taiwanese mode. To ensure that the above patterns were not resulted from different processing baselines of the two languages, Mandarin non-mixed sentences were compared with Taiwanese non-mixed ones. Statistics revealed that there were no differences between them (t(20)=.181, p=.858). The asymmetric results of N400 could thus be attributed to switching effect per se. See Figure 9 for the grand average ERPs for non-mixed sentences in both languages. Directly comparing the two switched conditions (from Mandarin to Taiwanese vs. from Taiwanese to Mandarin), with a paired t-test revealed a significant effect (t(20)=-3.429, p<.005), suggesting an underlying difference of switching directionality. See Figure 10 for grand average ERPs for the two switching directionalities.

Figure 7: Grand average ERPs from 21 subjects from all scalp sites, showing responses to Mandarin non-mixed sentences and Mandarin-Taiwanese code-mixed sentences.

Figure 8:Grand average ERPs from 21 subjects from all scalp sites, showing

responses to Taiwanese non-mixed sentences and Taiwanese-Mandarin

code-mixed sentences.

Figure 9:Grand average ERPs from 21 subjects from all scalp sites, showing responses to Mandarin none-mixed sentences and Taiwanese none-mixed sentences.

Figure 10:Grand average ERPs from 21 subjects from all scalp sites, showing responses to Mandarin-Taiwanese code-mixed sentences and

Taiwanese-Mandarin code-mixed sentences.

Finally, there were no three-way or four-way interactions among the four factors.

To investigate the post-N400 positivity (P600/LPC), a four-way repeated-measure ANOVA with factors of cloze probability (high, low), language mode (Mandarin, Taiwanese), switching (non-switched, switched), and anteriority (frontal, parietal) was performed. The results revealed main effects of language mode (F(1,20)=15.56, p<.005) and anteriority (F(1,20)=14.242, p<.005), indicating that language mode could affect the processing of the upcoming word, and that different polarity patterns could be observed between anterior and posterior regions. No main effects were found for the factors of cloze probability (F(1,20)=.095, p=.761) and switching (F(1,20)=.021, p=.887). However, there were two-way interactions between switching and anteriority (F(1,20)=9.508, p<.01). Pairwise comparisons revealed that the switched condition induced a stronger positivity in the posterior than in the anterior sites, which was not the interest of the current study and thus we will not discuss this interaction further.

The ANOVA also revealed a two-way interaction between cloze probability and anteriority (F(1,20)=28.590, p<.001). Paired t-tests revealed that no significant differences were observed over parietal sites between the high-cloze and the low-cloze conditions ((t(20)=2.048, p=.054), whereas over the prefrontal-frontal sites, differences were marginally significant ((t(20)=-2.581, p=.018). A topographic map of the high- and low-cloze conditions could demonstrate the appearance of the frontal positivity in the latter condition more clearly. As can be seen in Figure 10, response to the low-cloze condition was more positive over the left frontal region, compared with that to the high-cloze condition in the same region, where the brain response remained to be negative.

Figure 11: The topographic map (750-1000ms, 25 ms interval) for the high-cloze and the low-cloze conditions. Positivity was painted in red and negativity was in blue.

There was also a two-way interaction between language mode and switching (F(1,20)=8.333, p<.01). Paired t-test showed that in the Taiwanese mode, differences between the switched and the non-switched were marginally significant ((t(20)=

-2.818, p=.011), whereas in the Mandarin mode, such an effect was absent ((t(20)=

1.969, p=.069). That is, while Taiwanese-Mandarin code-mixed sentences elicited a P600/LPC effect compared with Taiwanese non-mixed sentences, this effect was not observed when comparing Mandarin-Taiwanese code-mixed sentences with Mandarin non-mixed sentences. To investigate whether the baselines of different languages could confound with the observed patterns, non-mixed sentences in Mandarin and in Taiwanese were analyzed. Statistics showed that there were no differences between

them (t(20)=-1.023 , p=.319). As for the effect of language mode on a language-switched target (i.e., Mandarin-Taiwanese vs. Taiwanese-Mandarin code-mixed sentences), a significant difference was found ((t(20)=-5.086, p<.001). In other words, sentences of different switching directionalities were processed differently.

Finally, there were no three- or four-way interactions among the four factors.

Chapter Five

Discussion and Conclusion

The current study aims to investigate how people process sentences with an unexpected ending and whether there are any differences between two switching directionalities. The ERP technique was adopted to address the following research questions: 1) How do people process sentences terminated by an unexpected but semantically plausible word? 2) How do people process sentences with terminal word switched into another language (i.e., code-mixed sentences)? and 3) what are the difference between sentences switched from Mandarin to Taiwanese and sentences switched from Taiwanese to Mandarin. Intriguing results were observed in responding to these questions.

To begin with, the expectancy of a sentence terminal word was indicated by its cloze probability. Sentences with low-cloze targets elicited a larger N400 effect over the parietal sites compared with those with high-cloze targets. Such a pattern is consistent with previous studies (Luck, 2005; Van Petten & Luka, 2012), indicating that the manipulation of cloze probability in the current study was successful. In addition to the N400 effect, more and more studies proposed that low-cloze conditions would also elicit a late positivity over frontal sites (DeLong, Urbach, Groppe, & Kutas, 2011; Thornhill & Van Petten, 2012; Van Petten & Luka, 2012). In line with previous studies, we also observed a frontal positivity at a later time window, despite the fact that the significance was somewhat marginal (.018). It is postulated that if the constraint of sentence stimuli was higher (an average of 69% for the current

Federmeier et al. (2007)), the low-cloze targets might be less predictable, and the effect of a frontal positivity might be larger. Another possible explanation is that such an effect might be more sensitive to visual stimuli. Most of the studies that observed this component were visual experiments. Auditory experiments seemed not to catch this effect (Diaz & Swaab, 2007; Van Petten et al., 1999). We will come back to address the cognitive function indexed by this late frontal positivity as we discuss ERP responses to cloze probability and to language switching.

When the sentence terminal word was switched into another language, a significantly larger N400 effect was observed. The finding replicates the results of previous literature (Chen, 2004; Proverbio et al., 2004), showing that language switching may lead to difficulties in lexical integration. Some of the previous studies (e.g., Moreno et al., 2002) added that an LPC/P600 effect was found in the switched condition, which was interpreted as an indication of sentence reanalysis. The LPC/P600 effect was also correlated with participants’ language proficiency.

Specifically, a larger LPC/P600 could be observed from participants of lower L2 proficiency. In the current study, at LPC/P600 time window, visual inspection indeed suggested that responses to the switched condition were more positive than the non-switched condition over the parietal sites; however, no statistically significant LPC/P600 effect was obtained. It is possible that since our participants acquired both languages before puberty, and their proficiency of both languages were fairly high, the LPC/P600 effect was thus absent. Another possibility for the lack of LPC/P600 is that, a word recognition task, instead of a sentence comprehension task, was adopted in the current study. Now that the participants’ task was to judge whether a probe had been presented in the previous sentence, they may not reanalyze the sentence as carefully as those subject in previous experiments using sentence comprehension

tasks. Existing research had reported that the P600/LPC amplitude could be affected by the demand of experimental tasks (Brouwer, Fitz, & Hoeks, 2012). When participants were requested to perform sentence acceptability/grammaticality tasks, the P600/LPC would be larger than reading sentences for comprehension (Kolk, Chwilla, van Herten, & Oor, 2003; Van Petten & Luka, 2012).

So far we have found that sentences terminated by a word with an unexpected concept (the low-cloze condition) and by a word in another language (the switched condition) would elicit an N400 effect. Such findings were in line with existing literature (Alvarez et al., 2003; Federmeier et al., 2007; Kutas & Hillyard, 1984;

Proverbio et al., 2004; Van Petten & Luka, 2012), showing that in both cases, participants may encounter difficulties in lexical integration. Nevertheless, these studies did not differentiate the language-switching-related N400 from the cloze-probability-related N400. Although it is well attested that the N400 elicited by a low-cloze word was largest over central-parietal sites (Luck, 2005; Thornhill & Van Petten, 2012), the scalp distribution of a language-switched N400was unclear.

Specifically, previous studies (Alvarez et al., 2003; Proverbio et al., 2004) presented their findings from a few selective channels, leaving the scalp topography unaddressed. With the design of the current study, we intended to tell the differences between the two. Table 3 summarizes the ERP responses to the manipulation of cloze probability and of switching. As can be seen, the low-cloze N400 was larger over parietal sites. However, the switched N400 did not interact with anteriority, showing an extensive scalp distribution. The different scalp distributions between cloze-probability-related N400 and language-switching-related N400 suggest that they may be processed differently. In the future, other research techniques (i.e., fMRI,

MEG) could be adopted. With better spatial resolution, the underlying mechanisms of processing the two types of unexpected endings could be revealed.

Table 3: ERP responses to cloze probability and to language switching

N400

In addition to different N400 scalp distributions, another difference was observed between the two types of ERP response to unexpected sentences. At a later time window, as indicated in Table 3, a frontal positivity, although not statistically significant, was observed only when comparing the low-cloze condition with the high-cloze condition. As has been discussed in chapter 2, researchers have not yet reached a consensus on the exact cognitive function indexed by this frontal positivity.

Federmeier et al. (2007) believed that the late frontal positivity indicated a need to reallocate resources to revise a wrong prediction. For Kutas (1993), it reflected an inhibition of words that were expected but not presented. DeLong et al. (2011) argued that the late frontal positivity revealed a disconfirmed prediction, whereas Thornhill and Van Petten (2012) took a step further, contending that it reflected a lexical form mismatch. In the following discussion, we will focus on DeLong et al.’s and Thornhill and Van Petten’s perspectives, because we may be able to differentiate whether such a frontal positivity indicates a disconfirmed concept or a disconfirmed lexical form with the current experiment design. If the frontal positivity revealed a lexical form mismatch, we should be able to observe this effect in the language-switched condition,

in which the concept was kept the same but the lexical form was switched into another language. However, the frontal positivity was not found in this case. Only when the concept of the presented word did not match that of an expected word/concept can such an effect be observed. Taken the results together, we agree with DeLong et al. that this frontal positivity indicates a disconfirmed prediction.

Specifically, it is more sensitive to a disconfirmed concept, rather than a disconfirmed lexical form as suggested by Thornhill and Van Petten.

As for switching directionality, it is addressed under the interaction between the factors of language mode and switching. While statistics revealed a main effect in language mode, indicating that language mode could affect the processing of the upcoming word, a more interesting result is that language mode actually interacted with switching at both N400 and LPC/P600 time windows. As a matter of fact, Mandarin-Taiwanese code-mixed sentences and Taiwanese-Mandarin code-mixed sentences behaved fairly differently at the two time windows. Table 4 summarizes the ERP responses to the two switching directionalities. At the N400 time window, Mandarin-Taiwanese code-mixed sentences elicited a larger effect compared with Mandarin non-mixed sentences. In contrast, such an effect was absent when

在文檔中句尾非預期效應對語句處理的影響：事件相關腦電位研究 (頁 48-95)