The current study adopts the ERP technique to investigate 1) unexpected ending in sentence processing and 2) directionalities in language switching. Sentences included in the study will build up a strong prediction toward their terminal word, and will be completed by a word with an unexpected concept and/or in an unexpected language. The sentences are in Mandarin and in Taiwanese, which are languages used interchangeably in Taiwan. Since the alternative use of languages occurs more frequently in spoken rather than in written form, the experiment stimuli will be
presented in auditory modality. The recruited participants are simultaneous bilinguals (i.e., who have acquired both Mandarin and Taiwanese before puberty), with high proficiency in both languages.
1.3 Significance
The current study provides physiological data on how bilinguals process sentences with an unexpected ending as well as on how different inhibition/activation levels of two languages affect code-mixed sentence processing. As will be discussed in Chapter 2, although similar brainwave component (i.e., N400) could be found in both types of unexpected endings (i.e., unexpected concept or unexpected language), existing research (Proverbio, Leoni & Zani, 2004) did not disassociate the N400 elicited by them. The current study would take the scalp distribution of N400 into consideration, and see whether there is any difference between the two types of unexpected endings. In addition, Van Petten and Luka (2012) recently proposed that sentence completions that did not lexically match participants’ prediction could elicit a frontal positivity 600 ms post-stimulus. Since such a frontal positivity has started to seize researchers’ attention (Federmeier, Wlotko, Ochoa-Dewald & Kutas, 2007;
Thornhill & Van Petten, 2012), the current study would like to test whether this effect could be replicated in the unexpected conditions.
Existing language switching studies had mainly focused on switching between participants’ first language (L1) and second language (L2). However, in the current study, we would like to compare sentences switching between Mandarin and Taiwanese, which are acquired in childhood for a majority of populations in Taiwan.
In fact, such an exploration is a follow-up of previous studies (Liao & Chan, 2011,
2012), in which we found an asymmetric cost in processing Mandarin-Taiwanese code-mixed sentences. We suggested that since Mandarin was more frequently used, it could be more active in the participants’ mind and thus was easier to retrieve. We would like to adopt the ERP technique to further investigate neural activities associated with such a cost.
In the future, the materials in the current study could be cross-examined by fMRI (functional magnetic resonance imaging) and MEG (Magnetoencephalography), which provide spatial (and temporal) resolution that allow us to unfold the neuro-anatomic network of processing sentences with an unexpected ending.
Chapter Two
Literature Review
2.1 Prediction from Sentence Context
It was generally believed that sentences built up a context for people to predict the upcoming words. With the consensus that people may generate predictions based on the preceding sentence context, researchers tried to manipulate the information load carried by a sentence. Sentence constraint referred to how strongly a sentence frame can lead to a particular final word. It was usually assessed by a normative procedure, in which a group of participants (separated from the main experiment) were requested to complete a sentence frame with the first word appearing in their mind. The percentage of the word used to complete a sentence frame was the cloze probability for that sentence (Taylor, 1953). For instance, assuming that over 90% of the participants filled in the sentence frame “Tom has washed all the dishes on the ______” with the word “table,” the sentence could be defined as a high constraint sentence. In contrast, sentence frames such as “In the forest, there were three _______”
could be filled in with various words (e.g., cabins, pigs, flowers, etc.). If “cabins” was the highest cloze word but was provided by only 35% of the participants, the sentence could be categorized as a low constraint sentence.
How sentence constraint influences the processing of the upcoming word(s) had been extensively discussed.1 In psycholinguistic studies, it was reported that high constraint sentences could facilitate the processing of an expected word. In lexical decision tasks, the response time of an expected word following high constraint sentences was significantly shorter than that of a word following low constraint sentences (Schwanenfluegel & LaCount, 1988; Schwanenflugel & Shoben, 1985).
Similar results could be found in naming tasks, in which participants’ naming time to words appearing in high constraint sentences was faster than those in low constraint ones (McClelland & O’Rgan, 1981). Studies with the eye-tracker technique found a higher skipping rate and less likelihood of regression on a contextually constrained word. If such a word was fixated, the fixation time was shorter than a less constrained word (Ehrlich & Rayner, 1981; Rayner & Well, 1996).
In the field of neurolinguistics, Kutas and Hillyard (1984) was the first ERP study that manipulated the levels of sentence constraint to investigate the effect of word expectancy during sentence processing. In this experiment, sentence frames of various constraining levels (high, medium, low) were terminated by words of different cloze probabilities (high, medium, low). The results showed that high constraint sentences with high cloze words elicited a board positive wave whereas other conditions elicited negative-going waves (N400)2. The brainwave differences indicated that participants responded differently when the presented word matched their prediction. It should be noted that sentence constraint and cloze probability
1 A considerable amount of researches have manipulated the sentence context to bias one of an ambiguous word’s meanings (Rayner, Cook, Juhasz, & Frazier, 2006;
Pickering & Frisson, 2001). However, this is beyond the scope of the current study and will not be discussed here.
2 N400, a component peaking around 400ms post-stimulus, had been shown to vary systematically with processing semantic information. It was actually more sensitive to cloze probability rather than the degree of sentence constraint. We will discuss the effect of cloze probability in Session 2.2.
might not be mutually independent. In fact, only highly constrained sentences had the possibility of being terminated by a very high cloze word.
To tease the factors of sentence constraint and cloze probability apart, Federmeier, Wlotko, Ochoa-Dewald, and Kutas (2007) adopted a fairly smart design with high constraint and low constraint sentences. In the experiment, each sentence was either terminated with its highest cloze word (the best completion) or with an unexpected but semantically plausible word of zero cloze probability. The reasoning was that, although the cloze probability of the best completions could not be easily controlled between two constraint levels, the cloze probability of the terminal word could be matched (i.e. zero) in the unexpected but semantically plausible conditions.
Any differences between the unexpected words could be attributed to the effects of sentence constraint per se. The ERP recording revealed that only high constraint sentences with an unexpected completion elicited a positive brainwave at the frontal electrode site. Such an effect, as termed frontal P600 by the authors, was found neither in the best completion in high constraint sentences nor in both completion types in low constraint sentences. The frontal P600 was different from a canonical P600/LPC, which was usually largest at central-parietal electrode site and considered to be an indication of syntactic reanalysis (Osterhout & Holcomb, 1992). Federmeier et al. (2007) interpreted the frontal P600 positivity as “an appreciation to mismatches and/or the allocation of resources necessary to revise a prediction” (p. 8).
Since Federmeier et al. (2007) had demonstrated sentence context could affect the processing of the upcoming word, to further investigate to what degree can people predict an upcoming item, Thornhill and Van Petten (2012) manipulated the semantic relatedness of the sentence completions. In the experiment, high and low constraint sentences were completed by the best completion (BC), by an unexpected
near-synonym of BC (related), and by an unexpected word semantically unrelated to the BC (unrelated). The cloze probability of related condition and unrelated condition were carefully matched. Examples of high and low constraint sentences respectively were “He was afraid that doing drugs would damage his brain (BC)/mind (related)/reputation (unrelated)” and “Penelope started to assemble her new bicycle but was missing the wheels (BC)/tires (related)/instructions (unrelated).” The results revealed that N400 was not only sensitive to target word expectedness (as indicated by cloze probability) but also semantic relatedness to the BC. In contrast, the late frontal positivity, or the frontal P600 in Federmeier et al. (2007), seemed to be more sensitive to lexical form expectations, as it was reduced only in high constraint BC condition. Other conditions did not have significant differences in terms of the amplitude of the late frontal positivity. Thornhill and Van Petten (2012) suggested that readers could predict both conceptual and lexical information of the upcoming words, although the processing of conceptual and lexical mismatches could be subserved by distinctive neural networks.
Both Thornhill and Van Petten (2012) and Federmeier et al. (2007) observed a late frontal positivity in high constraint sentences with unexpected sentence completions. In addition, they both agreed that the late frontal positivity was an indication of disconfirmed prediction. Thornhill and Van Petten (2012) took a step further, by attributing the late frontal positivity to unexpected lexical forms rather than unexpected concepts. However, a major difference arrived when comparing the low constraint conditions in the two studies. In Thornhill and Van Petten (2012), low constraint sentences elicited the late frontal positivity, but such positivity was absent in Federmeier et al. (2007). As for now, it seems that little consensus has been reached on the predictability effect of low constraint sentences. Still, researchers
generally agreed that high constraint sentences provided the context for people to predict the upcoming word. If the presented word did not match the prediction, a late frontal positivity would be elicited. In the current study, we would use high constraint sentences as the experiment materials. We will discuss more about the late frontal positivity in Session 2.2.
2.2 Expectedness of Sentence Completions
Whether the sentence terminal word meets participants’ prediction is reflected by its cloze probability, the percentage of a specific word that a normative group use to complete a sentence. Take the discussed sentence as an example, the sentence
“Tom has washed all the dishes on the ______” could be completed by “table,”
“countertop,” and other reasonable alternatives. If “table” is offered by 90% and
“countertop” by 5% of the normative group, “table” would be categorized as a high-cloze word while “countertop” as a low-cloze word. Behavioral studies had found a facilitating effect in processing high-cloze words (i.e., shorter response time) compared with processing low-cloze words (Schwanenflugel & Shoben, 1985). As for the ERP studies, the results showed that the amplitude of N400 component was graded with the cloze probability of a terminal word. Specifically, low-cloze words elicited larger N400 amplitude than high-cloze words (Federmeier et al., 2007; Kutas
& Hillyard, 1984; Van Petten & Luka, 2012). Some researchers believed that N400 indicated the degree of easiness in contextual integration (e.g., Kutas & Federmeier, 2000).
In addition to the amplitude of N400, the ERP components of high-cloze words and low-cloze words also differ in the late time window (between 500 to 900ms). In a
systematic survey, Van Petten and Luka (2012) detected that when sentences were completed by unexpected endings (low-cloze words), a late positivity was found at the frontal site. The scalp topography was different from a canonical P600, which was found to be larger at the central-parietal site, and was sensitive to syntactic violations (Osterhout & Holcomb, 1992). Researches in the past seemed not notice the late frontal positivity effect, so they left this time window unanalyzed. According to Van Petten and Luka (2012), among 18 studies that had analyzed this time window, 17 of them reported a frontal rather than parietal positivity. However, these studies had different interpretations toward this positivity. Federmeier et al. (2007) argued that the late frontal positivity indicated a need to reallocate resources to revise the prediction.
As for Thornhill and Van Petten (2012), they considered the late frontal positivity an indication of a lexical form mismatch. Another perspective is proposed by Kutas (1993), who speculated that the frontal positivity reflected inhibition of words that were expected but not presented. Nevertheless, as Van Petten and Luka (2012) pointed out, Kutas’ proposal was a fairly strong claim and should be further tested.
Admittedly, the exact cognitive function indexed by this frontal positivity remained unclear. The consensus so far was that the late frontal positivity indicated a conceptual or lexical mismatch between the expected word and the presented word.
While low-cloze words were less expected candidates in a sentence, the least expected case was created by sentences ended with a semantically inappropriate word.
This line of research examined the differences between semantic congruous and incongruous sentences. Kutas and Hillyard (1980a) were researchers who initiated this line of research.3 Their congruous and incongruous materials were sentences like
“It was his first day at work” and “He spread the warm bread with socks.” The ERP
3 In fact, it is in Kutas and Hillyard (1980a) that the effect of N400 is discovered.
recordings revealed that compared to congruous sentences, a larger N400 was elicited from the onset of the problematic word in incongruous sentences. A considerable amount of researches had been devoted to this line of research, and they all achieved similar results (Andrews et al., 1993; Revonsuo, Portin, Juottonen, & Rinne, 1998).
Recently, Van Petten and Luka (2012) added that while processing incongruous sentences, N400 was usually followed by a large positivity. The positivity was larger at the parietal site, which was fairly similar to the scalp distribution of a canonical syntactic P600/LPC. In fact, researches of complex sentences processing also found such a late parietal positivity. Specifically, their experiment stimuli were semantically incongruous and did not violate syntactic regulations (Kim & Osterhout, 2005;
Kuperberg, Sitnikova, Caplan, & Holcomb, 2003). Based on these findings, P600/LPC was interpreted as an attempt to revise/reanalyze the interpretation of the relatively complex sentence. Van Petten and Luka (2012) contended that the dichotomy of “syntax P600 and semantic N400” could be reconsidered. The late parietal positivity could be elicited by incongruous sentences, and it reflected an attempt to reanalyze the problematic sentences.
In sum, compared with high-cloze words, less expected low-cloze words elicited not only larger N400 but also larger late positivity at the frontal site. As for the least expected incongruous sentence completions, they elicited a larger N400 and a larger late positivity at the parietal site compared with congruous sentence completions. It could be possible that since the posterior positivity was too strong, existing studies did not examine frontal positivity in incongruous sentences. Van Petten and Luka (2012) argued that the two signals, late frontal positivity and late parietal positivity, should arise from different brain regions and should be attributed to different functional processes. In the current study, we will include both high-cloze
and low-cloze words and we predict that a late frontal positivity will be observed in low-cloze word conditions.
2.3 Expectedness of Sentence Code
2.3.1 Code-mixing
What we have discussed so far are studies on expectedness of sentence processing within one language. In a bilingual speech community, people tend to switch languages back and forth in conversations. The mixing use of two languages within one sentence was defined as code-mixing (Bokamba, 1989). Taken language to be used as an unpredictable factor, researchers initiated several lines of studies and each line had its own interest of focus. Some studies examined code-mixed word recognition in contexts (Li, 1996; Chien, 2000), some explored the effect of ambiguous cognates/interlingual homographs (Altarriba, Carlo, & Kroll, 1992; Duyck, Van Assche, Drieghe, & Hartsuiker, 2007; Schwartz & Kroll, 2006), and still some investigated code-mixed sentence processing (Chen, 2004; Moreno, Federmeier, &
Kutas, 2002). Since the former two lines of studies are beyond the scope of the current study, our discussion will be focused on code-mixed sentence processing.
In early off-line studies, Kolers (1966b) and Macnamara and Kushnir (1971) found that as the number of code-mixed words increased in a text, participants’
reading time also increased. However, the methods of both studies were criticized. It was argued that other factors, such as the grammaticality of experiment stimuli and the requirement of the task, could all confound with the results (Grosjean, 1982;
Paradis, 1980c). Recently, as computers and other new techniques were applied into
research, on-line experiments were conducted to investigate this inquiry. Under such a paradigm, participants were presented with code-mixed/non-mixed sentences and were requested to make a response by button-pressing. Participants’ response times were recorded and analyzed. The results of a majority of studies had shown that response times to non-mixed sentences were significantly shorter than mixed ones (Liao & Chan, 2011, 2012; Proverbio, Leoni, & Zani, 2004). Notice that such a processing cost was absent in Chen (2004). However, it was possible that the insignificant effect in Chen was due to the limited number of participants (n = 10).
In neurolinguistic researches, both fMRI and ERP techniques were adopted to investigate language switching. fMRI studies revealed that processing code-mixed sentence was a complex task. Code-mixed sentence processing was accomplished by an extensive neurological network, and there was no specific brain region specialized for language switching. Compared with non-mixed conditions, Wang, Xue, Chen, Xue and Dong (2007) reported that code-mixed conditions activate brain regions in the right superior prefrontal cortex, left middle and superior frontal cortex, and right middle cingulum and caudate. Abutalebi et al. (2007) presented similar results. That is, language switching involved bilateral prefrontal and temporal associative regions. In addition, Abutalebi et al. also investigated differences between code-switching (i.e., switching languages between sentences) and code-mixing (i.e., switching languages within a sentence frame). They discovered that while processing code-switched sentences activated brain regions related to lexical processing, processing code-mixed sentences entailed brain structures related to syntactic and phonological processing.
As for ERP studies, Moreno et al. (2002) was the first study that adopted the ERP technique to examine language switching. The experiment materials were English sentences completed by an English expected word, by a lexical switch (an
English synonym of the expected word), and by a code switch (the Spanish equivalent of the expected word). The ERP recordings revealed that while lexical switch condition elicited an N400 effect, code switch condition elicited a left anterior negativity (LAN) between 250 to 450 ms, which was more sensitive to syntactic processing, especially those that taxed working memory (King & Kutas, 1995).
Moreno et al. proposed that if this effect was a LAN, then it might be resulted from different morphological agreement systems in Spanish and English. Still, other language switching studies (Alvarez, Holcomb, & Grainger 2003; Chen 2004;
Proverbio et al., 2004) consistently reported an N400 effect rather than a LAN in this time window. Compared with an expected ending, language switching seemed to cause problems in lexical integration.
Back to the study of Moreno et al. (2002), another major difference between the lexical switch condition and the code switch one was observed in a late time window. Only the code switch condition elicited a large positive complex (LPC, also known as P600), especially at the parietal and occipital sites. Moreno et al. argued that the P600/LPC component revealed that participants required more resources for stimulus evaluation. In fact, although P600/LPC component was originally proposed as an indication of syntactic violations (Osterhout & Holcomb, 1992), as has been discussed in Session 2.2, recent studies suggested that P600/LPC could reflect a process of sentence reanalysis, such that complicated sentences (e.g., semantically incongruous sentences, garden-path sentences, etc.) would elicit this effect (Kim &
Osterhout, 2005; Osterhout, Holcomb, & Swinney, 1994). In addition, it was suggested that P600/LPC could be related to executive control in sentence level (Kolk
& Chwilla, 2007): schizophrenic patients, who may have difficulties in monitoring speech perception, showed a reduced P600/LPC effect (Kuperberg, Sitnikova, Goff, &
Holcomb, 2006). While it seemed that P600/LPC could be associated with various
Holcomb, 2006). While it seemed that P600/LPC could be associated with various