Chapter 2 Literature Review
2.4 Eye movement recording technique
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
occurred in the N400 component with a frontal central distribution. In addition, the homophone density effect was also observed in the late positive component of LPC during the process of experiment tasks. In general, this study demonstrated that ERPs is sensitive to orthographic congruency in spoken Chinese character recognition. In accordance with the view of Grainger and Ferrand (1994), 陳薇帆 (2011) supported the cross-code consistency which emphasize the importance of bi-directional interaction concerning the presentations of phonology and orthography.
2.4 Eye movement recording technique
The general characteristics of eye movements during reading have been studied in great depth over century (Rayner, 1995). Researchers have employed eye movement techniques to show that listeners are remarkably sensitive to subtle aspects of language, and engage the information in directing their gaze around a visual display (Rayner, 1998). In addition, numerous eye movement measurements fundamentally lead researchers to develop further analyze with a variety of indices.
Eye movements are generally viewed as a fundamental and essential role to the operation of sentence reading in visual system. With the eye movement recording technique, readers are settled to read sentences in a normal and natural circumstance.
The indexes of eye movements during reading such as gaze durations, saccade lengths,
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
occurrence of regressions, and a number of variations on these measures can furthermore be employed to infer moment-by-moment cognitive processing of a text by the reader (Rayner, Sereno, Morris, Schmauder, & Clifton, 1989). In general, the first-pass measures reflect the initial word recognition process, while the second-pass measures entail the time of the later stage of semantic or syntactic integration. Some typical first-pass and second-pass eye movement measures for the analyses are listed as following:
(1) First-pass duration and probability:
a. First fixation duration (FFD): the duration of the first forward fixation in the ROI.
b. Single fixation duration (SFD): the duration of only one first-pass fixation within the ROI.
c. Gaze duration (GD): the sum of all the first-pass fixations before moving out the ROI.
d. Skipping rate (Skip): the probability of skipping the ROI in the first-pass reading.
e. Refixation rate (RFR): the probability of refixating the ROI during the first-pass gaze.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(2) Second-pass duration and probability:
a. Total viewing time (TVT): the sum duration of all fixations in the ROI, regardless of the forward or backward eye movements.
b. Rereading rate (RRR): the probability of returning to the ROI after the first-pass reading.
By the utilization of eye-movement-contingent display technique to investigate the application of phonological and orthographic information from parafovea in Chinese characters identifications, Tsai et al. (2004) examined the possible connections between phonological preview benefit and the target characters’ O-P consistency value by two designed eye movement experiments. In Experiment 1, phonological and orthographic similarities between preview and target characters were manipulated orthogonally, according to whether they were homophones and whether they shared the same phonetic radical. The result of Experiment 1 suggested the significant phonological benefits specify that phonological coding is activated early when the character is in the parafovea. The main purpose in Experiment 2 was to determine whether the O-P consistency value of target characters would yield possible influence on phonological preview benefit. The main manipulation was that one set of target characters was with high consistency value and the other set of target characters with low consistency value. The result revealed that phonological preview
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
benefits were reliable only with the high consistency target characters. These two experimental outcomes suggested the relatively early activation of phonological codes at lexical as well as sublexical levels. In addition, the O-P consistency of the pronunciation between characters and phonetic radicals had the initial and critical role in sublexical phonology. The conclusion of different linguistic codes contributing to character identification was in accordance with an interactive activation model during the processes of Chinese character identification.
Not only for the investigation of sentence reading process, eye trackers have also allowed researchers to extend the online measurement of language processing to the perception and understanding of spoken language in a rich, naturalistic visual context (Richardson, Dale, & Spivey, 2007). When receiving the auditory input, subjects are gradually influenced by the incremental delivery of linguistic information. The continuously activating representations will arise during processing at the time course along with eye movements. In addition, eye movements have permitted the observation of powerful interactive effects between language and vision. Thus, features of eye movements can be exploited to study several factors in the time course of speech processing and language understanding. The incremental process of language is strongly constrained by appropriate visual contexts and the moment-by-moment visual perception which driven by subtle aspects of language
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
such as phonological and orthographic characteristics. Salverda (2010) used the visual world paradigm with eye movement recording to evaluate the retrieve of orthographic information and its time course regarding to the spoken word recognition. In Experiment 1, participants saw four words on a computer screen and listened to a spoken instruction to click on one of the words (e.g., Click on the word bead). The visual display mainly included the target word (e.g., bead), two orthographically and phonologically unrelated distractors (e.g., flop, dune), and a competitor with either a lower degree of phonological overlap with the target (e.g., bear) or a higher degree of phonological overlap with the target (e.g., bean). Both competitors had the same degrees of orthographic overlap with the target. The visual display is presented in Figure 4.
Figure 4. Example of a visual display (Salverda, 2010)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
In Experiment 2, the degrees of orthographic overlap with the target were manipulated. Different with Experiment 1, the visual display mainly consisted of the target word and a competitor with either a higher degree of orthographic overlap with the target (e.g., bear) or a lower degree of orthographic overlap with the target (e.g., bare). Due to the fact that the lower and higher orthographic overlap competitors had
the same degree of phonological overlap with the target, comparing the two conditions could examine the time course and the activation regarding to the orthographic information. The result was obtained by computing the mean proportion of fixations to the competitor between 200 and 717 ms after word onset for each participant and each item (i.e., during the acoustic length of the target word, assuming a 200 ms delay for programming an eye movement), the result of Experiment 2 is presented in Figure 5.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 5. Proportion of fixations to the target, the competitor, and the averaged distractor in the lower orthographic-overlap condition and the higher
orthographic-overlap condition in Experiment 2
Overall, the difference in the proportion of looks to higher and lower phonological overlap competitors was not significant, suggesting that the degree of phonological overlap between target and competitor did not affect the probability of fixating the competitor words. However, listeners were more likely to fixate a competitor with a higher degree of orthographic overlap with the target (23%) compared to a competitor with a lower degree of orthographic overlap with the target (17%) during the time interval. This result recommended that fixations to printed words were mediated by orthographic information connected with the spoken target
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
word; the degree of orthographic overlap between spoken word and printed word could influence the degree to which listeners fixate a word. In addition, the results also indicated that the orthographic code was activated during the acoustic time of the target word, implied that mapping between the phonetic signals and printed words is mediated by orthographic information which activated upon hearing the spoken target word.
Previous studies had demonstrated that eye movements recording techniques were uniquely connected as well as poised between perception and cognition, which can be viewed as a window into perceptual and cognitive processing. In the main, the eye movements pattern can provide researchers with a rich, dynamic data source regarding to the temporal dynamics and psychological processes that led up to the exploration of language’s intrinsic nature, as they allow for detailed measurements of how we interacting with the visual and auditory input regarding to phonology and orthography in language system. In present study, the eye movement recording technique is adopted to detect the phonological-to-orthographic (P-O) mappings in visual and auditory modality, so as to obtain further understandings regarding to the lexical recognition also cognition mechanism related to the Chinese language system in brain and mind.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 3
Experiment One:
P-O mappings of Chinese character in sentence reading
The goal of Experiment 1 was to investigate the P-O consistency and homophone density effects of Chinese characters in a sentence reading task. Three types of target characters were selected. Based on the interactive view between sound and spelling, we predicted that the reading process involves a dynamic and interactive coupling between phonology as well as orthography between different grain sizes, which was in accordance with the concept that the feedback mapping from phonology to orthography in point of fact influences the recognition of visual word. P-O mappings should have an essential role which demonstrates influence upon the normal reading process.
If the P-O mappings basically related to the triggering between phonological as well as orthographic codes, the orthographic processing units should spread to the associated phonological units when the visual target is presented in lexical and sublexical level, demonstrating the activation of mappings from phonology to orthography upon the normal reading process in visual modality. Specifically, at the
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
sublexical level, we predicted that eye movement indices regarding fixation proportions and fixation probabilities would show significant differences between High-HD/High-PO and High-HD/Low-PO characters, demonstrating the P-O consistency effect. Later, at the lexical stage, homophone density effect (HD) would be illustrated by the statistical differences between High-HD/High-PO and Low-HD/High-PO characters.
3.1 Method
3.1.1 Participants
Thirty-six university students, including 25 females and 11 males were paid to participate in the experiment. Their mean age was 22.5 years old, ranging from 19 to 28 years old. All participants were right handed. All of them were native speakers of Mandarin Chinese and they were naïve to the purpose of the experiment. All participants had normal or correct-to-normal vision. None of them took part in the norming study of cloze task.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.1.2 Materials and Design
The experimental design mainly based on the rationale of (陳薇帆, 2011). With the manipulation of P-O consistency ratio and homophone density value, there were three types of 72 target characters in our Experiment 1: (1) High-HD/High-PO: high homophone density and high P-O consistency characters, (2) Low-HD/High-PO: low homophone density and high P-O consistency characters, (3) High-HD/Low-PO: high homophone density and low P-O consistency characters. Stimuli were selected from a Chinese character corpus with the values of bidirectional consistency of spelling and sound developed by Brain and Language Laboratory at Academia Sinica in Taiwan.
The corpus lists all Chinese monosyllabic characters with their corresponding spelling, P-O consistency ratio and homophone density. The P-O consistency ratio varies between 0 and 1, which reflects the degree of consistency.
Target words in the three conditions matched in stroke, the neighborhood size of first constituent character (NS1) and word frequency. The ANOVAs on stroke, frequency and NS1 revealed that there were no significant differences across three conditions [F < 1]. The properties of target character are presented in Table 3.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Table 3. Means and standard errors (in parentheses) of P-O value, HD value, stroke, frequency and NS1 for target characters in Experiment 1
Condition P-O value HD value Stroke Frequency NS1
High-HD/High-PO 0.97 (0.0) 6.83 (2.5) 10.67(3.2) 2.27 (0.5) 17.04 (11.4) Low-HD/High-PO 1.00 (0.0) 1.00 (0.0) 9.96 (2.6) 2.29 (0.8) 25.25 (21.9) High-HD/Low-PO 0.39 (0.1) 8.62 (2.1) 11.25(2.7) 2.01 (0.8) 14.79 (11.0)
Each target characters in Experiment1 were embedded in disyllabic words, and every target word was embedded in a sentence. Seventy-two sentences were generated.
Targets words were located on the range between the eleventh to seventeenth characters of a sentence; the whole sentence contained 23 to 27 characters. The entire experiment consisted of 82 sentences in total, including 72 experimental sentences, 10 practice sentences. The practice trials were not included for analysis.
Prior to the eye-tracking experiments, contextual constraints were determined by a norming studies. Contextual constrains which have typically been recognized as predictability of a word from preceding contextual information. Empirical evidence has shown that word predictability can affect both fixation location and fixation duration (Binder, Pollatsek, & Rayner, 1999). Predictability norming is usually conducted via a cloze task to obtain the predictability scores. In the cloze task, participants wrote down a word to complete sentence fragments. The predictability values of target words were the proportion of target words generated by participants.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
For the present experiment, the targets’ predictability values from the leading context
were below 0.5 and equally unpredictable to the succeeding target word.
We constructed the preceding and succeeding disambiguating context and the sentential context for seventy-two target words. The questionnaire contained seventy-two sentence fragments preceding the targets. Participants were presented with the sentence fragments and were asked to write down the next potential word, which came to mind firstly to continue the sentences fragments. Twenty undergraduate and graduate students (9 males and 11 females) aged between 18-32 years old (mean age = 24.2) were paid to participate in the cloze task. The predictability values for target words were determined by the proportion of how many the exact targets were filled in across 20 participants. The predictability values revealed that there were no significant differences across three conditions (F < 1). The rating result confirmed to the experiment design.
The experimental sentences were mixed and randomly distributed into three lists.
In each of the lists, the number of each condition was equal, precisely, 8 items in each condition. Each sentence spanned one line and was presented in the middle of the computer screen. To ensure that the participants read and comprehend the sentences, in one-third of the trials, the completion of reading sentence was followed by a comprehension question. All the comprehension questions were designed to be
‧
Table 4. Examples of sentences for the three conditions in Experiment1
Condition Target Sentence
Note. The targets were presented with bolds and italics in the example sentences.
3.1.3 Apparatus character size was 34x34 pixels. Participants were seated 70 cm away from the eyes to the screen, and the width of one character with the space before it equated about one degree of visual angle.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.1.4 Procedure
Prior to the experiment, participants were given a consent form and tested for their dominant eye. Participants were tested individually in a dimly lit and noise-attenuated room. They seated in front of the monitor with their heads in a forehead and chin rest to eliminate head movement during the experiment. At the beginning of the experiment, the instruction was given to the participants to read the sentences for comprehension without memorizing them on purpose. The five-point calibration and validation were performed in the first trial of each block (three blocks in total). After the calibration was checked, participants were asked to fixate on a cross, where located at the position of the first character of the sentence. Once they had accurately fixated on the assigned area, the cross disappeared and the sentence was presented subsequently.
Prior to reading experimental sentences, four practice sentences were presented to participants in order to be familiar with the experimental procedure. Then they read 72 experimental sentences intermixed randomly. When the participants finished reading each sentence, they had to first fixate the right most cross located below the last character of the sentence. Then, they pressed a button on the response box and the next trial began. A comprehension question was asked approximately a third of the trials. Participants answered the true-and-false questions about sentences meaning by
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
pressing either the left button 'yes' (是 shi) or the right button 'no' (否 fou). Feedback was given after they pressed the response key. The average accuracy rate was 97%.
The experimental procedure is summarized in Figure 6.
Figure 6. A diagram for the procedure of Experiment 1
3.2 Data analysis of eye movements
Data were excluded from the analyses for the following reasons: (a) there was a blink on the target or prior to target, (b) fixations were out of the ROI range, (c) the fixation was in the beginning or the end of each trial, (d) the first fixation on the target
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
was longer than 800 milliseconds (ms) or shorter than 80 ms and (e) total viewing time on the target was more than 1500 ms or less than 80 ms. The data of 7 participants whose skipping rate of target words higher than 35% were dropped and replaced with new qualified data. The average skipping rate for the targets was 0.23.
Overall, less than 1 percent of trials were removed. In the present study, the eye-movement data were analyzed using the linear mixed-effects (lme) model (Baayen, Davidson, & Bates, 2008). We evaluated the fixed effects of homophone density and P-O consistency while taking participants and items as crossed random effects by using the lmer program (lme 4 package; Bates, Maechler, & Mächler, 2009) in R 2.10.1 (R Development Core Team, 2009). For the lme models, we report regression coefficient (bs), standard error (SEs) and t value estimated from priori contrast tests.
3.3 Result
3.3.1 Target word region
The means of first pass and second pass duration time and probability measures on the target word are shown in Table 5. There were totals of 1,899, 1,615, 1,896 and
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
1,979 observations available for FFD, SFD, GD and TVT analyses, respectively. No significant homophone density effect and P-O consistency effect was found in FFD (HD: b = -0.01, SE = 0.02, t = -0.52; PO: b = 0.03, SE = 0.02, t = 1.46), SFD (HD: b = -0.01, SE = 0.02, t = -0.55; PO: b = 0.03, SE = 0.02, t = 1.49) and GD (HD: b = 0.001, SE = 0.03, t = 0.05; PO: b = 0.04, SE = 0.03, t = 1.56).
The P-O consistency effect was found in the second-pass measures: TVT (b = 0.08, SE = 0.03, t = 2.39*) and RRR (b = 0.63, SE = 0.21, z = 2.96**), with longer fixation time for P-O inconsistent words than for P-O consistent words, demonstrating the P-O consistency effect when readers encountered phonology-orthography inconsistent characters.
For the probability measures, the average skipping rate for Low-HD/High-PO, High-HD/High-PO and High-HD/Low-PO was 0.23, 0.24, and 0.21 respectively. The overall pattern of skipping rate indicated the processing difficulty according to the characteristics of the character during the normal reading process. P-O inconsistent target words were skipped relatively lower than P-O consistent characters, indicating the greater effort of readers when dealing with the P-O inconsistent characters.
Although no statistically significant effect, the skipping rates of both High-HD/High-PO and Low-HD/High-PO were numerically higher than the target words of High-HD/Low-PO (skipping rate for High-HD/High-PO: 24%; skipping rate
‧
for Low-HD/High-PO: 23%; skipping rate for High-HD/Low-PO: 21%). In addition, P-O consistency effect was significant for rereading rate (b = 0.63, SE = 0.21, z = 2.96). Greater rereading rates also suggest that readers detected some difficulties when processing characters with P-O inconsistent radicals.
Table 5. Means and standard errors (in parentheses) of fixation durations and probability measures on three types of target words in Experiment 1
Duration
There was no significant homophone density effect and P-O consistency effect in pre-target words for FFD (HD: b = -0.02, SE = 0.03, t = -0.9; PO: b = 0.04, SE = 0.03, t = 1.38), SFD (HD: b = -0.03, SE = 0.03, t = -1.03; PO: b = 0.04, SE = 0.03, t =
‧
probability measures, there was also no significant homophone density effect and P-O consistency effect in skipping rate (HD: b = 0.15, SE = 0.21, z = 0.71; PO: b = 0.28, SE = 0.21, z = 1.32), refixation rate (HD: b = -0.16, SE = 0.23, z = -0.73; PO: b =-0.004, SE = 0.23, z = -0.02) and rereading rate (HD: b = -0.13, SE = 0.22, z = -0.59;
PO: b = 0.23, SE = 0.22, z = 1.05). No effect found in the first-pass duration measures in the pre-target region indicated that any possible effect would occur when targets were fixated. The means and standard errors of duration measures and probability measures in the pre-target region are shown in Table 6.
Table 6. Means and standard errors (in parentheses) of fixation durations and probability
Table 6. Means and standard errors (in parentheses) of fixation durations and probability