Stimulus construction - 專家與生手口譯員的句子理解歷程：事件相關腦電位研究

Chapter 3 Methods

3.2 Materials

3.2.2 Stimulus construction

For this experiment, the contextual constraint of the context sentence on the target item was first examined by using the Latent Semantic Analysis (LSA) methodology. The Chinese LSA Website⁶ was used to measure the cosine of the target item and the context sentence (“term to document” method⁷). The Academia Sinica Balanced Corpus of Modern Chinese⁸ was used as the referenced semantic space.

The process of constructing a sentence pair stimulus is described as follows.

First, a context sentence and its carrier body, along with the target item, are constructed for the upper set, as illustrated in Table 2.

Table 2

Example of congruent stimuli construction in upper set Context sentence

(upper set) Carrier body Congruent

Target item

熱帶小島的天氣變化很快。

(The weather of tropical islands changes very fast.)

你下午出門的時候要記得帶你的 (When you go out in the afternoon, remember to bring your)

雨傘。

(umbrella.)

The first sentence (“The weather of tropical islands changes very fast”) is the context sentence, the second sentence (“When you go out in the afternoon, remember to bring your umbrella”) is the carrier body, while the sentence-final noun (“umbrella”) is the target item. The cosine between the target item and the entire context sentence (“term to document” method) was examined by using the “pairwise comparison”

function on the Chinese LSA Website to ensure the relatedness of the target item.

6 http://www.lsa.url.tw/modules/lsa/

7 The “term to document” method is a function in the Chinese LSA Website that calculates the cosine between a term and a sentence/paragraph. Other functions include “term to term”, in which the cosine between two terms are computed, and “document to document”, in which sentences or paragraphs can be compared against one another.

8 http://app.sinica.edu.tw/cgi-bin/kiwi/mkiwi/scorp.pl

For the example given above, the cosine between “umbrella” and the context sentence (“The weather of tropical islands changes very fast”) was 0.161.

The next step is to construct an incongruent version of the carrier sentence. The

“near neighbor” function⁹ of the Chinese LSA Website was used to generate a list of terms that were related to the target item. These terms were not necessarily

“semantically” related to the target item, but simply appeared in the list because it was contained in the Balanced Corpus. The threshold of the cosine was set at 0.01 to generate terms that were more distant from the target item. From the list, a term that had a much lower cosine than and intuitively different from the original target item would be chosen. If an appropriate term cannot be found from the returned list, a new target item would be devised independently. In the example above, 護照 (passport) was chosen as the new target item, whose cosine with 雨傘 (umbrella) is 0.022. This new term would be used to construct the incongruent version of the second sentence:

你下午出門的時候要記得帶你的護照。

(When you go out in the afternoon, remember to bring your passport.)

As seen from the example above, the same carrier body will end with the new target item and the second sentence will become incongruent when preceded by the original context sentence, as illustrated in Table 3:

9 The “near neighbor” is a function in the Chinese LSA Website that allows users to check other terms that are in a user-defined semantic space. The terms in the list generated by this function have varying cosine values to the original term and may be semantically close or distant from it.

Table 3

Example of incongruent stimuli construction in upper set Context sentence

(upper set) Carrier body Incongruent

Target item

熱帶小島的天氣變化很快。

(The weather of tropical islands changes very fast.)

你下午出門的時候要記得帶你的 (When you go out in the afternoon, remember to bring your)

護照。

(passport.)

This new incongruent carrier sentence is actually identical to its congruent counterpart except for the sentence–final target item. The cosine between the incongruent target item (“passport”) and context sentence (“The weather of tropical islands changes very fast”) was then calculated. In this instance, it was -0.001. In the case that the “term to document” cosine is high, opinions from independent raters would be sought to confirm that the stimuli would not be mistaken as congruent. This completes the construction of a sentence pair in the upper set.

The incongruent version of the carrier sentence in the upper set was then used to construct a new context sentence in the lower set. This new context sentence would render the target item of the originally incongruent carrier sentence congruent.

See Table 4 for an example:

Table 4

Example of congruent stimuli construction in lower set Context sentence

(lower set) Carrier body Congruent

Target item

我們去曼谷的班機是晚上起飛，機票

和旅行箱我都處理好了。

(Our flight to Bangkok takes off in the evening, and I’ve already taken care of the plane ticket and luggage.)

你下午出門的時候要記

The cosine between the (congruent) target item and the newly devised context sentence would be checked for relatedness. In the above example, it was 0.249.

The incongruent version of the carrier sentence that would follow this new context sentence in the lower set is the corresponding carrier sentence in the upper set:

Table 5

Example of incongruent stimuli construction in lower set Context sentence

(lower set) Carrier body Incongruent

Target item

我們去曼谷的班機是晚上起飛，機票

和旅行箱我都處理好了。

(Our flight to Bangkok takes off in the evening, and I’ve already taken care of the plane ticket and luggage.)

你下午出門的時候要記得帶你的

(When you go out in the afternoon, remember to bring your)

雨傘。

(umbrella.)

The cosine between the (incongruent) target item and the context sentence would also be calculated. In this case, it was 0.077.

In short, a context sentence and its carrier sentence containing the congruent target item were first constructed for the upper set of materials. Then an incongruent target item was determined to form the incongruent version of the carrier sentence.

This new carrier sentence, which was identical to the previous one except for the target item, served as the basis for constructing a new context sentence for the lower set of materials that would in this case render the target item congruent. This process was repeated to create 120 pairs of carrier sentences that ended with 240 unique target items and would be preceded by 240 unique context sentences. The congruent and incongruent versions of the carrier sentence that followed the same context sentence were thus constructed simultaneously, ensuring consistency and counterbalance in sentence length and word frequency across conditions. In 5% of the cases, the cosine in the incongruent version would be higher than that of the

congruent one. In this case, independent reviewers would be consulted to judge the congruity of the two versions. If their judgment proved that the cosine values were counterintuitive, the cosines would be ignored and the sentence pairs would still be included into the materials.

All stimuli were first constructed in Chinese then carefully translated into English by this researcher so as to minimize the appearance of any non-idiomatic expressions while maintaining the sentence-final position of the target items.¹⁰

3.2.3 Pre-test and cloze probability

After all congruent version of the sentence pairs (240 Chinese and 240 English ones) were constructed, the level of semantic constraint was assessed a second time with a cloze test to finalize the target items. Each congruent sentence pair was truncated just before the target item, and 35 native speakers of Chinese volunteered to complete the Chinese version of the sentences with an appropriate bi-syllabic noun that immediately came to mind, while 18 native speakers of English performed the same task on the translated English sentences by completing the English sentences with a one-word noun whenever possible. Due to the design of the materials, the same carrier sentence would appear twice in the cloze test, albeit preceded by different context sentences. Therefore the second set of sentences were given later to minimize interference with the first set.

Some sentence pairs were more problematic and produced multiple target items with a cloze probability below 0.3.¹¹ A few sentence pairs had semantically

nonequivalent Chinese and English target items, probably due to cultural differences.

For example, in the sentence pair “Some students are very aggressive and would

10 All but one English target items were single-word nouns.

11 Cloze probability of a term is calculated by counting the percentage of respondents completing the sentence with the term. Usually, a cloze probability above .67 would be considered high (Block &

Baldwin, 2010).

physically attack teachers. When this happens, in order to not get yourself into trouble, it is best to first call the ____,” the words “police” and “principal” were the

top two choices of the native English speakers, but the native Chinese speakers produced the word “parents” almost unanimously in the Chinese version. The corresponding context sentences of these problematic target items were amended to increase the level of constraint, and the new sentence pairs were given to a different group of volunteers (33 for the Chinese version and 27 for the English version) to perform the cloze test. Native speakers of Chinese who were English majors in college were allowed to take part in the cloze test for the English version of the sentences in this round. After the second round of cloze test, stimuli that still had low cloze probability or were semantically nonequivalent across Chinese and English were eliminated from the materials. In the end, 100 sentence pairs remained in each set, forming a total of 200 balanced sentence pairs.

The cloze probability of the finalized target items confirmed the robustness of the LSA method to determine semantic relatedness. Mean cloze probability was high; 0.82 (SD = .15) for Chinese target items and 0.78 (SD = .17) for English ones.

The finalized target items in the lower set served as the incongruent endings of the sentence pairs in the upper set, while the finalized target items in the upper set served as the incongruent endings of the sentence pairs in the lower set. The incongruent target items were never identified as possible endings to the sentence pairs by cloze test participants. In addition, the target items were all unique and never appeared in the context and carrier sentences. On average, the cosines of congruent stimuli (M

= .147, SE = .008) were higher than incongruent stimuli (M = .083, SE = .006). This difference was significant between the two conditions t (378) = 6.58, p < .05. This statistical result again demonstrates the power of using LSA to confirm the level of congruity between target and context.

Also, the finalized target items were limited to nouns. Efforts were made to identify two-word (bi-syllabic) Chinese terms as target items whenever possible. In the final set of the material, there were 14 tri-syllabic and one four-syllabic terms in the upper set, while there were 8 tri-syllabic and 2 four-syllabic terms in the lower set.

The frequency of the targets in the upper and lower sets was well controlled: there was no significant difference in frequency between the upper and lower sets (Upper set: M = 228, SE = 410.03; Lower set: M = 270, SE = 675.19; t (198) = -.54, p = .591.) The Word Frequency Sinica Corpus¹² was used to examine the frequency of the target terms. The frequency of English target items were not assessed as they need to be semantic equivalents of the Chinese target items.

3.3 List generation

Table 6 below illustrates the arrangement of experiment materials.

Table 6

Arrangement of context and carrier sentences: A-H = Chinese carriers; M-T

=English carriers (translations);a-h = Chinese targets; m-t = English targets.

Context sentence

Carrier body Target nouns

Chinese English Chinese Cong.

12 http://elearning.ling.sinica.edu.tw/CWordfreq.html

The experiment materials were counterbalanced and divided into eight lists, each containing 25 sentence pairs of the four experiment conditions: non-switched congruent (Chinese context, Chinese carrier body, Chinese congruent target), non-switched incongruent (Chinese context, Chinese carrier body, Chinese incongruent target), switched congruent (Chinese context, English carrier body, English congruent target), and switched incongruent (Chinese context, English carrier body, English incongruent target). Each list was designed so that subjects would only hear each context sentence, carrier body, and target once. Table 7 illustrates the arrangement of experiment lists.

Table 7

Arrangement of experiment list: carrier sentences and targets. A-H = Chinese carriers; M-T =English carriers; a-h = Chinese targets; m-t = English targets.

List Non-switched Congruent

Non-switched Incongruent

Switched Congruent

Switched Incongruent

1 A+a B+f O+o P+t

2 B+b C+g P+p Q+m

3 C+c D+h Q+q R+n

4 D+d E+a R+r S+o

5 E+e F+b S+s T+p

6 F+f G+c T+t M+q

7 G+g H+d M+m N+r

8 H+h A+e N+n O+s

3.4 Recording

All materials were recorded with normal speaking rate and intonation by the experimenter. Context and carrier sentences were first recorded separately to ensure consistency across conditions. The carrier bodies were first read together with the sentence-final target items to ensure that the prosody remains natural and then the target items were truncated from the carrier bodies and re-recorded separately. All

materials were then edited to trim off periods of silence before the onset and after the offset of stimuli. The context and carrier bodies were then strung together to form a single audio file while the target item remained independent. The English context sentences (used in previous cloze test) were not recorded, since the experiment only examined the language switch condition from Chinese (context sentence) to English (carrier body and target item). Recording and editing were done by using an audio editor and recorder software Audacity (http://audacity.sourceforge.net).

Since all context sentences (in Chinese), Chinese carrier bodies, and Chinese target items appeared in both the non-switched congruent and incongruent conditions, the time length was perfectly matched between the two conditions. Context and carrier bodies combined ranged from 8.774 to 9.304 seconds and had an average of 9.039 seconds (SD = .135). Targets ranged from 0.697 to 0.731 seconds and had an average of 0.714 seconds (SD = .009). The stimuli (context and carrier body plus target) ranged from 9.486 to 10.019 seconds and had an average of 9.752 seconds (SD

= .136).

By the same token, all context sentences (in Chinese), English carrier bodies, and English target items would appear in both the congruent and incongruent conditions of the switched version of the experiment, so the time length was perfectly matched between conditions. Context and carrier bodies combined ranged from 8.882 to 9.412 seconds and had an average of 9.147 seconds (SD = .135). Targets ranged from 0.659 to 0.693 seconds and had an average of 0.676 seconds (SD = .009). The stimuli (context and carrier body plus target) ranged from 9.557 to 10.09 seconds and had an average of 9.823 seconds (SD = .136).

The design of the materials allowed a perfect match between congruent and incongruent versions of the sentence pairs within each language setting. All carrier bodies and all target items would appear twice in the materials, once as the congruent

version and once as the incongruent version.

There was a significant difference in the time length of English and Chinese target items, F (3, 796) = 6.19, p < .001, but when attached to their corresponding context and carrier bodies, the time length of the entire stimuli did not differ significantly between switched and non-switched conditions, F (3, 796) = .09, p

= .965.

For the 8 lists, the total length ranged from 969.16 to 991.9 seconds (M = 978.78,

SD = 7.44), with total sentence length (context plus carrier body, without target)

ranging from 901.21 to 922.42 seconds (M = 909.3, SD = 6.99) and total target length ranged from 67.95 to 70.94 seconds (M = 69.49, SD = 1.02). No significant length differences were found between the eight lists in terms of the total length [F (7, 792)

= .15, ns], the context plus carrier body length [F (7, 792) = .13, ns], and the target length [F (7, 792) = .67, ns].

The table below illustrates the length of each list.

Table 8

Length of experiment lists (in seconds).

List Total context length Total target length Total length of each list

1 902.30 68.51 970.81

2 922.42 69.48 991.90

3 912.59 69.83 982.42

4 909.95 70.94 980.89

5 909.57 70.66 980.23

6 901.21 67.95 969.16

7 912.73 68.94 981.67

8 903.61 69.57 973.18

Average 909.30 69.49 978.78

Note. Context refers to the first Chinese carrier sentence plus the second Chinese or

English carrier body.

3.5 Procedure

The single-session experiment was conducted in a sound-attenuated and dimly-lit room. Subjects were instructed to sit in front of a computer screen about 90 cm away. A response device with push buttons was placed in front of the subjects. A silver cross would be displayed on the center of the black screen 100 milliseconds prior to the presentation of the audio stimuli to signal the beginning of a trial. Then, the entire stimuli, sentences and target items combined, would be presented aurally through computer speakers placed at a distance from the subjects to avoid electrical interference. Subjects were instructed to pay attention to the stimuli and decide whether they made sense or not after listening to the sentences by pressing one of two designated buttons on the response device with their index fingers. The buttons assigned to indicate congruency were counterbalanced so that half of the subjects used their left index finger to indicate “congruent” and their right index finger to indicate

“incongruent”, while the other half used their right index finger to indicate “congruent”

and their left index finger to indicate “incongruent”. The silver cross remained on the screen until subjects have indicated their responses. There was no time limit for the button press. Once the self-paced responses were made, there would be a 3-second inter-trial interval before the next trial began. Subjects were instructed to fixate their eyes on the silver cross at the center of the screen and refrain from blinking and other movements, especially starting from the middle part of the stimuli. They were encouraged to blink after they have pushed buttons and before the next trial started.

Figure 1 illustrates the procedure of stimuli presentation.

The eight lists of stimuli were rotated across each group of subjects. Within each list, stimuli from the four experiment conditions were presented randomly so that subjects could not predict which type of stimulus would appear. Before the formal experiment, subjects were given 8 practice trials, two from each experiment condition,

to acquaint them with the procedure, volume, and task. They could ask questions after the practice session and were provided feedback about their blinking behavior.

During the experiment, a break was given every 35 trials. Subjects could press a button to resume the experiment when they were ready. Excluding instruction time, practice trials, and breaks, the experiment proper lasted around twenty minutes.

Figure 1. Procedure of stimuli presentation. A silver cross would be displayed on

the center of a black screen 100 milliseconds prior to the presentation of the audio stimuli to signal the beginning of a trial. After hearing the stimuli, subjects indicated their judgment of the stimuli’s logicality by pressing on a response box, after which the cross would disappear. Once the self-paced responses were made, there would be a 3-sec inter-trial interval before the next trial began.

3.6 Behavior and EEG recording

The E-Prime 2.0 software from Psychology Software Tools, Inc. was used under the Windows XP operation system to present all stimuli, record subjects’ behavioral

data (response time and accuracy), and send event codes to trigger acquisition of EEG signals on a separate computer. Three events were marked: onset of context and carrier bodies, onset of target item, and subjects’ response.

EEG was recorded with 30 tin electrodes sewed onto an elastic electrode cap.

The electrode positions were based on the international 10-20 system with seven at the anterior sites (FP1, FP2, F7, F3, FZ, F4, F8), nine at the central sites (FC3, FCZ, FC4, C3, CZ, C4, CP3, CPZ, CP4), six at the posterior sites (P3, PZ, P4, O1, OZ, O2), and eight at temporal sites (FT7, FT8, T3, T4, TP7, TP8, T5, T6). Two reference points were placed on the mastoids on both sides. Electro-oculogram (EOG) was recorded and monitored by placing two electrodes at the corner of the left and right eyes (HEOG) and two others above and below the left eye (VEOG). Electrode impedance was kept at or below 5 kΩ. The signals were amplified using the NuAmps system from NeuroScan Inc. with a sampling rate of 1000Hz and an

amplifier rate (gain) of 19, which corresponds to an input range of +/-131.5mV. The online low pass filter was 100Hz and high pass filter was set as DC recording.

3.7 Data Analysis

Raw ERP data was processed with NeuroScan Inc.’s software EDIT 4.5. A linear derivation file was first imported to convert the four mono-polar channels (VEOU, VEOL, HEOR, HEOL) recording eye movement to two bipolar channels (VEOG and HEOG). Ocular movement was reduced by setting the trigger as positive and the threshold as 10%. Data from a few subjects required the trigger to be set as negative. Blink value was set at 20 sweeps and 400-ms duration, and

在文檔中專家與生手口譯員的句子理解歷程：事件相關腦電位研究 (頁 38-0)