A total of four sub-studies were designed to apply semantic similarity measures and evaluate collocational congruency for cross-linguistic influences in a more objective and systematic way. In both the first and the second sub-studies, the online system of WordNet::Similarity was used to calculate and retrieve values of semantic similarity between selected word pairs. The first sub-study was intended to verify the applicability of semantic similarity measures in distinguishing semantic relations.
Semantic similarity values of word pairs from three sets of different semantic relations were calculated in the context of all word senses and retrieved for analytic comparison.
In the second sub-study, the evaluative focus was on the semantic similarity between an L2 collocate and a transferred word from the L1counterpart. Given a pair of semantically equivalent L2 and L1 collocations, the data collocation procedure, as depicted in Figure 3.4.1, started at selecting an L2 transferred word for surrogating the L1 collocate and paired it (as Word2) with the L2 collocate (as Word1) for semantic similarity evaluation. Next, word senses of both words were searched and observed with the online service of WordNet Search-3.1. The design of similarity measures involved two contexts of semantic similarity evaluation. The context of all word senses in deriving similarity values simulated learners’ full recognition on word meaning, while the context of single word sense represented learners’ limited recognition. Using the two measures of Adapted Lesk and Gloss Vectors, similarity values of Word1 and Word2 were derived in both contexts of all word senses and single word sense for a complete analysis.
Employing the data collection procedure on the WordNet::Similarity system involved the following steps.
In the context of all word senses:
1. Key-in the L2 collocate in the Word1 slot with the format word#part_of_speech, for example, observe#v.
2. Key-in the L2 transferred word from the L1 counterpart in the Word2 slot with the format word#part_of_speech, for example, celebrate#v.
3. Select one of the embedded measures, for example, Gloss Vectors, with a pull-down menu to calculate the semantic similarity of input words in Word1 and Word2.
4. Press the “Compute” button.
5. Receive the results, e.g, “the relatedness of observe#v#6 and celebrate#v#1 using vector is 1”. This showed that, among all word sense combination, word sense #6 of observe had the highest similarity to word sense #1 of celebrate, rated as 1 by the (Gloss) vector measure.
In the context of single word sense: (with the results from the all word sense context)
1. Click on the “View glosses (definitions)” link.
2. Inspect all word senses of the two words and determine a particular word sense for each word.
3. Key-in the L2 collocate in the Word1 slot with the format word#part_of_speech#sense, for example, observe#v#7 (follow with the eyes or the mind).
4. Key-in the L2 transferred word from the L1 counterpart in the Word2 slot with the format word#part_of_speech#sense, for example, celebrate#v#1 (behave as expected during of holidays or rites).
5. Select one of the embedded measures, for example, Gloss Vectors, with a pull-down menu to calculate the semantic similarity of input words in Word1 and Word2.
6. Press the “Compute” button.
7. Receive the results, e.g, “the relatedness of observe#v#7 and celebrate#v#1 using vector is 0.1822”. This showed that, for this specific word sense combination, the semantic similarity between observe and celebrate was rated as 0.1822 by the (Gloss) vector measure.
Collocate Base
Measures: LESK Vectors Full Recognition Value 1 Value 2 Limited Recognition Value 3 Value 4
Word 2:
L2 Transferred Word
Word 1:L2 Collocate
Search Word Senses of Word1 & Word2
Full Recognition All Word Senses of
Word1 & Word2
Limited Recognition Single Word Sense of
Word1 & Word2
WordNet Similarity Measures
Quantify Semantic Similarity of Word1 & Word2
Figure 3.4.1 Data Collection Procedure of Semantic Similarity Values
The objective of the third sub-study was to explore to what extent collocational congruency influenced learners’ performance in the collocation test. All of them took the collocation test which covered both categories of congruence and incongruence.
The test results were graded on the item level. Each correct answer of an L2 collocate in a blank-filling test item was rated as one point. Some alternative L2 collocates were also accepted as correct and received full points, such as “acquire (or gain)
knowledge” and “heavy (or pouring) rain”. With the graded test scores, multiple
dimensions of test results, such as collocation item, collocation category, participant group, can be aggregated for analytic comparison.The last sub-study was designed to collect learners’ responses to collocation use via the questionnaire. All of the participants completed the questionnaire during in-class hours. It was of significance for this study to understand how the learners approached the collocation test items. After evaluation of the responses, seven participants in each group were chosen to undertake think-aloud in out-of-class meetings. During the think-aloud task, the selected participants met individually with the researcher. The think-aloud protocol and provision of certain training sessions were necessary for the participants because most of them had no experience of verbalizing their thoughts (Chamot, Barnhardt, El-Dinary & Robbins, 1996; Bernard, 2000). The training session began with the researcher’s explanation of the think-aloud techniques. Later, the participants were informed of the purpose of the study, the data collection process, and approximate time required for think-aloud.
After training, the participants listened to a short recorded demonstration produced by the researcher. They then got a different version of the collocation test and practiced verbalizing their thoughts while completing the test. A black dot was marked under each collocation to remind the students to stop and think aloud. The participants could use English or Chinese to express themselves in the think-aloud
task. Meanwhile, the researcher could pose questions to get additional information.
All the think-aloud procedures and their processing tasks were audio-taped and kept in written notes for think-aloud protocols. Through a questionnaire and think-aloud, L2 learners’ patterns on tackling congruent and incongruent collocations could be elicited. Instances of collocation errors were used to discuss the participants’
difficulties, while successful instances of collocation processing were used to describe whether they transferred L1 word senses to L2 collocation use and how they approached collocation links.