The theme of the current research was using semantic similarity measures to evaluate congruency and analyzing congruency effects on L2 collocation learning.
The overall research design involved four sub-studies to address four research questions, as shown in Figure 3.1.1. The first three sub-studies conducted quantitative analysis to examine applicability of semantic similarity measures, to evaluate congruency classification, and to investigate congruency effects on collocation learning. The fourth sub-study surveyed and elicited factors of learners’ collocational priming with both quantitative and qualitative analyses.
The operational research framework, as depicted in Figure 3.1.2, was composed of research components with investigative roles and relations. The participants provided attributes of proficiency levels, collocation test data, questionnaire survey and think aloud data. Lexical similarity measure was used as a research instrument, whose utility was examined by the first sub-study to address the first research question. The collocation test was the core of study and was also a research instrument to gauge learners’ performance. The application of similarity measure to collocation led to objective examination of the notion of congruency, which was the focus of the second sub-study and addressed the second research question. Learners’
performance on collocation use was analyzed in the third sub-study to address the third research question, with learners’ proficiency level and collocational congruency as two independent variables. Finally, questionnaire and think aloud method were also research instruments with data from participants and test performance to address the fourth research question in the fourth sub-study.
To examine applicability of similarity measure
Figure 3.1.1 Overall Research Design
Quantitative Studies Qualitative Study
To evaluate congruency classification
To investigate congruency effects on collocation learning
To survey and elicit factors of collocational priming
Using Lexical Semantic Similarity Measures for Analysis of Congruency Effects on L2 Collocation Learning
Figure 3.1.2 Operational Research Framework
The first sub-study involved verifying the applicability of semantic similarity measures. Based on literature review, it was found that WordNet (Miller, 1995) incorporated eight computational algorithms of semantic similarity measures and provided convenient online use. Of the eight algorithms, two (Adapted Lesk and Gloss
Vectors) were selected in terms of their computational features and measuring
stability. Three sets of word pairs with different semantic relations were composed and tested for lexical similarity values by the two measures. The distinguishability of similarity values between different semantic relations was examined so as to establish the use of semantic similarity measure as a research instrument.The second sub-study applied the two semantic similarity measures to the experimental set of collocation so as to objectively evaluate the properties of congruency. Semantic similarity between a collocate and a transferred word from L1 counterpart was quantified by the two computational semantic similarity measures.
Distribution of semantic similarity values was observed and analyzed for congruency classification. Congruency derived from lexical semantic similarity was cross-examined with congruency based on human judgment. Statistical and analytical
Learner Performance Questionnaire
&Think Aloud
Participants Collocation Similarity Measure
Congruency Proficiency Level
RQ1
RQ2
RQ4 RQ3
comparisons were made, which led to further understanding of the potential advantage of exploiting semantic similarity for congruency evaluation. The quantitative results and analytic derivation addressed the second research question.
In the third sub-study, it was hypothesized that L2 collocations with semantic components disparate to that of corresponding L1 counterparts would be more challenging to L2 learners. In other words, incongruent collocations would be error-prone to L2 learners. It was also conjectured that learners’ proficiency levels played a role in the interaction of congruency and performance. The objective of the experiment was to find the correlation between two independent variables, congruency and proficiency level, and one dependent variable, L2 collocation performance. The experimental results and the analytic derivation addressed the third research questions and provided comparison to previous research findings in the literature. For example, Yamashita and Jiang’s (2010) indicated that L1 and L2 congruency and L2 exposure influenced L2 collocational acquisition. They also suggested that it was more difficult to learn incongruent L2 collocations, but once stored in their lexical network, they were processed independently of L1 linguistic competence. However, the current study does not tackle the issue about whether L2 collocational knowledge develops in parallel with vocabulary knowledge. This aspect is beyond the research scope.
In the fourth sub-study, a questionnaire survey was conducted to collect learners’
conceptions about collocational congruency processing. L2 learners’ conceptions towards L1 and L2 congruency processing could reveal more in-depth thoughts about how they processed congruent and incongruent collocations. It was conjectured that learners’ conceptionss acted on lexical semantic processing and the processing activated collocational priming and production. As such, an investigation on learners’
conceptions on L2 collocation processing could gain a fuller understanding of under
what conditions congruency is a facilitator or a hindrance in L2 collocation production.
After the questionnaire survey, the think-aloud protocol was compiled to get insights into the learners’ dynamic thinking process within certain context of engagement. The fourth sub-study was beneficial to explore how L2 learners constructed collocational meanings. The protocol also elicited learners’ introspective thoughts on L2 collocational congruency processing, which led to potential induction of learning factors. Such a process-oriented study design was well-suited to collect narrative accounts. Based on this orientation, the present study adopted the think-aloud method to look into sophisticated performance of language learners’
thinking. The contents of think-aloud represented explicit linguistic knowledge and think-aloud could be used as a complement after language tests to further elicit more explicit knowledge. Also, it was useful to identify some general trends and patterns of the processing. With questionnaire responses and think-aloud transcribed data, both quantitative and qualitative analysis was conducted for comparison and interpretation, which addressed the fourth research question.