The instruments included two semantic similarity measures, a set of collocation test, a set of questionnaire, and the think-aloud protocol. By the operational definition, a collocation is formed by a collocate and a base. Given a pair of equivalent L2 and L1 collocations, the subject of study is usually the semantic relation between the pair of cross-linguistic collocates. However, currently all semantic similarity measures were designed to operate on word pairs of the same language. To evaluate the semantic relation between the pair of cross-linguistic collocates by semantic similarity measures, an L2 transferred word of the L1 collocate was used as a surrogate that embedded the word sense of the L1 collocate. As a design feature, semantic similarity measures also allowed semantic similarity evaluation between word pairs in both contexts of all word senses or designated word senses. When operated in all word senses, semantic similarity measures computed all possible combinations of word senses and gave the highest value that reflected the most similar senses of the two words. Alternatively, when a particular sense of each word was selected, semantic similarity measures provided similarity values of the two designated word senses.
As a convenient and useful semantic search instrument, WordNet Search-3.1 was employed to consult for word senses in glossary. As shown in Figure 3.3.1, the online system of WordNet Search-3.1 (Miller, 1995) was different from other online dictionaries because it showed not only lexical meanings and part of speech, but also its synset relation and word relation. For the purpose of this study, WordNet
Search-3.1 provided word sense observation and selection for both L2 collocates and
L2 transferred words as surrogates of L1 collocates.Figure 3.3.2 Online Use of Semantic Similarity Measures Figure 3.3.1 Semantic Search: Word Sense Selection
The use of the two semantic similarity measures, Adapted Lesk and Gloss Vectors, as a research instrument was operationalized with the online service of
WordNet::Similarity, as shown in Figure 3.3.2. In fact, WordNet::Similarity
conveniently integrated the online service of WordNet Search-3.1 (Miller, 1995) with hyperlinks and provided semantic similarity calculation by a straightforward process of data input and results output. The process of calculating and retrieving lexical similarity values was discussed in the later section of data collocation procedure.The second instrument was a contextualized collocation test to evaluate collocation production. Forty items were selected according to collocation properties such as restriction, semantic transparency or opacity, and L1-L2 congruency.
Semantic congruency was one of the primary properties, indicating whether or not component words could be replaced with L2 near-synonyms transferred from L1 word sense. In this study, L1 influence was considered as one of the most crucial factors in collocation processing. The influence of L1 transferred meaning caused by non-L1 equivalence has been recognized by many researchers (Bahns & Eldaw, 1993;
Caroli, 1998; Granger, 1998; Nesselhauf, 2003; Murao, 2004; Shehata, 2008). This study took L1 transfer into account to examine congruency relations and their influences in learners’ performance. The selection criteria for collocation candidates included collocational types, length, and property of restrictedness. The types were limited to two syntactic part-of-speech structures, including verb-noun (VN) and
adjective-noun (AdjN). The other properties of collocational candidates were
semi-restricted and at the span of two words, i.e., the binary type.According to the criteria, a set of collocation candidates were extracted from the collocation lists of previous studies on common miscollocations. Lin (2010) inquired that Taiwanese EFL learners commonly misused verb-noun collocations and provided a list of 210 miscollocations. Another study conducted by Chen (2009) provided a
collocation list of the adjective-noun combination. Three online retrieval tools, including WebCollocate (Chen, 2011), TANGO (Jian, Chang, & Chang, 2004) and
ozdic.com (Oxford Dictionary Online, McIntosh, Francis & Poole, 2009), were
consulted to ensure correctness of the collocation candidates. Table 3.3.1 summarizes the design of collocation categories and the allocation of collocation items. In addition, word frequency was also used as a selection criterion. Only collocations with high frequency words were considered and the candidate list of collocation items were also selected to reach a balance of difficulty.Table 3.3.1 Category Design and Allocation of Collocation Items
Collocation Type Restriction Length Congruent Incongruent
verb-noun semi- binary 10 10
adj.-noun semi- binary 10 10
The final set of collocation test included four categories of collocation items, congruent verb-noun collocations, incongruent verb-noun collocations, congruent adjective-noun collocations, and incongruent adjective-noun collocations. Each category consisted of ten collocation items, as shown in Table 3.3.2, with given bases and expected collocates. The collocation test was conducted in limited class hours to evaluate productive knowledge of L2 collocations. All of the collocation items were embedded in contextual sentences where the equivalent L1 collocations were given in Chinese, as given in the appendix. The test required participants to complete a blank-filling task by providing a collocate (verb or adjective) with the given base (noun) in a contextual sentence. By filling in a blank word in a contextual sentence with the specified meaning of an L1 collocation, the collocation test attempted to evaluate participants’ collocation competence. The graded results of the collocation test were used as an indication of learners’ collocation performances in the context of
the present study’ research design.
Table 3.3.2 Experimental Set of Collocation Items in Four Categories Collocatio
acquire
knowledgesurf
Internetseek
informationsolve
crimemake
effortmake
apologysee/watch
(the) playstudy
Englishimprove
abilitycarry
lanternmaintain
relationshipease/lessen
worrypreserve
culturemake
conclusionmake
troubleconduct
heattake
actionsmake
impressionovercome
challengerestore
vitalityAdjective-Noun pair
full
speedheavy
drinkerhigh
technologysound
dietinside
informationnarrow
victorymental
ageregular
customerThe objective was to gain insights of learners’ cognition towards collocational congruency. With 15 evaluative statements, the questionnaire used a five-point Likert scale to construe participants’ thoughts in the collocation processing and indicate their level of agreement (ranging from five for strongly agree to one for strongly disagree).
Of the 15 questionnaire items, one was purposely formulated negatively to force students to evaluate every statement carefully. The four foci of the questionnaire were learners’ cognitive views on collocational learning, collocational composition, congruency effects and L1 influence. All of the questionnaire items pivoted on two aspects, cognition and congruency processing.
The collocational use questionnaire (CUQ) was administrated to all of the participants during the class time after the collocation test. The first part of CUQ gathered participants’ demographic information. Further on, the statistical analysis of
Cronbach’s alpha and KMO & Bartlett’s Test (Ratray & Jones, 2007) were adopted to
inspect reliability and validity of the CUQ survey.The last instrument was a think-aloud protocol. Documented for qualitative data collection, the protocol was used to elicit factors and patterns of congruency processing. Prior to compiling the protocol, it was essential to carefully take trustworthiness of qualitative studies into account. The crux of a well-established qualitative study lies in its trustworthiness of data interpretation. In this regard, Lincoln & Guba (1985, p. 301) posed the criteria for evaluation of trustworthiness, including truth value, applicability, consistency, and neutrality. Further, Creswell &
Miller (1997) synthesized alternative verification methods in the literature and proposed eight verification procedures, including (i) prolonged engagement and persistent observation, (ii) triangulation, (iii) peer debriefing, (iv) negative case analysis, (v) clarifying researcher bias, (vi) member checks, (vii) thick description, and (viii) external audits. Creswell (1998) put forth that qualitative studies should conduct at least three out of the eight verification procedures to ensure trustworthiness of data analysis. In terms of the procedures, Li (2004) suggested the following procedures as voluntary participation, anonymity assurance, purposeful sampling, (data or method) triangulation, prolonged engagement and (near-) natural situation for
data collection; peer debriefing, stepwise replication, inter-coder reliability and member checks for data analysis; and thick description, application refrained from generalizing for interpretation and reporting. The current study adopted four procedures to conduct the think-aloud protocol, including voluntary participation, anonymity assurance, purposeful sampling, and peer debriefing.
Seven participants from each group were selected in terms of the questionnaire data to further undertake the think-aloud process. At the very beginning, a pilot procedure of think-aloud was administrated to get seven participants familiar with uniform verbalization instruction. In addition, the procedure subsumed a warm-up task and questions to clarify any doubts in the think-aloud context. The researcher periodically reminded participants to continue verbalizing their thoughts. All of the observation and qualitative data were collected in the protocol. Verbalized ideas and further explanations of the excerpts could reveal whether or not they applied L1 conceptual links and lexical network to determine a felicitous L2 collocate and how they tackled L2 collocational problems.