This chapter mainly presents both the quantitative and qualitative results to answer the two research questions of the current study. The data of the three measurements were described, compared, and analyzed. Besides, the responses in the questionnaire were also analyzed to further investigate the participants’ perceptions of the assigned referencing tools on collocation learning.
Results of the Tests and the Task
To answer the first research question: “Which group performs better in the collocation tests? The group with online general-purpose dictionary, the group with
disc-version of collocation dictionary, or the one with the collocation retrieval tool?”,
the results of the pretest, the treatment task, and the posttest were compared by descriptive mean.
Results of the pretest.
As presented in Table 4, the mean scores of the three groups were: 9.79 (OCDE), 10.29 (LDCE), and 11.00 (WEBC) respectively. All three groups of participants were able to answer about one-third of questions in the pretest. Then One-Way ANOVA was applied and the result showed no significant difference among groups ( F (2,101)
= .792, p = .456 > .05). It could be inferred that, before the experiment, the collocation knowledge the three groups had were similar to one another.
79
Table 4 The Mean Scores of Each Group in Pretest, Treatment Task, and Posttest
Group Pretest Treatment Task Posttest
M SD M SD M SD
OCDE 9.79 3.370 19.85 3.993 11.30 5.163
LDCE 10.29 3.836 25.07 3.958 12.73 6.091
WEBC 11.00 4.267 21.23 5.532 14.47 4.769
Results of the treatment task.
The scores of the treatment task were compared to analyze which of the
referencing tools assisted the participants better in searching the proper collocates. In Table 4, the mean scores were: 19.85 (OCDE), 25.07 (LDCE), and 21.23 (WEBC) respectively. Compared to their performance in the pretest, all the three groups have made improvement in the treatment task. And the LDCE group’s performance, especially, was found significantly more outstanding than those of the other two groups (F(2, 101) =13.664, p < .001).
Results of the posttest.
As to the posttest, which was conducted to investigate how many collocations have been learned after the use of referencing tools, the results of the three groups all regressed when compared to those in the treatment task, but improved when
compared to those in the pretest. Based on Table 4, the mean scores of the posttest were—11.30 (OCDE), 12.73(LDCE), and 14.47 (WEBC) respectively. It could be inferred that all the three groups did make collocation gains after using the
referencing tools. However, no significant difference was found between groups since
80
the p-value was slightly higher than the 0.05 level ( F(2, 101) = 2.653, p = 0.075 >
0.05).
The comparison of the scores for the tests and the treatment task.
In addition to the analysis of the mean scores in each test and task, the researcher further investigated if the collocation assistances the referencing tools provided were significant in the treatment task among each group. With the application of
paired-sample t-test, the participants’ performances in the pretest, treatment task, and posttest were compared (see Table 5).
Table 5The Score gains of Each Group in the Pretest, Treatment Task, and Posttest
Group Pair Mean SD SEM t df Sig.
Note. PreT=Pretest; TT=Treatment task; PT=Posttest. *.The mean difference is significant at the 0.05 level.
The results listed above showed that the three referencing tools did provide significant collocation assistances for each group in both the treatment task and the posttest. In the treatment task, the LDCE group improved the most, 14.780, nearly half of the total score, while the OCDE and the WEBC group gained 10.061 and 10.233 respectively (OCDE: p <.001; LDCE: p <.001; WEBC: p <.001). As to the retention of collocations, the WEBC improved the most, with a 3.467 score gain,
81
while the OCDE had 1.515, the LDCE 2.439 respectively (OCDE: p = .025; LDCE: p
= .001; WEBC: p <.001).
Based on the previous statistics shown, the three referencing tools indeed
significantly facilitated the participants’ collocation learning in both the treatment task and posttest. The LDCE group outperformed the other two groups significantly in the treatment task—not only scored the highest but also improved the most. On the other hand, it was the WEBC group that improved the most when the scores of pretest and posttest were compared (see Table 6).
Table 6 The Comparison of the Performances Among Groups Pretest Treatment
The differences in the correct rates of each item among the groups.
After analyzing the overall results of the pretest, the treatment task, and the posttest, the researcher further explored the effectiveness of the three tools in helping the participants retrieve appropriate answers to each of the 30 questions. The three groups’ rates of correct answers to each item in the treatment task were computed by Scheffe. The significant differences were found in the correct rates of the following 18 items: Q2, 4, 6, 7, 9, 10, 12, 13, 17, 19, 20, 21, 22, 24, 25, 26, 28, and 29. Among the
82
18 items, whose results were found significantly different between groups, the LDCE group performed the best in eight items (Q4, 6, 7, 9, 20, 25, 26, 28), while the OCDE group turned out to be the most inferior in six items (Q2, 10, 12, 17, 22, 24) (See Table 7).
Table 7 The Comparison of the Performance of Each Item Among Groups Referencing
tool
Performance Items Results
LDCE > OCDE & WEBC 4, 6, 9, 28 win 8 items
> OCDE 20 25, 26
> WEBC 7, 29
WEBC > OCDE 19, 21 win 2 items;
lose 1 item
< OCDE & LDCE 13
OCDE < LDCE & WEBC 2, 10, 12, 17, 22, 24 lose 6 items
Results of the Questionnaire
After the responses of the questionnaire were collected, the participants’
perceptions towards the referencing tools were investigated to answer the second research question—What are the students’ perceptions of the referencing tools? What are the strengths and weaknesses of these referencing tools from their points of view?
Results of the perceptions.
Immediately after the treatment task, a questionnaire was distributed to the participants. It included 11 items: the first five items aimed to probe into the
participants’ background knowledge of collocation and learning behavior before the experiment; the next five items intended to examine the participants’ perceptions
83
towards the application of the referencing tools after the experiment. The last item was an open-ended question to investigate the participants’ perceptions on the
strengths and weaknesses of the referencing tools. The following is the percentage of agreement and disagreement of the 10 items in the questionnaire (see Table 8).
84
Table 8 The Percentage of the 10 Items in the Questionnaire Q2 When learning an English word, I pay attention to both the
word and its collocates.
27 82 6 18 35 85 6 15 25 83 5 17 87 84 17 16
Q3 When learning an English word, I memorize the word and its collocation, instead of just a single word.
21 64 12 36 32 78 9 22 21 70 9 30 74 71 30 29
Q4 While reading, I pay attention to the phrases and collocations in the text.
25 76 8 24 37 90 4 10 27 90 3 10 89 86 15 14
Q5 When encountering problems of word usages, I turn to online collocation referencing tools.
17 52 16 48 26 63 15 37 19 63 11 37 62 60 42 40
Q6 I think OCDE/LDCE/WEBC can help me learn collocations. 29 88 4 12 35 85 6 15 22 73 8 27 86 83 18 17 Q7 Learning collocations with the help of OCDE/LDCE/WEBC
can enhance my ability to memorize vocabulary.
25 76 8 24 30 73 11 27 20 67 10 33 75 72 29 28
Q8 Learning collocations with the help of OCDE/LDCE/WEBC can enhance my ability to use the words.
29 88 4 12 34 83 7 17 23 77 7 23 86 86 18 17
Q9 The interface of OCDE/LDCE/WEBC is user-friendly. I can easily find the collocates I need.
19 58 14 42 21 51 20 49 16 53 14 47 56 54 48 46
Q10 If possible, I would like to use OCDE/LDCE/WEBC to help me learn other types of collocation, i.e. ADJ+N or ADV+ADJ.
28 85 5 18 31 76 10 24 25 83 5 17 84 81 20 19
85
When being asked about their background knowledge of collocation and learning behavior before the experiment, most of the participants contended that they know what the term collocation means, and that they pay attention to both the word and its collocates when learning new words or during reading. Most of them also maintained that usually they memorize the word and its collocation, instead of just a single word when learning an English word. Over 60% of them agreed that they turn to online collocation referencing tools when encountering problems of word usages. No significance was found in these first five questions. That is, the participants’
collocation background knowledge and learning behaviors before the experiment were in no significant difference.
After using the collocation referencing tools in the treatment task, most of the participants claimed that the referencing tools can not only help them learn collocations but also enhance their ability to memorize vocabulary and to use the words. If possible, they’d like to use the assigned referencing tools to learn other types of collocation, i.e. ADJ+N or ADV+ADJ. However, in item 9, merely half of the participants agreed that the interface of the assigned referencing tool is user-friendly and that they can find the collocates they need easily. Only one significant difference between groups was found in item 6—I think OCDE/LDCE/WEBC can help me learn collocations (p = 0.006 < 0.05).
86
To further investigate the significance between groups in the questionnaire, the average score of each item was calculated (see Table 9). It was found that when being asked about the perceptions of using the assigned referencing tool, the participants in the OCDE group, despite their poor performance in the treatment task, still held a positive attitude toward the tool. Their mean scores of item 6 to item 10 were all higher than the overall average scores.
In item 6, where the significant difference was found between groups, as being asked if they consider the assigned referencing tool helpful in collocation learning, the OCDE group showed the strongest agreement by scoring the highest (Mean=3.33), followed by the LDCE group (Mean=3.15). Among the three groups, it was the WEBC group that held a rather neutral attitude toward the helpfulness of the referencing tool since its average score was slightly lower than the overall average mean 3.03 (Mean=2.53)
Furthermore, the OCDE group (Mean=2.55), surprisingly, showed more approve than the other two groups (Mean scores of LDCE and WEBC were 2.32 and 2.17 respectively) when being asked if the interface of the referencing tool was user-friendly and if they could easily find the collocates they needed (item 9). The OCDE group were also the one who showed the strongest motivation to use the assigned referencing tool in leaning other types of collocations (M=3.09) (item10).
87
Table 9 The Average Scores of the 10 Items in the Questionnaire
Item Description OCDE LDCE WEBC Average
Q1 I know what the term “collocation” means. 3.12 2.98 3.17 3.08
Q2 When learning an English word, I pay attention to both the word and its collocates. 3.00 3.05 3.03 3.03 Q3 When learning an English word, I memorize the word and its collocation, instead of just
a single word.
2.67 2.83 2.60 2.71
Q4 While reading, I pay attention to the phrases and collocations in the text. 2.91 2.95 2.77 2.88 Q5 When encountering problems of word usages, I turn to online collocation referencing
tools.
2.42 2.59 2.40 2.48
Q6 I think OCDE/LDCE/WEBC can help me learn collocations. 3.33 3.15 2.53 3.03
Q7 Learning collocations with the help of OCDE/LDCE/WEBC can enhance my ability to memorize vocabulary.
2.88 2.78 2.43 2.71
Q8 Learning collocations with the help of OCDE/LDCE/WEBC can enhance my ability to use the words.
3.12 2.95 2.60 2.90
Q9 The interface of OCDE/LDCE/WEBC is user-friendly. I can easily find the collocates I need.
2.55 2.32 2.17 2.35
Q10 If possible, I’d like to use OCDE/LDCE/WEBC to help me learning other types of collocation, i.e. ADJ+N or ADV+ADJ.
3.09 2.76 2.70 2.85
Note. The full score of each item is four.
88
Results of the written comments.
To explore the strengths and weaknesses of these tools, the feedbacks in the item 11, an open-ended question, were collected and analyzed. Overall, a hundred and ten comments were collected from nearly 70% of the participants. The numbers of comments made by the OCDE, LDCE, and WEBC group were 50, 44, and 16 respectively. What follows is a summary of the strengths and weaknesses of the referencing tools, and the descriptions listed by the students (see Table 10).
Table 10 The Strengths and Weaknesses of the Referencing Tools OCDE (50)
89
LDCE (44)
Features Students’ descriptions
Strengths (21) Fast and convenient interface (12)
In terms of the strengths of the referencing tools, participants from all three groups pointed out that the assigned tools contained rich information of collocation.
Eight percent of the comments from OCDE (four out of 50), 11% from LDCE (five out of 44), and six percent from WEBC (one out of 16) mentioned this advantage.
Moreover, 10% of the comments from OCDE (five out of 50) and 27% from the LDCE (12 out of 44) group reflected that the tools’ interfaces were clear and
convenient. As to the layout, eight percent of the comments from OCDE (four out of
90
50) and seven percent from LDCE (three out of 44) made by the students showed that the layouts of the assigned referencing tools were clear. Surprisingly, two participants in total from these two groups even view the monolingual interface as an enforcement to improve their English ability.
Besides the strengths proposed above, some disadvantages of the assigned tools were also pointed out by the participants. 20% of comments from the OCDE group (ten out of 50), nine percent from LDCE (four out of 44), and 19% from WEBC (three out of 16) were complaints about the interfaces that were considered not user-friendly.
In addition, eight percent (four out of 50), 16% (seven out of 44), and six percent (one out of 16) of the comments in the OCDE, LDCE, and WEBC group respectively mentioned that layouts of the referencing tool messy and unclear.
Furthermore, 20% of the OCDE group suggested that more information, such as explanations and example sentences, be needed while only five percent of LDCE and six percent of WEBC made the same complaint. The monolingual feature of the referencing tools also seemed to cause comprehension obstacles since all the three groups mentioned the problem (12%, 16%, and six percent respectively).
Ten percent of the comments made by the OCDE group, in particular, suggested that the tool could be more suitable for learners of higher proficiency level. In other words, these participants might have found themselves incapable of using the
91
assigned tool.
When it comes to the information processing speed, the WEBC group was the only group who made complaints. Around 44% of the comments from the group indicated that the slow processing speed impeded the whole searching process. On the other hand, four participants in total (one from the OCDE and three from the LDCE group) complained that it was too troublesome for them to look up the proper collocates by using the computer or the Internet.
92