The results of the Three Tests

Table 4.2 presents the mean and standard deviation of Concordancer Alpha Group, Yahoo Dictionary Group and Control Group on pretest, posttest, and delayed posttest.

The means suggest that the groups performed similarly on the pre-test. However, the differences among the groups were larger on the posttest and the delayed posttest.

The results in the posttest showed that the mean scores of the Concordancer Alpha Group (Mean=50.8) was higher than Yahoo Dictionary Group (Mean= 44.3) and Control Group (Mean=31.9). In addition, the results in the delayed posttest, administered two weeks after the posttest, showed that the mean scores of the Concordancer Alpha Group (Mean=46.1) was also higher than Yahoo Dictionary Group (Mean= 39.1) and Control Group (Mean=31.4). As the Table 4.2 indicates, all three groups performed the best on posttest. Even though they obtained lower scores on the delayed posttest, these scores were still higher than the pretest scores. Furthermore, it seemed that Concordancer Alpha Group performed better than the other two groups on both the posttest and delayed posttest, and that Yahoo Dictionary Group performed better than Control Group on both the posttest and delayed posttest.

56

Table 4.2 The results of the three tests in Three Groups Concordancer

Group Dictionary Group Control Group Number of

In order to determine whether these differences were statistically significant, a 3 (Group) × 3 (Time) mixed-design repeated measures ANOVA was conducted. Since Mauchly’s test indicated that the assumption of sphericity had been violated (χ2(2) = 43.4, p < .001), degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity. The results are shown in Table 4.3.

Table 4.3 Repeated Measures ANOVA Summary for Group, Time, and Type

Source SS df MS F ηp2

57

distinctively. In addition, there was also a significant main effect of Time on Group (F(1.51, 165) =.513, p <.001, ηp2 =.824), with participants showing different average scores for pretest (mean = 30.7), posttest (mean = 41.9) and delayed posttest (mean = 38.6). Moreover, the interaction between treatment type and time was also significant (F[3, 166]= 129, p < .001, ,ηp2 = .701), suggesting that means of the groups differed depending on the time of measurement. In this case, simple main effect instead of main effect would be taken in to account so that the interaction effect could be further explored. The interaction is illustrated in Fig. 4.1.

Figure 4.1 Mean scores of the Three Tests

As Figure 4.1 suggests, the Concordancer Alpha group seemed to have the best performance on both posttest and delayed posttests, while the Yahoo Dictionary group ranked second and the Control group ranked last. As for the differences among the three different groups, as shown in Table 4.4, post hoc comparisons using Bonferroni test

58

indicated that the Concordancer Alpha Group significantly outperformed the Yahoo Dictionary Group (Mdiff=4.14, 95% CI [1.26,7.02], p=.002) and the Control group (Mdiff=11.3, 95% CI [8.4,14.1], p<.000).

Note: *p<0.05

In Fig. 4.1, the scores the two experimental groups exhibited similar pattern over time. They performed the best in immediate posttest, right after the treatment; however, their scores regressed in the delayed posttest. The post hoc analyses revealed significant differences between pretest and immediate posttest (Mdiff=11.2, 95% CI [12.1,10.2], p<.000), between immediate and delayed posttest (Mdiff=3.298, 95% CI [2.73,3.87], p<.000), and also between pretest and delayed posttest (Mdiff=7.87, 95% CI [8.87,6.87], p<.000).

To follow up the significant two-way interaction between the Treatment type (Group) and Time, the simple effects of Group were analyzed at each level of Time. As illustrated in Table 4.5, there existed a significant simple effect of Group for Posttest (F [2, 300] = 225, p<.001) and Delayed Posttest (F [2, 300] = 136, p<.001). On the other hand, the simple effects of Time were analyzed at each level of Group. It was found that significant simple effect of Time could be observed in Concordancer Alpha Group (F[1.4, 182] = 906, p<.001) and Yahoo Dictionary Group (F[1.5, 182] =263, p<.001), Table 4.4 Post-hoc test analysis using Bonferroni

(I)

59

while in control group no significance was found. Since there were simple main effect of time and treatment found, they were further analyzed. The results were illustrated in Table 4.5 and Table 4.6.

Note: *p<0.05

The post-hoc comparison of Time*Treatment (as shown in Table 4.6) indicated that, in both Posttest and Delayed Posttest, CA Group performed significantly better than YD Group and that YD significantly outperformed Control Group. In addition, the post-hoc comparison of Treatment *Time (as shown in Table 4.7) also revealed that participants in both Concordancer Alpha Group and Yahoo Dictionary Group progressed significantly in the posttest, which indicates that both referencing tools adopted in the present study helped participants to retrieve correct translation. In brief, these results suggest that both bilingual referencing tools help participants perform better on translation task and facilitate participants’ translation learning. However, providing bilingual concordancer seems to lead to better translation performance.

Table 4.5 Summary of Simple Main Effect of Group and Time

Source SS df MS F sig

Group (Between-subjects variable)

▶Pretest 1.57 2 0.785 0.051 0.95

▶Posttest 6914.274 2 3457.137 225.07 <.000*

▶Delayed Posttest 4170.93 2 73.769 135.772 <.000*

Error(Time) 10137.756 330 30.72 Time (Within-subjects variable)

▶CA Group 7309.463 1.406 5200.58 905.55 <.000*

▶YD Group 3547.807 1.543 2299.683 263.03 <.000*

▶Control group 12.359 2 6.179 0.706 0.495

Error(Group) 1593.704 182.27 8.743

60

To understand participants’ retention and residual effect, the score differences between the posttest and the delayed posttest and between the pretest and the delayed posttest in each group were compared respectively. It was observed that the two experimental groups obtained significantly lower scores in delayed posttest compared to their posttest scores, which suggested that participants forgot what they retrieved or learned from their assigned bilingual reference tool two weeks after the treatment.

Nevertheless, although they both regressed significantly in the delayed posttest, their delayed posttest scores were still significantly higher than their pretest scores. This showed that participants regressed two weeks after the treatment ended, but their translation performance was still better than the entry knowledge in the pretest.

Moreover, as illustrated in Table 4.6, participants in CA group still scored significantly higher than YD Group in delayed posttest. Considering their entry knowledge was in the same baseline, this means that although both groups of participants’ retention dropped significantly, the residual effect in CA group was still better than YD group.

61

Table 4.6 Pairwise Comparisons of Time*Treatment

Time (I) Treatment (J) Treatment

Mean

*. The mean difference is significant at the .05 level.

b. Adjustment for multiple comparisons: Bonferroni.

62

Table 4.7 Pairwise Comparisons of Treatment *Time

Treatment (I) Time (J) Time

*. The mean difference is significant at the .05 level.

b. Adjustment for multiple comparisons: Bonferroni.