3.3.2 Experimental analysis
3.3.2.1 Learning performance analysis
Figure 12 displays the comparison result of the learning performance for both the pre-test and post-test. Table 5 shows the summarization of learning performance for both the participating groups. Figure 12 reveals the score differences of pre-test and post-test in the control group are closer than those in the experimental group. In addition, the percentages of learners with progress score in both the experimental and control groups are 94% and 67%, respectively. The results show that the scores of both the groups have progress, but the score progress of the experimental group is obviously superior to the control group.
In order to investigate whether there are significant differences in score progress between both the experimental and control groups, SPSS statistical software was used to analyze the result of pre-test and post-test. The SPSS analysis results of pre-test are presented in Tables 6(a) and 6(b).
In this work, the Independent-samples T Test was employed to analyze the collected data of two participating groups. Before performing the PDA learning process for English vocabulary learning, the mean score of the experimental group on the pre-test is 10.39 and the standard deviation is 3.032. The mean score of the control group on the pre-test is 12.61 and the standard deviation is 5.315. The result of Independent Sample T Test (sig of t = 0.135 > 0.05) indicates that these two groups are not significantly different on the pre-test; therefore, the English vocabulary abilities of two groups can be viewed as identical before conducting the designed learning process.
Table 5. Summarization of learning performance for both participating groups Group
Comparison Item Experimental group Control group
Number of learners 18 18
Number of learners with
progress score 17(94%) 12(67%)
Number of learners with
retrogression score 0(0%) 5(27%)
Number of learners with
constant score 1(6%) 1(6%)
Table 6. The Independent Samples T Test of pre-test between two groups (a) Group statistics
18 10.39 3.032 .715
18 12.61 5.315 1.253
class experimental group control group pretest
N Mean Std. Deviation Std. Error Mean
(b) Independent Sample T Test
27.004 .135 -2.222 1.442 -5.182 .737
pretest
df Sig. (2-tailed) Mean Difference Std. Error
Difference Lower Upper
95% Confidence Interval of the Difference t-test for Equality of Means
Experimental group
0 5 10 15 20 25 30
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Learners
Grade
pre-test post-test
(a) The students’ learning performance in the experimental group
Control group
0 5 10 15 20 25 30
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Learners
grade
pre-test post-test
(b) The students’ learning performance in the control group Figure 12. The learning performance of two participating groups
Next, Tables 7(a) and 7(b) illustrate the Independent Samples T Test of post-test between two participating groups. The mean scores of the experimental and control groups on the post-test are 15.61 and 14.56, respectively. The T test result (t=0.684, sig of t=0.489 > 0.05) shows that two groups are not significantly different on the post-test. Thus, this study further compared the pre-test and post-test within each group using the Paired-Samples T Test.
Table 7. The Independent Samples T Test of post-test between two groups (a) Group statistics
N Mean Std. Deviation Std. Error Mean
(b) Independent Sample Test
.489 .489 .684 34 .499 1.056 1.543 -2.081 4.192
posttest
Tables 8(a) and 8(b) shows the result of Paired Samples T Test of the experimental group. In the experimental group, the difference of the mean scores between pre-test and post-test is –5.222 and the Paired-Samples T Test result reaches the significant level. In other words, after performing the proposed learning process, the promotion of learning performance in the experimental group is significant and the mean testing score increases 5.222 points. Tables 9(a) and 9(b) show the result of Paired Samples T Test of the control group. Similarly, the promotion of learners’ learning performances is also significant and the mean testing score increases 1.94 points in the control group. Hence, these two groups made significantly progress whether using the personalized English vocabulary learning system with or without context-aware service for English vocabulary learning. However, the promotion of the testing score in the experimental group (5.222) is higher than that in the control group (1.944). Thus, this study logically inferred that the learning performance of learners who used personalized English vocabulary learning system with context-aware service is superior to the learners who used personalized English vocabulary learning system without context-aware service.
Table 8. The Paired Samples T Test of the experimental group (a) Paired samples statistics
Mean N Std. Deviation Std. Error Mean
(b) Paired Sample T Test
-5.222 3.246 .765 -6.836 -3.608 -6.8 17 .000
pretest - posttest
Table 9. The Paired Samples T Test of the control group
Mean N Std. Deviation Std. Error Mean
(b) Paired Sample T Test
-1.944 3.589 .846 -3.729 -.160 -2.299 17 .034
pretest - posttest
In order to further explain inter-group variation associated with pre-test (covariance) and adjust the treatment (group) effect, an analysis of covariance (ANCOVA) was used to analyze the collected pre-test and post-test data. The first step is to analyze the homogeneity of regression coefficients. Table 10 shows the analysis result (F=0.438, sig of F=0.513). The F test result does not reach the significant level, thus it means the regression slope of two groups is equivalent. This result confirms the assumption of homogeneity of coefficients, so this study further preceded the analysis of covariance.
Table 10. The analyze result of the homogeneity of regression coefficients
Dependent Variable: posttest
4.882 1 4.882 .438 .513 Source
group* pretest
Type III Sum
of Squares df Mean Square F Sig.
Table 11 shows the ANOCA result (F=5.785, sig of F=0.022) after adjusting the dependent effect (group) with respect to the covariance (pre-test), and it reaches the significant level. This result indicates that the post-test of two groups has significantly different. Next, Table 12 displays the estimated score of post-test after removing the effect of covariance and this study found that the post-test score of the experimental group is higher than that of the control group. Thus, this study concluded that the learners who used the proposed vocabulary learning system with context-aware service had better learning performance than the learners who used the vocabulary learning system without context-aware service.
Table 11. The ANOCA result of two groups
Dependent Variable: posttest
63.336 1 63.336 5.785 .022 Source
group
Type III Sum
of Squares df Mean Square F Sig.
Table 12. The estimated score of two groups after adjusting the dependent effect with respect to the covariance
Dependent Variable: posttest
13.711a .793 12.097 15.326
16.455a .793 14.841 18.069
class control group experimental group
Mean Std. Error Lower Bound Upper Bound 95% Confidence Interval
Covariates appearing in the model are evaluated at the following values: pretest = 11.50.
a.