3.3.2 Experimental analysis
3.3.2.2 Questionnaire analysis
To assess learner satisfaction degree for the proposed personalized context-aware English vocabulary learning system, a questionnaire involving 23 questions dealing with four areas was designed to measure whether the proposed English vocabulary learning system satisfies the requirements of most learners. The four question types contain personal information relating to learner learning by PDA, the convenience of system operation, the investigation of learner learning attitude towards using the proposed learning system, and the self assessment of learners’ English vocabulary ability before and after using the proposed English vocabulary learning system. Table 13 summarizes the descriptions of question types. Eighteen learners in the experimental group were invited to complete this questionnaire after attending the two week learning activity.
Table 13. The descriptions of question types
Question Type The number
of questions Description
Personal Information
about Using PDA 3 To get the personal information about learners who attend the learning activity using the proposed system
System Operation 9 Questions related to the user interface and the content of learning materials
Learning Attitude 10 To investigate whether the system can enhance learners’ learning motivation or interests and promote their learning achievements or not
Self Assessment 1
To ask learners for self-assessing their English vocabulary abilities before and after using the proposed learning system
Table 14 lists the results of satisfaction evaluation. To simplify the analytical results, the responses “strongly agree” and “agree” are merged into the single response of “approved”, and the responses of “strongly disagree” and “disagree” are merged into the single response of “disapprove”
The evaluation results indicate that the satisfaction degree of “approved” achieves 54.9% in terms of system operation and 56.1% with regard to learning attitude, as listed in Tables 14(b) and (c).
Table 14. The satisfaction evaluation results of questionnaire (a) The investigation results of the personal information
The Number of Learners
(b) The investigation results of the system operation
Satisfaction Degree (%)
Question Type Question strongly
disagreed disagreed no
opinion Agreed strongly agreed I think that the proposed context-aware
ubiquitous English vocabulary learning system
provides a friendly user interface. 0 0 27.8 44.4 27.8 I am very clear about the learning procedure of
the proposed context-aware ubiquitous English
vocabulary learning system. 0 0 11.1 72.2 16.7
I agree that applying location-based context-aware technique in the leaning is novel
and it can assist my learning. 0 5.6 50 22.2 22.2 I agree that the English vocabulary materials
presented on the PDA are very clear. 0 17 38.9 44.4 0 I agree that the vocabularies recommended by
the system are highly relevant with my learning
location and learning time. 0 5.6 61.1 27.8 5.56 I think the proposed context-aware ubiquitous
English vocabulary learning system is a useful learning tool to assist English vocabulary learning.
0 5.6 38.9 33.3 22.2
I agree that learning English vocabulary by PDA is very convenient; because I can perform
English learning at any time and place. 17 5.6 27.7 27.8 22.2 The proposed context-aware ubiquitous English
vocabulary learning system can effectively
assist my learning. 5.6 5.6 44.4 38.9 5.56
If there are similar English curriculums in the future, I am pleasure to use the proposed system
to learn English once again. 5.6 11 22.2 38.9 22.2 System
Operation
Average 9.4 35.7 54.9
(c) The investigation results of the learning attitude
Satisfaction Degree (%)
Question Type Question strongly
disagreed disagreed no
opinion Agreed strongly agreed The design learning materials on the proposed
context-aware ubiquitous English vocabulary learning system can promote my learning interests.
0 11 44.4 33.3 11.1
Learning Attitude
I often increase my learning time because learning by the proposed context-aware ubiquitous English vocabulary learning system promotes my learning interest.
5.6 5.6 38.9 38.9 11.1
I think that using the proposed context-aware ubiquitous English vocabulary learning system can effectively promote my English vocabulary ability.
5.6 5.6 27.8 50 11.1
I agree that using PDA to learn English
vocabulary is a very interesting learning mode. 0 17 22.2 44.4 16.7 The pronunciation of English vocabulary
provided in the proposed system can help me deepen the memorized impression on vocabulary.
0 11 33.3 33.3 22.2
I think that the integrating English vocabulary learning with the real learning environment is
beneficial to my English vocabulary learning. 0 0 38.9 38.9 22.2 After learning some vocabulary, a cloze test
immediately given from the proposed system for the learned vocabulary is very helpful to vocabulary learning.
0 0 27.8 55.6 16.7
The learning activity offered by the proposed context-aware ubiquitous English vocabulary learning system is more diversified than that of the traditional classroom English vocabulary learning.
0 11 38.9 33.3 16.7
The proposed context-aware ubiquitous English vocabulary learning system satisfies user-oriented design.
5.6 11 33.3 44.4 5.56
Compare to the traditional vocabulary instruction, learning English vocabulary based on the learning location and learning time is helpful to promote my linking and memorization on vocabulary.
0 11 33.3 38.9 16.7
Average 10 33.8 56.1
(d) The investigation results of the self-assessment
Vocabulary ability
Question Type Question Learner
Number
This question aims at self-assessing your vocabulary ability.
Please use the notations Δ and ○ to indicate your vocabulary abilities based on the following ability scale before and after using this mobile English vocabulary learning system, respectively.
17 5 5.5
0 1 2 3 4 5 6 7 8 9 10
18 0.4 0.6 Average 3.867 5.272
Moreover, in terms of self-inspection, 17 of 18 learners thought that their vocabulary abilities promoted after using the proposed system for English vocabulary learning, with just one learner not agreeing that their vocabulary abilities improved, and no learners believing that their vocabulary abilities had deteriorated. Table 14(d) lists the self-assessment results. The self-assessment results are then submitted to analyze whether learner English vocabulary abilities differ before and after performing learning activity. The analytical results are listed in Table 15 and reveal that learner self-assessments improved significantly after using the proposed system (t= -6.188, sig of t=0.000<0.05). This indicates that most learners agreed that their English vocabulary abilities were enhanced after using the proposed context-aware ubiquitous English vocabulary learning system.
Table 15. The Paired-Samples T Test result of self-assessment English vocabulary ability (a) Paired samples statistics
(b) Paired Samples T Test
-1.4056 .9637 .2272 -1.8848 -.9263 -6.188 17 .000
before -after Pair 1
Mean Std.
Deviation Std. Error
Mean Lower Upper
95% Confidence Interval of the Difference Paired Differences
t df Sig.
(2-tailed)
Additionally, to further investigate whether learners significantly prefer the context-aware learning mode provided in the experimental group to the non-context-aware learning mode in the control group, all students in the control group were invited to use the proposed context-aware ubiquitous English vocabulary learning system for one week after finishing their learning activity and post-test. That is, the 18 students simultaneously experienced both the learning modes (i.e.
without and with context-aware service), and then a questionnaire was designed to measure learners prefer which learning mode. Table 16 summarizes the questionnaire result. Among the 18 students investigated by the study, this study found that 72.2% of learners preferred to use the proposed context-aware ubiquitous English vocabulary learning system for English vocabulary learning. Meanwhile, 94.4% of learners agreed that using the proposed system with context-aware service could promote their learning motivation. More importantly, all students thought that the proposed context-aware ubiquitous English vocabulary learning system could help them develop a deeper impression of vocabulary than the mobile English vocabulary learning system without context-aware service. Additionally, 88.9% of learners are happy to learn other English curriculums using the proposed context-aware ubiquitous English learning system in the future.
Table 16. The preference comparison of both the learning modes Do you prefer to learn English vocabulary by the
proposed system with or without context-aware
service? 72.2 27.8
Do you think that which learning mode can promote
your learning motivation much more? 94.4 5.6
Do you think that which learning mode can make your
score progress much more? 61.1 38.9
Do you think that which learning mode can deepen your memorized impression on vocabulary much
more? 100 0
If there are similar English learning activities in the future, which learning mode do you prefer to use again?
88.9 11.1
4. Discussions
In this section, how to integrate the proposed personalized context-aware ubiquitous English vocabulary learning system with the formal classroom learning is first discussed. Additionally, how to apply the proposed scheme to the vocabulary learning of any second languages is addressed.
Moreover, the convenience of learning English vocabulary by mobile devices is also discussed herein.