Results of Questionnaire Survey - EXPERIMENTAL RESULTS

CHAPTER 4 EXPERIMENTAL RESULTS

4.3 Results of Questionnaire Survey

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

adopting game-based learning approaches indeed received improvemen t in terms of vocabulary learning performance. In addition, this result also supports Hogle‘s assertion that games can assist learners in memorizing the learned information (Hogle, 1996).

In conclusion, this study confirmed that MVLA-GF can be considered as an effective learning tool for assisting learners in acquiring and retaining English vocabulary.

4.3 Results of Questionnaire Survey

Next, a questionnaire aiming to investigate the participants‘

perceptions and attitudes toward the use of the assigned MVLA as a vocabulary learning tool was conducted. The employed questionnaire consisted of three dimensions including effectiveness, usability, and satisfaction . A reliabilit y anal ysis of the questionnaire and questionnaire assessment results of two groups are detailed in the following sections.

4.3.1 Reliability Analysis of Questionnaire

As shown in Table 4.6, reliabilit y of the employed questionnaire was confirmed by Cronbach‘s Alpha (Overall=.883; Effectiveness=.857;

Usability=.803; Satisfaction=.887). These results confirmed that the used questionnaire provides satisfactory reliability —with Cronbach's alpha

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

exceeding 0.7.

Table 4.6 Reliability Analysis of Questionnaire Cronbach‘s

Alpha

Cronbach‘s Alpha Based on Standardized items

Number of Items

Effectiveness .857 .862 5

Usability .803 .744 5

Satisfaction .887 .887 5

Overall .883 .851 15

4.3.2 Comparisons of Questionnaire Survey Results between Two Groups

To investigate the perceptions and attitudes generated by the participants in each group, analyses of the employed questionnaire were further performed. Results of descriptive statistics and Wilcoxon signed rank test are presented as follows .

As shown in Table 4. 7 and Table 4. 8, the mean score of effectiveness dimension (M=4.34, SD=.21187) of experimental group was significantly higher than that (M=3.4200, SD=.31903) of control group (Z=-3.782, As ymp Sig=.000). Similarly, the mean score of satisfaction dimension (M=4.3600, SD=.24585) of experimental group was also significantl y higher than that (M=3.2800, SD=.23476) of control group (Z=-3.819, Asymp Sig=.000). However, the mean score of usabilit y dimension (M=3.6000, SD=.32660) of experimental group was

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

not significantl y different from that (M=3.5400, SD=.31340) of control

group (Z=-.533, As ymp Sig=.594).

Table 4.7 Descriptive Statistics of Questionnaire Assessment

Group (N) Mean SD. Min. Max. Comparison of Mean scores Effectiveness EG (10) 4.3400 .21187 4.00 4.60

EG > CG CG (10) 3.4200 .31903 3.00 4.00

Usability EG (10) 3.6000 .32660 3.00 3.80

EG ≈ CG CG (10) 3.5400 .31340 3.20 4.00

Satisfaction EG (10) 4.3600 .24585 4.00 4.80

EG > CG CG (10) 3.2800 .23476 3.00 3.80

Table 4.8 Results of Wilcoxon Signed Rank Test of Questionnaire Assessment between Two Groups

Mann-Whitney U Wilcoxon W Z Asymp. Sig. (2-tailed)

Effectiveness .5000 55.500 -3.782 .000***

Usability 43.500 98.500 -.533 .594

Satisfaction .000 55.000 -3.819 .000***

4.3.3 Summary of Questionnaire Assessment Results of Two Groups As shown in Tables 4.9 and 4.10, no significant difference between two groups in terms of usabilit y was found. This result may derive from the fact that usabilit y dimension concerns mostly with users‘ experiences in operating the assigned MVLA-GF or MVLA-NGF, and the two assigned mobile vocabulary learning APPs —apart from the difference in the existence of game -related functions —are basically identical.

In contrast, anal ytical results confirm that the learners of the

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

experimental group utilizing MVLA-GF rated the adopted learning tool more effective and more satisfactory than did those in the control group.

This result is consistent with several previous studies (David et al., 2009;

Tarng & Tsai, 2009; Franciosi, 2011; Kang, 2012; Uzun et al., 2013) asserting that learners hold positive attitudes toward learning tools with game-related functions.

Thus, findings in th is study confirm ed that, in the facilitation of English vocabulary learning, MVLA -GF contributes to higher effectiveness and satisfaction than MVLA-NGF dose. In other wor ds, MVLA-GF can be taken as a promising tool for improving tedious, less-favored traditional vocabulary learning experience.

4.4 Correlations among Usage Behaviors of MVLA -GF and Learning Performance in the Experimental Group

To confirm whether significant correlations existed among the usage behaviors of MVLA -GF and learning performance, Pearson correlation analysis was conducted in this study. Since game-related functions were removed from the mobile vocabulary learning APP assigned to the learners of the control group, correlation analysis was conducted onl y on log files belonging to those in the experimental group.

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

4.4.1 Descriptive Statistics of Usage Behaviors of MVLA -GF in the Experimental Group

Descriptive statistics of usage behavior of MVLA -GF in the experimental group is presented in Table 4. 9. Among six major functions including pre-established learning path, word list, customized word list, traditional assessment, gamified assessment, and ranking among friends , pre-established learning path received the most mean clicks (M=282.1000, SD=37.71074), followed by gamified assessment (M=136.000, SD=15.64182).

Table 4.9 Descriptive Statistics of Usage Behaviors of MVLA -GF in the Experimental Group

N Mean SD Minimum Maximum

Usage Time 10 23.1000 .91439 22.00 24.50 Total Click Times 10 617.0000 17.80137 589.00 642.00

Pre-Established

Learning Path 10 282.1000 37.71074 210.00 329.00 Word List 10 39.8000 5.80804 29.00 49.00 Customized Word List 10 61.7000 17.34006 34.00 87.00 Traditional Assessment 10 48.5000 6.51920 41.00 60.00 Gamified Assessment 10 136.000 15.64182 117.00 162.00 Ranking among Friends 10 48.9000 6.11828 42.00 59.00 4.4.2 Results of Pearson Correlation among Usage Behaviors of MVLA-GF and Learning Performance in the Experimental Group

As shown in Table 4.10, with regard to correlations among

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

vocabulary acquisition and usage behaviors of MVLA -GF, this study found that significant positive correlations exist among vocabulary acquisition and several usage behaviors , including usage time (r=.669, Sig=.034), total click times (r =.830, Sig=.003), pre-established learning path (r=.786, Sig=.007), gamified assessment (r=.660, Sig=.038), and ranking among friends (r=.712, Sig=.021). Moreover, a significant negative correlation exists between vocabulary acquisition and traditional assessment (r=-.780, Sig=.008). However, neither word list (r=-.306, Sig=.389) nor customized word list (r=-.117, Sig=.747) was confirmed significant correlation s.

In terms of correlations among vocabulary retention and usage behaviors of MVLA -GF, this study found that significant positive correlations exist in the corresponding sections of MVLA -GF usage behaviors—usage time (r=.682, Sig=.030), total click times (r=.681, Sig=.030), pre-established learning path (r=.782, Sig=.008), gamified assessment (r=.708, Sig=.022), and ranking among friends (r=.637, Sig=.047). Similarly, a significant negative correlation also exists in traditional assessment (r=-.674, Sig=.033), but no significant correlation exists either in word list (r=-.048, Sig=.894) or in customized word list

‧

MVLA-GF and Learning Performance in the Experimental Group

Acquisition Retention Usage Time Total Click Times Pre-EstablishedLearning Path Word List CustomizedWord List Traditional Assessment GamifiedAssessment Rankingamong Friends

Acquisition

GamifiedAssessment

Pearson

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

*p < .05 ， **p < .01 ， ***p < .001

4.4.3 Results of Pearson Correlation among Usage Behaviors of MVLA and Learning Performance in the Control Group

Descriptive statistics of usage behavior of MVLA in the control group is presented in Table 4.11. Traditional assessment received the most click times (M=222.400, SD=28.19062), followed by two word lists—customized word list (M=161.4000, SD=14.04912) and w ord list (M=147.4000, SD=15.68580).

Table 4.11 Descriptive Statistics of Usage Behaviors in the Control Group

N Mean SD Minimum Maximum

Usage Time 10 21.3250 .55340 20.75 22.25 Total Click Times 10 531.2000 49.05960 455.00 600.00

Word List 10 147.4000 15.68580 119.00 168.00 Customized Word List 10 161.4000 14.04912 140.00 187.00 Traditional Assessment 10 222.4000 28.19062 175.00 259.00

As shown in Table 4.12, with regard to correlations among vocabulary acquisition and usage behaviors of MVLA, this study found that significant positive correlations exist among vocabulary acquisition and several usage behaviors, including usage time (r= .759, Sig=.011), total click times (r= .863, Sig=.001), word list (r=.725 Sig=.018),

‧

customized word list (r=.809 Sig= .005), and traditional assessment (r=.695 Sig=.026). However, i n terms of correlations among vocabulary retention and usage behaviors of MVLA , onl y a significant positive correlations exist in traditional assessment (r=.707 Sig=.022) with no other significant correlations being found.

Table 4.12 Results of Pearson Correlation among Usage Behaviors of MVLA and Learning Performance in the Control Group

Acquisition Retention Usage Time Total Click Times Word List CustomizedWord List Traditional Assessment

Acquisition

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Sig.(2-tailed) .026 .022 .048 .000 .397 .002

*p < .05 ， **p < .01 ， ***p < .001

4.4.3 Summary of Pearson Correlation Results among Usage Behaviors of MVLA-GF and Learning Performance

As shown in Table 4.1 0, two subsections related to actual usage, including usage time and total click times, showed significant positive correlations with overall vocabulary learning outcomes. This result is consistent with the assertion , argued by Sandberg et al. (2011), indicating that learners who invest more time on vocabulary learning activities will obtain better outcomes in vocabulary tests. That is, learners‘ involvements in vocabulary learning activities indeed exert

positive impacts on their learning performance.

Furthermore, significant positive correlations among vocabular y learning performance and several gamification -related dimensions, such as pre-established learning path , gamified assessment, and ranking among friends, were confirmed . Previous studies have concluded that games can facilitate learning performance (Richard, 2006), promote learners‘ engagement (Franciosi, 2011), and spark learners‘ motivation

(Kang, 2012). In conclusion , results in this study confirmed that

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

significantly positive correlations among game-related functions of MVLA-GF and learning performance indeed exist. Restated, this study demonstrates that the more time a learner invests in game-related functions of a given MVLA-GF, the better learning performance a learner can achieve.

Additionally, a significant negative correlation between vocabulary learning performance and traditional assessment is confirmed, whereas vocabulary learning performance and gamified assessment showed a significant positive correlation. Moreover, no significant correlations among vocabulary learning performance and the two word lists—both of which are unrelated , or less-related, to gamification—were found.

In conclusion , this study demonstrates that learners with better learning outcomes tend to (1) actively engage themselves in learning activities; (2) invest more time in gam e-related functions of a given MVLA-GF; and (3) spend less time on traditional learning activities, such as traditional assessment.

4.5 Results of Semi -Structured Interview

A semi-structured interview was conducted to collect qualitative data from the learners in the experimental group. All of the ten learners

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

of the experimental group were invited, but onl y eight of them participated in the interview. Prior to the interview, the interviewees‘

permissions were obtained in order to record the interviewing process for further anal ysis. The semi-structured intervie w consisted of eight open-ended questions , and the interview time of each interviewee lasted approximately thirty minutes. Excerpts and anal yses are presented as follows.

Question 1: Describe how you used to learn English vocabulary.

All of the eight inter viewees expressed that they often used word lists organized either by different publishing companies or by themselves to expand their vocabulary size. However, half of them mentioned that such word lists were hardl y effective since they frequentl y forgot what they had learned in no time.

This result is consistent with several studies (Liao, 2004; Li e t al., 2010) concluding that repetition is still the most -used vocabulary learning strategies among Taiwanese EFL learners . In addition, it also

confirmed the graduated interval recall hypothesis (Pimsleur, 1967), which denotes that ―those acquired words would inevitably fade from learners‘ memory if there are without effective reviewing processes.‖

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

“…Well, I use vocabulary lists a lot, but I usually forget what I have learned when reaching section M! It’s quite frustrating.” (Participant 1)

“…I just try to remember them. Sometimes, I would write down words on my note book or even made some highlights, but I didn’t find it useful.” (Participant 5)

“…I use vocabulary lists often. Also, I usually read English articles online and use oxford dictionary to learn words.” (Participant 7)

“…I bought some vocabulary lists for different purposes, but it’s not that helpful as it ha d claimed to be.” (Participant 8)

“…I usually use vocabulary lists with some blanket papers. Well, it usually turns out to be a waste of time.” (Participant 4)

Question 2: Describe how you learn ed words with the assigned MVLA-GF.

Most of the interviewees indicated that they had heavil y relied on the 30-day learning schedule offered by pre-established learning path . Some of them declared that they, once in a while, used word list for referring to example sentences and word pronunciation. In addition, some of them utilized customized word list for reviewing purposes. Most

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

importantly, interviewees‘ time was also massivel y invested in using gamified assessment to evaluate their own learning pro gress.

“…For me, I used the vocabulary li st for the pronunciation of each

word, and I used pre-established learning path a lot. Well, if I got some

time, I tended to use gamified assessment.” (Participant 4)

“…I simply followed the schedule offered by pre-established learning path, and I used gamified assessment a lot - simply because I wanted to know whether I had learned well.” (Participant 2)

“…I mostly used pre-established learning path and then shifted onto the vocabulary list for pronunciation or example sentences. At last,I used

gamified assessment to test myself ” (Participant 7)

“…Well, I tend to use pre-established learning path, and then I prefer to use gamified assessment for reviewing.” (Participant 5)

“…I love skimming the vocabulary list first, adding words with which I’m not familiar to my own word bank. So, I can use pre-established learning path and gamified assessment for active use.”

(Participant 1)

Question 3: What’s your using priority among the six major functions

provided by the assigned MVLA-GF? Why?

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Interviewees‘ responses were tallied in a scale from one (the least

important) to six (the most important). As shown in Table 4.11, pre-established learning path was favored by most of the interviewees (M=5.2), followed by gamified assessment (M=4.9). In addition, the interviewees rated ranking among friends (M=3.5), customized word list (M=3.3), and word list (M=3.1) at a close level . Lastly, all of the eight interviewees marked traditional assessment (M=1.0) as the least

important.

Table 4.13 Results of Participants’ Using Priority among the Six Major Functions Provided by the Assigned MVLA-GF

Mean Ranking Pre-established learning path 5.2 #1

Gamified assessment 4.9 #2

Ranking among friends 3.5 #3

Customized word list 3.3 #4

Word list 3.1 #5

Traditional assessment 1.0 #6

As revealed in excerpt s of the interview, most of the interviewees favored pre-established learning path and gamified assessment since

these two functions provided incentives including ―stage-clearing‖ and

―gamification.‖ Such merits not only effectively motivated the

interviewees in the learning processes but also led to effect of edutainment, a benefit brought by MVLA -GF.

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

Some of the interviewees favored customized word list because it was the question pool of gamified assessment and traditional assessment.

Word list, which provided example sentences and pronunciation of words, was also championed by some learners for referring purposes. On the other hand, ranking among friends was regarded as an effective incentive for promoting lea rning motivation, which generates from a sense of competition—an attribute of MVLA-GF. Unanimously, traditional assessment was considered to be the least important by all interviewees since it was considered an unfavorable approach to accomplishing assessments.

“…I think pre-established learning path and gamified assessment

are the most important . It helped me review what I had learned. Word list

was a bit messy for me, so I don’t like it.” (Participant 2)

“…I favored pre-established learning path and gamified assessment but also needed customized word list to review. I don’t actually care

about the rest, except for the useless traditional assessment. ”

(Participant 8)

“…pre-established learning path and gamified assessment are my favorite, of course , but I somehow like to add and review some words that

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

100

I’m not familiar with through customized word list as well. Ranking among friends is motivating. You know. You’d like to compete with others.”

(Participant 5)

“…pre-established learning path and gamified assessment provided an easy way of learning. Two word lists were both fairly important, since

I want to refer to a dictionary-like tool.” (Participant 4)

“…Gamified assessment is the best! Otherwise, it’d be quite boring

to learn words. But the learning schedule is quite useful as well. It saved

a lot of time for me. ” (Participant 6)

Question 4: What’s the difference between MVLA-GF and

MVLA-NGF?

Most of the interviewees agreed that MVLA-GF and MVLA -NGF can both (1) accomplish ubiquitous learning and fragmented learning ; (2) provide convenient, diversified learning methods; and (3) offer authentic learning materials and instant assessments.

Eight interviewees further expressed that attributes brought by MVLA-GF—a sense of competition , challenge, gamification, and stage-clearing elements , for example—were expected to (1) offer better interaction; (2) reduce boredom and repetitions; (3) facilitate their

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

101

motivation effectively; and (4) conduce to active involvements. Thus, all of them preferred MVLA-GF over MVLA -NGF.

These findings respond to several previous studies indicating that the a sense of competition allows undertakers to achieve ego-gratification (Prensky, 2001); that games, compared with traditional teaching methods, can promote learners‘ learning motivation (McFarlane et al., 2002); that games are engaging and seductive (Mitchell &

Saville-Smith, 2004); that games assist learners in fostering habits and understanding (Klopfer et al., 2009); and that the fantasy and challenge attributes of games can immerse learners in the learning process (Kang, 2012).

“…Either way works for me. I can learn anytime and anywhere,

unlike traditional learning tools. But it’d better if I can play games and

learn words at the same time! That’s so much fun!” (Participant 1)

“…Well, for me, either way provides various learning methods. I can use them even when I’m commuting, so convenient! But I think games can motivate me more, compared to other learning APPs, I think.”

(Participant 4)

“…Learning with games is an effortless way of learning, I got to say.

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

102

Learning vocabulary used to be quite boring, so it’s great to have it supported by games.” (Participant 5)

“…Either way is fine! You can learn with real materials and engage in assessments. But I prefer game -related apps, for stage-clearing and competition really motivates me a lot!” (Participant 2)

“…Both looks fine to me, compared with traditional word lists. But I have to say that I still prefer those with games, actually. I can interact with it. It’s fun! And I also like to compete with my friends , so I kind of like the challenge part. ” (Participant 8)

Question 5: What are your explanations to the research finding s that the vocabulary learning performance of experimental group learners was significantly higher than those of control group learners ?

In terms of vocabulary acquisition, the interviewees responded that gamification -related functions embedded in the employed MVLA -GF not onl y provided a favorable learning approach but also brought about additional motivation and active involvements , thus effectively facilitating vocabulary acquisition. This result is consistent with the study of Huang et al. (2010) asserting that incorporating game attributes in learning systems can encourage learners to sustain their motivation.

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

103

Their research also revealed that learners reacted conspicuousl y to extrinsic motives such as witnessing other learners‘ scores.

With regard to vocabulary retention, the interviewees expressed that MVLA-GF, compared with traditional learning methods, offers a challenging, favorable, engaging, interesting, interactive, authentic, and exciting learning environment. Such learning experience can assist learners in effectively acquiring vocabulary. Most importa ntly, MVLA-GF can help learners firml y retain vocabulary they have learned—as evidenced by the results that learners utilizing MVLA-GF retained vocabulary conspicuousl y better than those in control group did. This viewpoint casted by the interviewees corresponds to several game attributes mentioned by previous studies, including challenge, goals (Prensky, 2001); interest cultivation, flow (Garris et al., 2002);

excitement, fun (Schwabe & Goth, 2005); sensory stimuli (Wilson et al., 2009); interaction (Kang, 2012); and encouragement (Huang et al.,

2012).

“…When playing vocabulary games, I found myself more easily to acquire and retain vocabulary, probably because it’s fun and it got some interactive and engaging elements ” (Participant 8)

‧

國

立政治大學

‧

N a tio na

l C h engchi U ni ve rs it y

104

“…As for me, I used game-related functions a lot because I wanted to top the ranking list. It’s interesting and fun , and we were even more motivated. It might be the reason why we outperformed the other group, I

guess.” (Participant 3)

“…I was motivated by game-related functions and the interaction functions, so I did remembered words firmly, I supposed.” (Participant 6)

“…It’s interesting, and I know I can learn something even when I’m

playing games. That’s great! So, it should be the reason why we did well.

Well, I hope I was right.” (Participant 1)

“…It’s about the competition brought by games, I guess. It provided

在文檔中遊戲式行動英語字彙學習APP之學習認知與成效評估研究：以臺灣 EFL 學習者為例 - 政大學術集成 (頁 96-0)