CHAPTER 3 RESEARCH METHODOLOGY
3.4 Research Instruments
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
62
significant differences between female and male learners in terms of vocabulary learning performance. Second, Huang et al., (2010) confirmed that learners at dissimilar English proficiency levels may have different vocabulary learning outcomes. Hence, gender and original vocabulary level is considered as the group-assigning criteria in this study.
The recruited participants were divided into two evenl y-distributed groups based on the group-assigning criteri on: their gender and original vocabulary level assessed by the vocabulary pre test. One group was labeled as the experimental group with the support of MVLA-GF for vocabulary learning, while the other group was labeled as the control group with the support of MLVA-NGF. The purpose of forming two
evenl y-distributed groups is to minimize the possible interference caused by the participants‘ dissimilar vocabulary levels or their gender
difference.
3.4 Research Instruments 3.4.1 The Employed MV LA-GF
PHONE words (全 民 瘋 單 字 ), developed by Alice Education Studio (學 騰 教 育 有 限 公 司 ), was selected as the employed MVLA-GF in this
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
63
study. PHONE Words, which is a mobile vocabulary learning APP with game-related functions (MVLA-GF), aimed to facilitate English vocabulary learning for certain tests, such as TOEIC and TOEFL.
Nowadays, TOEIC has been chosen as one of the graduation requirements in several universities in Taiwan—including NCCU; therefore, this study chose PHONE Words (TOEIC version) as the research instrument. In addition, the employed MVLA-GF, i.e. PHONE words, not onl y provides learners with the official TOEIC word list released by ETS, but also
integrates mobile -assisted learning with additional game -related features to promote learners‘ learning motivations and learning performance.
Thus, PHONE Words was chosen as the research instrument in this study.
Six major functions provided by PHONE Words—including word list, customized word list, pre -established learning path, traditional assessment, gamified assessment, and ranking among friends —are detailed as follows.
As revealed in Figure 3.2, the employed MVLA -GF, i.e. PHONE Words, allows learner s to use word list to learn a set of vocabulary in an alphabetical manner. Detailed information on each word, such as definition of words, pronunciation of words, and example sentence s with
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
64
audio clips , would be presented after a click of a given word. In addition, learners can mark words so that those marked words would be put into customized word list for reviewing purposes. Vocabularies archived in customized word list are also served as the question pool for either traditional assessment or g amified assessment.
Figure 3.2 Word List and Customized Word List of PHONE Words
Figure 3.3 shows pre-established learning path offered by PHONE Words. Using pre-established learning path, learners must finish the task assigned to each day and pass the assessment—either traditional assessment or gamified one. After finishing each day‘s task, learners would be granted a course badge, which allows them to move on to the next task and also gives them a higher ranking on ranking among friends.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
65
Figure 3.3 Pre -Established Learning Path of PHONE Words
Furthermore, two assortments of self-assessment —traditional assessment (Fig. 3.4) and gamified assessment (Fig. 3.5) —are provided by PHONE Words. Learners can choose either archived vocabulary in customized word list or FIAQ (frequentl y incorrectly-answered questions) to take the assessment.
Figure 3.4 Traditional Assessment of PHONE Words
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
66
(a) Tic-Tac-Toe (b) Tug-of-VOC (c) Star VOC Figure 3.5 Gamified Assessment of PHONE Words
As depicted in Figure 3.5, three vocabulary games, Tic-Tac-Toe (Fig.
3.5(a)), Tug-of-VOC (Fig. 3.5(b)), and Star VOC (Fig. 3.5(c)), are offered by PHONE Words.
Tic-Tac-Toe is a game for two players —one uses noughts, while the other uses crosses—taking turns marking the spaces in a 3-by-3 grid.
Learners would be tested on a multiple-choice basis. When a player successfull y answers a vocabulary question, the player can mark a space in the grid. Aspiring to win this game, t he player ought to succeed in placing three respective marks in a horizontal, vertical , or diagonal row.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
67
Tug-of-VOC, with its name adapted from the sport, t ug-of-war, is a mission-based game in which players must spell the tested word successfull y in order to save the sheep from the black bear. This blank-filling vocabulary assessment can test learners ‘ ability to spell the learned words.
Star Voc—adapted from name of the movie, Star Trek—is a mission-based game in which a user pilots a spacecraft and has those approaching vocabulary questions answered along the journey. When a question appears, the player has to choose which answer best represents the vocabulary being tested.
As shown in Figure 3.6, PHONE Words is embedded with a game competition mechanism of ranking among friends. The ranking is based on three dimensions including number of vocabulary stickers, familiar words, and course badges granted from pre -established learning path . When successfull y answering a vocabulary question in either traditional assessment or gamified assessment, the learner would receive a vocabulary sticker. When a given word is successfull y answered three times in a row, it would be added into the learner ‘s familiar words. When using ranking among friends, learners can see how many vocabulary
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
68
stickers, familiar words, and course badges other learners have received.
This function is expected to provide learners with a sense of competition and extrinsic learning motivation as Prensky (2001) and Huang et al.
(2010) suggested in their research.
Figure 3.6 Ranking among Friends of PHONE Words
Generall y speaking, all three vocabulary games belong to q uiz games under the classification of E nglish vocabulary learning games (see Table 2.5) proposed by Hung and Young (2007). Admittedly, quiz games are, to certain extent, criticized as ―less preferable since they somewhat remind learners of an exam‖ (Hung & Young, 2007). However, Dondi and Moretti (2007) argued that q uiz games are one of the most effective approaches for fostering factual knowledge , assisting learners in
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
69
recalling the acquired knowledge, and promoting knowledge retention.
Thus, the provided vocabulary games embedded in MVLA-GF which bring about game attributes such as interaction , fantasy, goals, sensory stimuli, and fun (Prensky, 2001; Garris et al., 2002)—supported by other game-related features including stage -clearing and badge-granting elements offered by pre -established learning path, as well as a sense of competition and extrinsic learning motivation provided by ranking among friends—were expected to assist learners in successfull y acquiring and retaining vocabulary.
3.4.2Vocabulary tests
To assess the effects of utilizing MVLA-GF as a learning tool on vocabulary learning, this study adopted questions from New TOEIC Official Test -Preparation Guide III, the official test -preparation guide, to
evaluate the participants‘ vocabulary learning performance. The adopted questions consisted of incomplete sentences and text completion.
The vocabulary pre test, immediate vocabulary post test, and two-week delayed vocabulary post test are detailed as follows.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
70
3.4.2.1 Vocabulary Pretest
A vocabulary pre test was assigned to the twent y participants before performing the experiment al treatment. The purpose of vocabulary pretest is to assess the participants ‘ original vocabulary level prior to the experimental treatment. In addition, the pre test results were further employed as the group -assigning criterion aiming to establish two evenl y-distributed groups.
3.4.2.2 Immediate vocabulary post test
An immediate vocabulary p osttest, which was assigned to the participants of both groups on the last day of the experimental treatment was used to examine the participants‘ vocabulary gains in the given learning period of time. The purpose of immediate vocabulary posttest is to evaluate the difference in vocabulary acquisition between the experimental group using MVLA-GF and the control group using MVLA-NGF.
3.4.2.3 Two-week delayed vocabulary posttest
To evaluate the participants ‘ vocabulary retention, this study conducted a two-week delayed vocabulary posttest. The content of the
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
71
delayed posttest was identical with that of the immediate post test;
however, the order and answer sets of questions were rearranged. The purpose of t wo-week delayed vocabulary post test is to assess the difference in vocabulary retention between the experimental group using MVLA-GF and the control group using MVLA -NGF.
3.4.3 Questionnaire Survey
A questionnaire survey was conducted to evaluate the participants‘
perspectives and attitudes toward using the assigned MVLA-GF and MVLA-NGF. The questionnaire survey was conducted on the last day of the experiment al treatment. The questionnaire consisted of three sections and is detailed as follows.
The first section was designed to elicit the demographic information of the participants, which enables this study to further anal yze data obtained from the experiment al process. The second section consisted of three dimensions: effectiveness, usability, and satisfaction —each with five corresponding statements. The participants were asked to express their perspectives in a five -point Likert scale , ranging from ―strongl y agree‖ to ―strongl y disagree. ‖ Moreover, the last section—composed of four open-ended questions —was designed to elicit participants ‘ opinions,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
72
problems, or suggestions toward using MVLAs as a vocabulary learning tool.
3.4.4 Log Files Gathered from the Employed MVLA-GF
With the help of the game developer, log files—which recorded the participants‘ click times of major functions provided by the assigned MVLA-GF—were anal yzed. The log files were used to confirm whether significant correlations existed among learners‘ usage behaviors of MVLA-GF and their learning performance. Since game-related functions, the main focus of this study, were removed from the MVLA assigned to the control group , anal yses of the log files were conducted onl y on the experimental group.
3.4.5 Semi-Structured Interview
To collect valuable qualitative data which may fail to be revealed by the questionnaire and the vocabulary tests, t he study conduct ed a semi-structured interview at the end of the experiment. With the flexibilit y embedded in a semi-structured interview allowing the interviewer to bring up adequate, related questions, the study can thus tailor the questions during the interview processes. Such pattern allows this study to dig out much information on each interviewee ‘s distinctive
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
73
perspectives or personal experiences.
In short, th is study adopts mixed methodology combining quantitative and qualitative approaches to explore the research questions.
The purposes of employing mixed methodology in this study are to reveal the participant ‘s in-depth, distinctive perceptions, to assist this study in explaining the analyzed quantitative data, and to further achieve triangulation.