This chapter provides a thorough discussion on the results of participants’
performances on the pretests, posttests as well as delayed posttests, coupled with participants’ feedback on their experiences in MALL. The discussion is presented in the order of the research questions:
1. Do students improve their listening ability and their knowledge on target vocabulary items as well as grammar points after participating in the podcasting project?
2. What are students’ perceptions about the use of podcasts for language learning via smartphones?
3. Do students’ attitudes toward mobile assisted language learning change after the one-month experiment?
The Effects of Podcasts via Smartphones on Language Leaning General Listening Ability
In the present study, with an attempt to answer the first research question, the learning effects of podcasts via smartphones were measured and discussed in the order of general listening, vocabulary, and grammar tests.
From the results of listening tests (plotted in Figure 5), it could be found that both PC group and MIX group made significant progress. This result supports many researchers’ suggestions that podcasts can serve as rich, authentic input and thus be a kind of useful leaning material to improve students’ listening ability (McBride, 2009;
Schimidt, 2008; Stanley, 2006). However, when utilizing the identical material with the PC group and MIX group, participants of the SP group did not seem to have significantly different performances in pretest and posttest.
Figure 5: Participants’ Performances in Listening Pretest and Posttest
There are two possible reasons for this result: first, the listening tests implemented in the present study might fail to measure the participants’ improvement; secondly, the devices participants used could have influenced the effects of learning in listening.
While the content of the podcast series participants listened to focus on target vocabulary items and grammar points, the instrument used to assess their performances is General English Proficiency Test (GEPT), which is designed to examine learners’ general ability. It is very likely that the scale of the test is too large to precisely demonstrate participants’ improvements in listening comprehension. If tests can be tailor-made to target on a specific learning scope, like what has been doing for high school monthly listening test, the progress of the SP group probably will be assessed more accurately.
On the other hand, while there is limitation in how participants’ gains were measured, the influence of learning tools and learning environments could not be excluded. When asked about some difficulties in listening to podcasts via smartphones, many participants reported disadvantages related to the tool itself and the environment. Five students complained that the screen is too small, four students
said the noisy environment made it difficult to concentrate, three students did not like that they were unable to take notes when learning on the move with smartphones, and one reported it was inconvenient that he had to download the podcasts beforehand since Wifi connection is not available everywhere.
For these reasons, some students originally assigned to the SP group gradually switched to computers and thus contributed to the existence of the MIX group. It could be inferred that when podcast listening was treated as learning instead of a casual activity like listening to music, it demands undivided attention and concentration. Thus a quiet and “more like learning” environment might make students more comfortable when they perceived that they are going to learn something in the podcasts. On the other hand, for students that still stuck to smartphones, these disadvantages might have negative influence on their leaning and thus limited their improvements in listening.
Vocabulary and Grammar Knowledge Gain
Unlike previous research that only reported on learners’ preferences of learning tools (Stockwell, 2010) or the increase of learning motivation (Wang and Looi, 2010;
Levy and Kennedy, 2005), the present study aimed to objectively measure the learning gains of students in terms of vocabulary and grammar. It turned out that learning materials delivered through smartphones did have positive effects on students’ vocabulary and grammar learning and their progress is statistically significant. From Figure 6 and 7, it could be seen that for both the MIX and SP group, students’ knowledge about target vocabulary items and grammar points had a noticeable increase.
Figure 6: The Performances Changing with Time (MIX group)
In addition to the short-term effects, an attempt was made to see if the learning effects could last long. Therefore, during the experiment, a delayed posttest held two weeks after the posttest was designed to evaluate the long-term effect of the treatment.
Figure 7: The Performances Changing with Time (SP group)
While participants knew that they would be tested on some target words and grammar points in the posttest, they were not aware of the existence of the delayed posttest. In other words, while the participants still had those materials at hand, there was no motive for them to review what they had learned after finishing the posttest, and it turned out that they still retained their learning gains.
This maintenance of the learning gains is an encouraging finding indicating that the educational value of smartphones is more than a source of fun and a trigger for leaning. The present study suggests that using smartphones as a learning device do help with students’ knowledge gains and the gains are not lost in a short time.
The Learning Effects of Various Learning Tools
While the learning effectiveness of materials delivered through smartphones had been found, one question needs to be asked, however, is “will students perform even better with computers or other media”. To answer this question, the comparisons of mean differencesbetween each set of tests across the three groups were made through One-way ANOVA together with Post Hoc Scheffe test, and the three groups’
changes over time are illustrated in Figures 8, 9, and 10.
From Figure 8, it could be found that the mean scores of vocabulary tests of the three groups all changed in a very similar pattern. First, an obvious increase of mean scores could be noted between pretest and posttest, followed by a slight decrease between posttest and delayed posttest. Actually, according to the results of One-way ANOVA, none of the mean differences of any group between pretest and posttest was found to be significantly different with other groups (Post Hoc Scheffe Sig. = .615), nor did the decline between posttest and delayed posttest show any significant disparity (Post Hoc Scheffe Sig. = .495).
Figure 8: Vocabulary Test Mean Scores Comparison Across Groups (BBC)
Similar trend could be found in another set of vocabulary tests that assessed participants’ knowledge about words and phrases learned from the podcast English Bites. The results are displayed in Figure 9.
Figure 9: Vocabulary Test Mean Scores Comparison Across Groups (EB)
Though the podcast series English Bites is in the form of video and some students did complain about the low resolution and small screen when watching the program on their smartphones, it is found that the progress made by the SP group actually has no significant difference compared with the other two groups (Post Hoc Scheffe Sig. = .924). In other words, while students using smartphones to watch clips for learning might suffer from the limited physical conditions of the device, the learning effect seems to be as good as watching the clips on a personal computer.
Likewise, the results of grammar tests indicate that there is no noticeable difference in learners’ progress (Post Hoc Scheffe Sig. = .419) or maintenance (Post Hoc Scheffe Sig. = .714) across groups (see Figure 10). To summarize, learning tools seem to have no great effects on students’ performances in the present study.
However, this result is inconsistent with what Thornton and Houser had reported in 2005.
Figure 10: Grammar Test Mean Scores Comparison Across Groups
In Thornton and Houser’s study (2005), three mini lessons were sent to
students’ mobile phones via e-mail messages at specific time every day to enhance vocabulary learning. At the same time, a mobile-phone website was prepared for other students to read the identical materials on the identical schedule. According to Thornton and Houser, the number of words learned by students reading the mini lessons through mobile emails was significantly more than students reading the identical materials on the web.
The inconsistent findings may result from the differences in students’
background and the experiment design in the two studies. Participants in Thornton and Houser’s study were Japanese college students who were reported to use mobile phones much more often than personal computers for connecting to the Internet. It was mentioned that the cost of Internet connection was very low and almost every participant had constant access to the web. On the other hand, many of the participants using smartphones in the present study did not have a constant access to the Internet due to the high cost of 3G. The great majority of them only made use of the free school Wifi. Furthermore, students in Taiwan are more likely to connect to the web through personal computers instead of mobile phones.
As to the experiment design, though identical material and schedule were supplied for both groups in Thornton and Houser’s study, when students in the mobile group were pushed to learn new words by the email messages three times a day, students in the web group were only reminded of the learning materials once a week.
Added that the before-experiment survey showed the average number of times students received email in one week was 82.8 while the times they surfed the web was only 3.9, the difference in reminding frequency and learners’ original habits could have a great effect on students’ actual learning behavior, and thus influenced the learning gains in Thornton and Houser’s study. On the other hand, for the present study, many of the participants in MIX and SP groups were more experienced in
receiving emails through PC than through smartphones, and all participants received identical emails with the same frequency from the researcher. No further reminder was provided or pushed to the SP or MIX group.
Therefore, unlike Thornton and Houser’s report, it turned out that no significant learning differences occurred across groups in the present study. Thus, it might be suggested that students’ experiences and habits could be one factor that impacts the learning effects of various learning tools.
Participants’ Perceptions about Mobile Assisted Language Learning Students’ Experiences in MALL
To investigate participants’ prior experiences in mobile learning as well as their thoughts about it, questionnaires were collected and analyzed to present participants’
ideas both quantitatively and qualitatively. In addition, follow-up interviews were held to have students further elaborate their responses when necessary.
In the questionnaire pretest, participants were first asked whether they were using smartphones or not and if they had ever learned English with smartphones. The results of these questions are plotted in Figure 11.
Figure 11: Students’ Smartphone Use and Experiences in MALL
From Figure 11, it can be found that even though almost half of the participants had smartphones of their own (n = 23), only 11 out of the 23 students had ever tried to use smartphones for English learning. Furthermore, when asked if they still had the habit of learning with smartphones, only six students provided a positive answer and said they had used dictionaries, apps, or some Internet resources to support their language learning. This result actually corresponds to the findings of Stockwell (2008) and Barrs (2011) that simple ownership of a technology and having the skills to use it does not necessarily relate to whether learners will use it or not. Subsequent to these questions, the researcher further investigated the reasons why some students never used or stopped using smartphones as a language learning tool, and the results are summarized in Table 19.
Table 19: Reasons Why Learners Never Use or Stop Using Smartphones for English Leaning
N Reasons
Never Stop
22 0 I didn’t have a smartphone.
8 0 I didn’t think of it.
11 0 It’s too expensive to connect to the Internet with Smartphones.
12 1 There’s no suitable resource for smartphone language learning.
17 5 I’m more comfortable with other media like computer and paper.
3 1 Other reasons.
Note: N = the number of students who chose the reason
The total number of students who never used smartphones for English learning is 37. The total number of students who once used smartphones for English learning but quitted is 6. Students were allowed to choose more than one reasons.
From responses of the participants who never had experiences in MALL, one can find that except for the absence of smartphones, the high cost of the Internet
connection, the lack of appropriate learning resources, and students’ learning habits seem to be major factors that kept learners from trying.
To increase learners’ motivation in learning with smartphones, and to have learners experience the true mobility of learning materials, one way is to make use of applications that need no connection when operation. Most of the open applications of this kind utilized by language learners are dictionaries or translation tools (Google Translator), e-books, and apps assisting vocabulary memorization or acquisition (My Word Book, 我愛背單字, 英文單字王 EngKing, AccelaStudy). Though these apps are free from connection problem, they are seldom updated and thus the information provided is usually limited. Thus applications that are rich in content, easy to use, and dealing with more language learning aspects like listening and speaking are in need.
In addition to developing more useful apps, another way to motivate smartphone language learning is to improve the quality of Internet connection by making free Wifi hotspots more pervasive or the cost of connection lower. Thus learners can make use of the abundant resources on the web.
Other than the problems of high cost and limited resources, another major contributing factor that influenced the use of smartphones seems to be learners’
learning habits and their psychological barriers. “Feeling more comfortable to learn with other media like computer and paper” is not only the reason selected by many participants who had no experiences in MALL, but also the major reason why students once learned with smartphones quitted. This finding actually is congruent with Stockwell’s (2010) suggestion that one possible factor making students chose PC over smartphones is that learners just perceived PC as easy to use and sufficient for their learning needs. Therefore, it seems like even though the mobile technology is now available, it still takes time to witness the changes in learners’ behaviors and perceptions.
Students’ Feedback on Podcasts
In the present study, three podcast series were used as the self-learning materials to support vocabulary and grammar learning, and were evaluated by the participants on interest, comprehensibility, usefulness, and learners’ preference degree.
Among the three podcast series, BBC The English We Speak is the most popular one, receiving very high scores in each aspect (M = 4.73, 4.29, 4.23, 4.63). In each episode, a target vocabulary or phrase was introduced through a situated conversation between two hosts, with plenty of example sentences assisting learners’ understanding. In the survey, over forty students described its content as “interesting,”“comprehensible,” or
“life-related.” Many thought the program was concise and the time (2-3 minutes) was moderate. Only few students said the definitions provided were not clear enough or the items introduced were not very useful.
As for English Bites produced by Australian Network, students also gave quite positive responses to the four aspects (M = 4.19, 4.15, 4.15, 4.10). It was a program that uses activities visitors can do in Australia as the theme of each episode and teaches conversational expressions in English at the same time. What praised the most by students in this series are the beautiful scene of Australia that attracted their eyes and the visual aids that assisted their learning. However, while some students appreciated the ample knowledge they could acquire besides the language, others felt it is not focused enough and said they had difficulty in grasping something clear and specific to memorize. Another problem pinpointed by several students is accent. Since each segment of the program was hosted by some local Australian people and also different foreign visitors, the inconsistent accents sometimes made it difficult to comprehend, especially for learners who have only exposed to American or British English.
Compared with the other two podcasts, Grammar Girl is not so popular among
the participants (M = 2.79, 2.56, 3.94, 2.96). In each episode, the host introduced one complex or confusing grammar point by offering lots of examples and tips to memorize, and the lecture usually lasts for 6 to 8 minutes. The pace of Grammar Girl (about 160 words per minute) is much faster than The English We Speak (about 100 words per minute), and Grammar Girl also contains words that are not so familiar to the participants. Thus, a majority of students reflected that the content is difficult and the host speaks too fast for them to catch up. However, though most students perceived this program to be challenging, over half of them still reported the instruction as useful and clear.
In addition to participants’ perceptions about the content of podcasts, information about the time and the place they listened to podcasts were collected and recorded from participants’ daily reply. From the replies of participants in all three groups, it was found that the great majority of participants chose to listen to the podcasts after they finished all of their classes or extracurricular activities and went back to the dorm or their home. Only a few of them made use of the break at noon or the time they were having breakfast. For students who used smartphones to listen to the podcasts, only 6 reported once listening to podcasts on the bus or MRT, in the library, coffee shop, or the hospital. In other words, most participants preferred to set aside time for podcast listening on desktop or laptop computers. Even when students have portable devices to listen to podcasts on the move, they still prefer to do it in a quiet environment at a time they can concentrate. This observation is similar to what Lee and Chan (2007) had found in their study.
In Lee and Chan’s study, subject-related podcasts were provided to a group of distance education students for approximately 10 days. Among the 18 students who responded to their questionnaire, 13 (72%) reported that they preferred to listen to the podcasts using a desktop or laptop computer, 3 (17%) preferred portable devices, and
2 (11%) used both. Fourteen students (78%) usually listen to the podcasts at home instead of other places or on the move. When asked about when they listened to the podcasts, only 5 (28%) responded while travelling, while waiting, or while carrying out other tasks; on the other hand, 13 students (73%) set aside time to listen to the podcasts.
These usage patterns observed do not corroborate the vision of learning on the move as portrayed or expected by many researchers. While mobility is viewed as one of the major advantages of mobile learning, learners’ behaviors reveal that
These usage patterns observed do not corroborate the vision of learning on the move as portrayed or expected by many researchers. While mobility is viewed as one of the major advantages of mobile learning, learners’ behaviors reveal that