CHAPTER IV RESULTS
The experiment was conducted to examine if voice-based CMC in line with the text-based one is beneficial for language learners to improve their speaking ability. In the present study, there were 113 students (38 students in Gong; 38 students in Skype;
37 students in control group) participating in the program at first. For the experimental groups, nevertheless, the data is retrieved from those who completed all the topics and then is discussed in the results section. After the 8-week treatment, 32 (males: 32) students in Gong group, 34 students (males: 30; females: 4) in Skype, and 37 students (males: 35; females: 2) in control group are included in the result section.
The purpose of the study is to investigate (a) the effect of voice-based CMC on improving EFL learners’ oral proficiency; (b) the comparison of the effect voice-based SCMC and ACMC had on building EFL learners’ speaking ability; (c) the perception EFL learners’ have on voice-based CMC.
The first section includes the results of pre-study questionnaires, where the
participants’ English learning backgrounds along with their computer literacy were
described, and the results of inter-rater reliability in the current study. The second
section shows the answers to the three research questions as well as the results of
post-study questionnaire. Both descriptive and inferential statistics are adopted to
analyze the results.
4.1 Results of Pre-study Questionnaire and Inter-rater Reliability
For learners’ general language learning background, all the 113 students had learned English for 4.5 years on average. In regard to learners’ computer literacy, all students had been using computers for two years or longer and all had a home connection to the Internet. As Coniam and Wong (2004) pointed out, it is valid and important for a computer-based study that participants in the context of CMC are computer literate with the hardware in general. Results from the pre-study questionnaires showed that this pre-condition was satisfied.
As for inter-rater reliability, it played an influential role in the subjective testing
(oral test) since raters’ personal judgments were called for. Table 4.1 presents the
descriptive statistics and correlations between the two raters’ scorings. The resulting
correlation coefficient is .878 and also achieves the statistic significant level p<.05,
which shows the inter-rater reliability in pre-test and post-test.
Table 4.1: Inter-rater Reliability—Peterson’s Correlation Descriptive Statistics
Mean Std. Deviation
Rater1 42.3333 18.51064
Rater2 44.6667 17.75957
Correlations
Rater1 Rater2
Rater1 Pearson Correlation 1 .878**
Sig. (2-tailed) .000
Sum of Squares and
Cross-products 9936.667 8373.333
Covariance 342.644 288.736
* Correlation is significant at the .05 level
** Correlation is significant at the .01 level (2-tailed).
4.2 The Effect of Voice-based CMC on Oral Proficiency
In answering research question one, whether voice-based CMC can improve EFL learners’ oral proficiency, voice-based ACMC (Gong Group) learners’ performance in pretests and posttests as well as voice-based SCMC (Skype Group) learners’ was carefully examined.
4.2.1 Voice-based ACMC
First of all, Table 4.2 summarizes the mean scores and standard deviation (SD) for Gong group in pre- and post-tests. The mean score of Gong Group in the posttest (54.69) is higher than that of the pretests (39.38) by 15.31, indicating learners’
progress in oral proficiency. Besides, the standard deviation of the posttest (13.44) in
Gong Group is lower than that of posttest (13.66), which indicates the variance within
Gong Group is decreased after the voice-based ACMC treatment. Followed by the
basic descriptive statistics, the result of the ANOVA further shows that there is a significant difference between pretest and posttest. The effect of voice-based ACMC on improving EFL learners’ oral proficiency has reached the statistical significant level (p< .05).
Table 4.2
Gain Scores: Pre- and Post-tests in Gong Group—Results of One-Way ANOVA
Pre-Post Mean Std. Deviation N
Pretest (Gong) 39.38 13.66 32
Posttest (Gong) 54.69 13.44 32
Source
Type III Sum
of Squares df Mean Square F Sig.
Corrected Model 3751.56 1 3751.56 20.43 .000
Intercept 141564.06 1 141564.06 770.97 .000
pre_post 3751.56 1 3751.56 20.43 .000**
Error 11384.38 62 183.62
Total 156700.00 64
Corrected Total 15135.94 63 a R
2= .248 (Adjusted R
2= .236)
* p< .05 ** p< .01
In addition to the overall improvement on oral proficiency in Gong group,
participants’ performance of pronunciation & fluency and grammar & content in pre-
and post-test was examined respectively, further reporting the effect of voice-based
ACMC in detail. The total score (100) on the oral test in GEPT Elementary-Level is
points account for the grammar of learners’ speech productions. As can be seen in Table 4.3 and 4.4, the mean score of pronunciation in the posttest (31.56), compared with 19.06 in the pretest, reaches the significant level (p= .00), while the progress in grammar from pretest to posttest (p= .16) fails to indicate statistic significance (p< .05). In other words, the finding suggests that voice-based ACMC improve EFL learners’ oral proficiency, especially in the aspect of target language pronunciation and fluency.
Table 4.3
Gain scores of Gong in Pronunciation & Fluency—Results of One-Way ANOVA
pre_post Mean Std. Deviation N
Pretest (Gong_Pro) 19.06 6.89 32
Posttest (Gong_Pro) 31.56 7.23 32
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 2500.00 1 2500.00 50.10 .000
Intercept 41006.25 1 41006.25 821.78 .000
pre_post 2500.00 1 2500.00 50.10 .000**
Error 3093.75 62 49.90
Total 46600.00 64
Corrected Total 5593.75 63
a R
2= .447 (Adjusted R
2= .438)
* p< .05 ** p< .001
Table 4.4
Gain Score Gong in Grammar & Content—Results of One-Way ANOVA
pre_post Mean Std. Deviation N
Pretest (Gong_Stru) 20.31 8.97 32
Posttest (Gong_Stru) 23.44 8.65 32
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 156.25 1 156.25 2.01 .161
Intercept 30625.00 1 30625.00 394.03 .000
pre_post 156.25 1 156.25 2.01 .161
Error 4818.75 62 77.72
Total 35600.00 64
Corrected Total 4975.00 63
a R
2= .031 (Adjusted R Squared = .016) 4.2.2 Voice-based SCMC
Subsequent to the outcomes of Gong program, Table 4.5 presents the results of
One-way ANOVA for performance of Skype Group in pre- and post-tests. The mean
score of Skype Group in the posttests is 50, which is higher than that in the pretests
(40.59). Furthermore, the results of the standard deviation in the posttest (13.71), in
accordance with what shown in voice-based ACMC, is lower than that of pretest
(14.96). The lower standard deviation after the treatment presumably implies that the
difference of learners’ target language proficiency within Skype Group was reduced
due to voice-based SCMC treatment. A significant main effect of voice-based SCMC
on improving EFL learners’ oral proficiency (F= 7.314, P= .009) is shown, achieving
the significant level (p< .05). The assumption that the practice of voice-based SCMC
could significantly improve EFL learners’ oral proficiency is positive.
Table 4.5
Gain Scores: Pre- and Post-tests in Skype Group—Results of One-Way ANOVA
Pre_Post Mean Std. Deviation N
Pretest (Skype) 40.59 14.96 34
Posttest (Skype) 50.00 13.71 34
Total 45.29 15.01 68
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected
Model 1505.88 1 1505.88 7.31 .009
Intercept 139505.88 1 139505.88 677.60 .000
Pre_Post 1505.88 1 1505.88 7.31 .009*
Error 13588.24 66 205.88
Total 154600.00 68
Corrected Total 15094.12 67
a R2= .100 (Adjusted R Squared = .086)After confirming the positive effect of voice-based SCMC upon participants’
general oral proficiency, its influence on pronunciation as well as grammar is worth
further investigating. Table 4.6 and 4.7 present the gain score of Skype in
pronunciation and grammar respectively. In terms of pronunciation, learners in Skype
group significantly made progress (p=.005). However, the p-value of progress made
in grammar from pre-test to post-test is .068, which doesn’t reach the statistically
significant level (p< .05). Consistent with the result shown in Gong group,
voice-based SCMC has significant effect on EFL learners’ oral proficiency on the
aspect of pronunciation and fluency in particular.
Table 4.6
Gain Score of Skype Group in Pronunciation &Fluency—Results of One-Way ANOVA
pre_post Mean Std. Deviation N
Pretest (Skype_Pro) 20.29 6.74 34
Posttest (Skype_Pro) 25.59 8.24 34
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 476.47 1 476.47 8.42 .005
Intercept 35788.24 1 35788.24 632.35 .000
pre_post 476.47 1 476.47 8.42 .005**
Error 3735.29 66 56.60
Total 40000.00 68
Corrected Total 4211.77 67
a R2 = .113 (Adjusted R Squared = .100)
Table 4.7
Gain Score of Skype Group in Grammar & Content: Results of One-Way ANOVA
pre_post Mean Std. Deviation N
Pretest (Skype_gra) 20.29 10.29 34
Posttest (Skype_gra) 24.41 7.86 34
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 288.24 1 288.23 3.43 .068
Intercept 33976.47 1 33976.47 405.12 .000
pre_post 288.24 1 288.24 3.44 .068
Error 5535.29 66 83.87
Total 39800.00 68
Corrected Total 5823.53 67
a R2= .049 (Adjusted R Squared = .035)
4.2.3 Control Group
To further examine the effect of voice-based CMC, the scores of the control group whose English input comes mainly from English classroom instruction are further analyzed. As shown in Table 4.10, the mean score of the control group in the posttest (40.81) is slightly higher than that in the pretest (40.27) by .054. However, based on the result of ANOVA, although the mean score in the posttest increased a little, the p-value was .019, which did not reach the significant level (p< .05). The outcome indicates that for those who received only the daily instruction in English, hardly could they make progress or even have significant improvement on English oral proficiency. The result also serves to filter out the potential variable, routine instruction in English, which may exist in the two experimental groups.
Table 4.8
Gain Scores: Pre- and Post-tests in Control Group—Results of One-Way ANOVA
pre_post Mean Std. Deviation N
Pretest (Control) 40.27 17.40 37
Posttest (Control) 40.81 16.05 37
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 5.40 1 5.41 .019 .890
Intercept 121621.62 1 121621.62 434.08 .000
pre_post 5.41 1 5.41 .019 .890
Error 20172.97 72 280.18
Total 141800.000 74
Corrected Total 20178.378 73
a R2 = .000 (Adjusted R2= -.014)With the above mentioned comparisons of learners’ performance between pre- and post-test, the answer to the first research question is positive. That is, the results of the present study suggest that compared with the control group, which receive no treatment in speaking ability, voice-based CMC can significantly enhance EFL learners’ general oral proficiency and the target language pronunciation and fluency in particular.
4.3 The Comparison of the Effects Voice-based ACMC and SCMC Have on
Learners’ Oral Proficiency
To answer question number 2, if learners in voice-based SCMC could
significantly outperform those in voice-based ACMC group with reference to EFL
learners’ oral proficiency, the increase in gain scores from pretest to posttest was
analyzed. Table 4.9 provides the descriptive statistics of One-way ANOVA for the
increase in gain scores, reporting the degree of the participants’ progress in oral
proficiency. After the 8-week voice-based ACMC, the average post-test scores is
15.31, which is higher than 9.41 in the voice-based SCMC and 0.54 in the control
group. The result of One-way ANOVA indicates that the increase of gain scores from
pretest to posttest among the three groups reaches the significant level, p= .000
(p< .05). Scheffe’s post hoc comparison , as can been seen in Table 4.10, further
suggests that Gong group make greater progress than Skype group as well as the
control group, reaching the significant level of .032 and .000 respectively (p<.05).
Different from the original hypothesis in the present study and Abram’s (2003) findings in the effect of text-based CMC, it was voice-based asynchronous group that outperformed voice-based synchronous group in terms of oral proficiency.
Table 4.9
Gain Scores: The Progress of the Three Groups after the 8-Week Treatment
Group Mean Std. Deviation N
Gong 15.31 9.50 32
Skype 9.41 10.71 34
Control .54 5.24 37
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 3837.35 2 1918.68 25.33 .000
Intercept 7279.27 1 7279.27 96.11 .000
Group 3837.35 2 1918.68 25.33 .000**
Error 7574.30 100 75.74
Total 18100.00 103
Corrected Total 11411.65 102
a R2= .336 (Adjusted R2= .323)
* p< .05 ** p< .01
Table 4.10
Scheffe’s Post Hoc—Multiple Comparison
95% Confidence Interval
(I) Group
(J) Group
Mean Difference (I-J)
Std.
Error Sig.
Lower Bound
Upper Bound
Scheffe Gong Skype 5.90(*) 2.14 .026(*) .57 11.23
Control 14.77(*) 2.10 .000(**) 9.55 19.99
Skype Gong -5.90(*) 2.14 .026(*) -11.23 -.57
Control 8.87(*) 2.07 .000(**) 3.73 14.01
Based on observed means.
* p< .05 ** p< .01
Not only has the comparison of the overall progress in oral proficiency but also that of the specific progress in EFL learners’ target language pronunciation and fluency among the three groups deserved careful examination. As mentioned in section 4.2, both Gong and Skype have significantly positive effect on participants’
pronunciation and fluency. The result of one-way ANOVA on pronunciation and fluency (Table 4.11) shows that the average increase of gain scores in pronunciation is 12.50 in Gong group, which is higher than that in Skype group (5.29) reaching the significant level of .000 (p< .05).
Simply put, the answer to the second research question is negative, suggesting
that regarding the progress participants made in overall oral proficiency as well as in
the aspect of pronunciation, voice-based ACMC group significantly outscores
voice-based SCMC group, yielding to contrary results of the previous studies on
text-based CMC (Abrams, 2003).
Table 4.11
Result for One-Way ANOVA for the Pronunciation and Fluency Progress of
Gong and SkypeGroup Mean Std. Deviation N
Gong 12.50 6.72 32
Skype 5.29 7.06 34
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 855.97 1 855.97 17.98 .000
Intercept 5219.61 1 5219.61 109.63 .000
Group 855.97 1 855.97 17.98 .000**
Error 3047.06 64 47.61
Total 9000.00 66
Corrected Total 3903.03 65
a R2 = .219 (Adjusted R2= .207)* p< .05 ** p< .001
4.4 Learners’ Perception of Voice-based CMC
In an effort to understand EFL learners’ perceptions of voice-based CMC (the third research question), the post-study questionnaire was carefully analyzed. In this section, qualitative data from the post-study questionnaire completed by the two experimental groups indicates several interesting points regarding participants’
attitude toward voice-based CMC (Gong/Skype), their intention of using the program in the future, and their preference for the eight different topics assigned in the program. In general, the findings reflect learners’ positive perception of incorporating voice-based CMC into spoken language learning.
4.4.1 Learners’ Attitude toward Voice-based CMC
First of all, Table 4.12 presents the learners’ responses to Question 1, showing
their general perception of voice-based CMC. Overall, learners reported positive attitudes toward having topic discussion in the context of voice-based CMC to improve their oral proficiency in comparison with face-to-face class discussion. There are as much as 84.4% (Strongly Agree, 31.3% and Agree, 53.1%) of all the Gong participants in favor of the statement that if they like to improve oral proficiency through Gong activities; in the same fashion, more than two-thirds of Skype participants (Strongly Agree,11.8% and Agree, 55.9%) hold positive attitude toward voice-based SCMC activities.
Table 4.12
Responses to Question 1 of participants in Gong and Skype
SA A M D SD N % N % N % N % N %
ACMC (Gong) 10 31.3 17 53.1 4 12.5 1 3.1 0 -
SCMC (Skype) 4 11.8 19 55.9 7 20.6 2 5.9 1 2.9
Note. N = Numbers of participants.
SA = Strongly Agree, A = Agree, M = Moderate, D = Disagree, SD = Strongly Disagree Q1: I like to enhance my English oral proficiency through Gong/ Skype.
To explore participants’ attitude, learners were required to further explain their preferences for voice-based CMC as well as their difficulties with CMC programs.
Table 4.13 summarizes frequency of the reasons why participants in Gong preferred
voice-based ACMC rather than face-to-face oral discussion. For all the reasons listed
alternatives if necessary. The top four reasons why they enjoyed Gong activities include three factors related to learners themselves and one to the distinguishing feature of the software, Gong: (1) lower affective filter in the context of voice-based ACMC (F=27), (2) preparation of the discussion content in advance (F= 25), (3) freedom to decide when and where to do the audio-recordings (F=21), and (4) the convenient function of voice speed up/down in Gong (F=19). Aside from the reasons listed on the posttest questionnaire, 11 users chose ‘others’, pointing out their appreciation of the repeated recording function served in Gong, which provides them more opportunities to perfect the recordings by speaking out the target language again and again.
Similarly, frequency of the reasons why participants in Skype preferred voice-based SCMC to class discussion are presented in Table 4.14. The top four reasons reported by Skype users are: the freedom to decide when and where to have the discussion (F=25), the low-anxiety environment of learning spoken language (F=23), the handy use of the Internet for collecting topic-related information (F=19), and the benefits of adequate preparation for the discussion topics (F= 18).
Furthermore, two participants ticked the item of ‘others’ in the questionnaires strongly
claiming that Skype activities strengthen their confidence on their own speaking
ability.
Below are some selections from the questionnaire.
“It never occurs to me that I can communicate with people in English. I believe
and feel that my English speaking ability is now being improving little by little. The
online discussion makes me confident on my English.” (Student 1)
“Discussion in the context of Skype encourages me to utter the language and
also gave me the confidence on speaking English. Whenever I do the oral discussion
at home, my parents are always surprised at my performance.” (Student 2)
Table 4.13
Responses to Q2: Frequency of the Reasons Why Gong was Preferred as Reported by Users
Reasons: Gong users can Frequency(F)
1. feel less anxious when leaving the voiced message at home. 27 2. enjoy the freedom to decide the way of discussion. 9 3. have equal chances to join in the conversation. 5 4. prepare the content of voiced message before doing the recording. 25 5. have the opportunities to listen to as well as imitate others’ pronunciation. 17 6. decide when and where to leave the voiced message. 21 7. surf the Internet to collect the information related to the discussion topic. 10 8. adjust the voice speed to facilitate the understanding of messages left by
others. 19
9. Others 11
Table 4.14
Responses to Q2 – Frequency of the Reasons Why Skype was Preferred as Reported by Users
Reasons: Skype users can Frequency
1. feel less anxious and comfortable without the teacher’s domination 23 2. enjoy the freedom to decide the way of discussion. 14 3. have equal chances to join in the conversation. 4 4. prepare the content of discussions before doing the recording. 18
5. decide when and where to have the discussion. 25
6. surf the Internet to collect the information related to the discussion topic. 19
7. Others 2
Despite the fact that the majority hold positive attitude toward voice-based CMC, there were still one participant in Gong and three participants in Skype reporting that they preferred class discussion to that in voice-based CMC. Three out of these further pointed out that the conversation would be more interesting and richer if they could see the interlocutor’s facial expression and body language while one reported that it was the numerous computer problems he encountered that blunted his interest in voice-based CMC.
Regarding the difficulties students faced during the online topic discussion, Table
4.15 and 4.16 report learners’ opinions on voice-based ACMC and SCMC by
frequency. For all the problems they encountered, participants were once again
allowed to choose more than one of the alternatives. On one hand, as can be seen in
Table 4.17, 8 out of 32 participants in Gong claimed that they had no problems at all
when holding the oral discussion. For the remaining 24 participants, most of them
complained about the lack of instant response/interaction with the interlocutors (F=10), which highlights the major feature of voice-based ACMC, a significant delay of speech time (D, Crystal, 2001). In Table 4.18, on the other hand, there were 6 out of 34 participants in Skype group reporting that seldom did they experience difficulties when carrying out the programs. Yet, up to 20 participants mentioned how difficult for them to deal with time arrangement after school. Some students further explained that due to their tight schedule after school, it is rather hard to find the common time for the synchronous online discussion. Instead of the problems resulting from computer software and hardware, it is time arrangement that becomes Skype users’ main concern. Apart from the issue on time arrangement after school, the second problem reported by high frequency (8) is the lack of interaction between interlocutors such as facial expression and body language.
In summary, the results above-mentioned not only suggest participants’ overall
positive attitude toward voice-based CMC but also provide further understanding of
their preference for CMC to class discussion, accompanied by the problems they
encountered in the context of voice-based CMC.
Table 4.15
Responses to Q3—Frequency of Difficulties in Using Gong
Difficulties Frequency
1. The unfamiliarity with the platform of Gong. 2
2. The unfamiliarity with the platform of Moodle. 1
3. The unfamiliarity with the operation of computer. 0 4. The lack of instant response to the message left on the board. 8 5. The lack of interlocutor’s facial expressions and body language. 9 6. The lack of the immediate interaction with the interlocutor. 10 7. The strangeness of talking in face of a computer. 1
8. The access to the Internet is unavailable. 0
9. The microphone for the voiced recording is hard to get. 1
10. No problems at all 8
11. Others 2
Table 4.16
Responses to Q3—Frequency of Difficulty in Using Skype
Difficulties Frequency
1. The unfamiliarity with the platform of Skype. 0
2. The unfamiliarity with the operation of the recording software. 3 3. The unfamiliarity with the operation of computer. 0 4. The lack of interlocutor’s facial expressions and body language. 8 5. The strangeness of talking in face of a computer. 5 6. The lack of access to the Internet or enough bandwidth of the Internet 2 7. The microphone for the voiced recording is hard to get. 2 8. The difficulty in arranging the time for online discussion after class. 20
10. No problems at all 6
11. Others 0
4.4.2 Learners’ Intention to Have Voice-based CMC
Subsequent to the participants’ attitude toward voice-based CMC, the survey of
their potential intention to keep having oral discussion in the context of voice-based
CMC was included in the post-study questionnaire. Table 4.17 displays the percentage
and numbers of participants in Gong and Skype who have inclination to use voice-based CMC in the future. Up to 81.2% of participants in Gong answered positively when being asked if they are willing to use Gong in the future while 67.6%
of participants in Skype showed inclination to keep using Skype for language learning.
Learners in Gong have stronger intention to use it. Though the majority of participants have intention to use voice-based CMC in the future, the rest of students in Gong (18.8%) and Skype (32.4%) reported either negatively or neutrally on the intention to continue using the programs.
Some participants’ comments on the intention issue are shown as follows:
“I am willing to use Skype in the future. Through the discussion activity, I have
more chance to speak English, which I think it can positively improve my oral
proficiency.” (Student 1) [Positive]
“The activity is quite interesting and I can have further understanding of my
classmates, having closer relationship with them.” (Student 2) [Positive]
“The conversation in Skype is a real conversation. I enjoy speaking English in
the real living environment rather than in class.” (Student 3) [Positive]
“I think the program is good for my English oral ability; however, here is no
knowing that I will have time to continue the program every week. Therefore, I am not
sure if I will use Gong in the future.” (Student 4) [Neutral]
“I am such a busy student that I have no time to carry out extra activities after
school.” (Student 5) [Negative]
“The loading of schoolwork is too heavy for me to continue the program after
class.” (Student 6) [Negative]
In a word, most students have strong intention to keep using Gong/ Skype in the future because they perceived the convenience and the effect of having extra oral discussion in the context of voice-based CMC and also considered the activities interesting. For those who reported unfavorably to use Gong/Skype, on the other hand, that the activities might bring extra schoolwork loading and could deprive them of free time is their main concern.
Table 4.17
Reponses to Q4—Intention to use voice-based CMC in the future as Reported by
Gong/Skype UsersPositive Neutral Negative Total N % N % N % N %
Gong 26 81.2 3 9.4 3 9.4 32 100
Skype 23 67.6 5 14.8 6 17.6 34 100
Note. N = Numbers of participants.
Q4: I am willing to use Gong/Skype in the future.