Nineteen articles studied on the higher level thinking ability. The researcher performed random model and got total weighted mean ES =.83, 95% CI [.52, 1.15], standard deviation= .60. The homogeneity statistic (QT = 19.07, p >.05) is insignificant, meaning that the data in this research is homo, so the total weighted mean ES is settled. The details are showed in Table 4.23.
Research results showed that total weighted mean ES (d+) is positive and 95% CI [.52, 1.15] is significantly different from zero, meaning that the effect of applying DGBL on students’ higher level thinking is significantly better than the effect of applying Non-DGBL on students’ higher level thinking and achieved large effect size level.
As for the publication bias examination, the Fail-safe N= 198.6 > Tolerance Level (5×19+10=105), meaning that the publication bias is unlikely to occur.
Table 4.23
Calculations of Higher Level Thinking N=1701
Year Author Total N subject matter Educational level Sample location ES(d) SD
1992 Faryniarz 58
Environmental problem solving
University France .06 .09
2003 Costabile 34 Logic Junior High Chile .18 .01
2009 Sanchez 273 Science Junior High America .25 .01
2009 Gao 44 Civics & Society University Netherlands .26 .13
2010 Eow 69 Nature & science Elementary Taiwan .36 .06
2011 Eseryel 280 STEM education University France .37 .08
2011 Wouters 30 Biology Elementary Germany .43 .04
2011 Eseryel 251 STEM education Secondary France .43 .07
2012 Hung 65 Math Junior High America .46 .02
2012 Chang 92 Math Elementary Taiwan .86 .05
2012 Cheng-Yu 17 Geography Elementary Taiwan .89 .13
2012 Hwang 50 Nature & science Elementary Italy .97 .08
2012 Yang 44 Civic and Society Elementary Taiwan 1.08 .09
2013 Erhel 44 Medical disease Elementary Taiwan 1.14 .09
2013 Erhel 48 Medical disease University America 1.16 .08
2013 Lucht 116 English vocabulary Senior High Taiwan 1.23 .11
2013 Lu 107 Insects Junior High Taiwan 1.23 .11
2014 Verpoorten 28 Physics Elementary Taiwan 1.93 .03
2014 Hsiao 51 Nature & science Junior High Malaysia 2.85 .12
Note. The researcher compiled the table. The list only showed the first author’s last name.
Chapter Five Discussion 1. Overall Weighted Mean ES
Based on the result of this research, academic achievement overall weighted mean ES is.58; affective outcomes total weighted mean ES is .47; higher level thinking total weighted mean ES is .83. These three domain ES are positive and 95% CI are different from zero, meaning that the learning effect of applying DGBL in academic achievement, affective outcomes and higher level thinking are significantly better than Non-DGBL.
Compare among these three domains, academic achievement and affective outcomes achieved medium effect size level while higher level thinking achieved large effect size level. Though the effect size may be different, one thing is for sure: DGBL benefits higher level thinking the most; academic achievement is second, and the third is affective outcomes. The result confirmed Vygotsky (1979)’s claim that playing offers the cognitive support needed to develop higher order mental processes.
Given that this research is a cross cultural research and adopted world-wide samples, the researcher is concerned about whether other meta-analysis studies showed the same result. Therefore, the research also compared the research result with meta-analysis articles which studied on specific location samples in order to see if the phenomenon is universal and then discussed the findings. The studies of Liao (2010) and Wang (2013) fitted the criteria for they all set on Taiwan samples and their target are students, too.
Compared to Liao (2010) and Wang (2013), the comparison results are as below. First, total weighted mean ES of these three studies are all positive and 95% CI are significantly different from zero, meaning that the effect of applying DGBL on students learning is significantly better than the effect of applying Non-DGBL on students’ learning.
Second, in this research, the learning effect on higher level thinking is the largest; the second is academic achievement, and the third is affective outcomes. Wang (2013) showed the same effective order as this research shows.
Third, compare this research’s total weighted mean ES with Liao (2010) and Wang (2013), the ES of this research is higher than both of them. The study of Liao (2010) only studied on academic achievement (ES =.39), achieving small to medium effect size level. The study of Wang (2013) studied on academic achievement(ES=.40), achieving small to medium effect size level, affective outcomes (ES=.38), achieving small to medium level and higher level thinking(ES=.61), achieving medium to large effect size level while in this research, academic achievement (ES=.58) achieved medium effect size level, affective outcomes (ES=.47) achieved small to medium level and higher level thinking (ES=.83) achieved large effect size level (see Table 5.1).
The effect of applying DGBL on students’ learning in academic achievement, affective and higher level thinking domain is significantly better than the effect of applying Non-DGBL on students’learning in academic achievement, affective and higher level thinking domain confirmed the claim that game, play, and learning have positive relation toward one another.
Skinner (1993) once stated the shortcoming of three types of learning: learning by doing, learning from experience and learning by trial and error to support his claim that the arrangement of learning is important. DGBL needs to arrange the learning environment in advance so the learner can be involved and learn from it. Therefore, DGBL is a combination product of these theories and now proved it really can benefit students’ learning effect and better than Non-DGBL. To educator, this message is important.
Table 5.1
Total Weighted Mean ES comparison
Author
Domain
Liao(2010)
Wang (2013)
This Research
(2015)
Academic Achievement .39 .40 .58
Affective Outcomes — .38 .47
Higher level thinking — .61 .83
Note. The researcher compiled the table. The list only showed the first author’s last name.
2. Moderator Analysis
The researcher integrated academic achievement and affective outcomes homogeneity test results and then compared the moderator homogeneity test with the secondary moderator homogeneity test results and with meta-analysis of Liao (2010) and Wang (2013) and got the following conclusions and discussions:
2.1 Study characteristics 2.1.1 Subject matter
The researcher found that except social study in affective outcomes, the effect of applying DGBL on students’ academic achievement and affective outcomes in other subject matters are significantly better than Non-DGBL. The homogeneity test showed that subject matter moderates both academic achievement and affective outcomes, meaning that the effect of applying DGBL on students’ academic achievement and affective outcomes in different subject matter will have significant difference. Performing DGBL on language and nature and science achieved large effect size level and are significantly better than other subject matters and math has the lowest effect size. In affective outcomes, the result is the same.
Wang (2013) also conducted meta-analysis and researched on the effect of DGBL on 1-12 grades Taiwan students. In her study, subject matter is not the moderator in both academic achievement and affective outcomes while in Liao (2010), his study showed that language has the highest ES and math has the lowest ES. The result of Liao (2010) corresponded to the researcher’s research.
Is it true that some subject matters are specifically suited to game-based learning? According to the research, the answer is yes. Results show that except social study, other kinds of subject matters are significantly suited to digital game-based learning and post hoc shows that language and nature and science are significantly higher than other kind of subject matters.
Though Wang (2013) showed that educational level is not the moderator of both academic achievement and affective outcomes, Liao (2010) showed that language has the highest ES and math has the lowest ES. Wouters (2013) also showed that serious games are particularly effective in language while Randel et al. (1992) found out math, physics, and language arts are particularly suited to game-based learning.
Based on the findings and comparisons above, play indeed constitutes an important part of children’s cognitive and social development (Csikszentmihaly, 1990; Provost, 1990; Rogoff, 1993; Rosas et al., 2003). Piaget (1951) once stated that in the context of cognitive development, playing was considered fundamental to the stabilizing processes that were essential for the development of cognitive structures because it was indiscernible for cognitive development by way of assimilation and accommodation processes. Through this research finding, the relation among game and play and academic achievement learning is confirmed to have positive correlation and proved that Piaget’s theory may be true.
2.1.2 Publication type
The researcher found that except proceedings in affective outcomes, in publication type, the effect of applying DGBL on students’ academic achievement and affective outcomes is significantly better than Non-DGBL.
The homogeneity test showed that publication type does not moderate academic achievement, meaning that the learning effect of applying DGBL in different publication type does not show significant difference. In affective outcomes, only journal shows the result that the learning effect of applying DGBL is
significantly better than Non-DGBL.
Wang (2013) showed that publication type is not the moderator of academic achievement while comparison in affective outcomes is impossible because of sole type publication.
Ellis (2010) once stated that a potential danger in a meta-analysis is the "file drawer problem": the concern that the studies in the meta-analysis are not a correct reflection of all studies that are actually conducted (Ellis, 2010) because studies published in peer-reviewed journals and/or proceedings are more likely to have achieved statistical significance and larger effect sizes than are studies that have not been published (Rosenthal, 1995).
However, the result in this research is sound because the fail-safe N result showed that the publication bias is unlikely to occur though Sitzmann (2011) discovered that the extent to which the simulation game group learned more than the comparison group was greater in published than unpublished studies and Wouters et al. (2013) found out only studies in peer-reviewed journals report higher learning gains for serious games. For proceedings and unpublished papers the effect sizes are even negative.
2.1.3 Publication year
The researcher found that after 2001, the effect of applying DGBL on academic achievement and affective outcomes is significantly better than Non-DGBL.
The homogeneity test showed that publication year moderates academic achievement, meaning that the effect of applying DGBL on students’ academic
achievement in different publication year will have significant difference. The effect of applying DGBL during 2006-2010 and 2011-2015 are significantly better than 2001-2005. Publication year does not moderate affective outcomes, meaning that different publication year will not significantly affect the result.
Wang (2013) showed that publication year is not the moderator of academic achievement and affective outcomes while Liao (2010) showed that the mean ES for studies published before 2000 was significantly higher than the mean ES for studies published in 2000-2005 and 2006-2009. What’s more, the mean ES for studies published in 1993-1999, 2000-2005, and 2006-2009 were all positive and 95% CI significantly different from zero. The research result of Liao (2010) is opposite to this research. Therefore, the result of this research in publications year is totally different from Liao (2010).
Kulik et al. (1980) found that the publication year is related to size of effect:
more recent studies produced results more favorable to programmed instruction.
Kulik et al. (1983) confirmed that effects on final examinations tended to be somewhat higher in more recent studies. In this research, the result is correspondent with theirs: the more recent studies show the more effective result. The result may due to the reason that in recent years, computer and multimedia technologies became more and more advanced and user-friendly.
From the computer and video games sales in the United States in 2012 amounted to $14.8 billion (ESA 2012), the result is not surprising.
2.2 Sample characteristics
2.2.1 Educational level
The researcher found that no matter in which educational level, DGBL is significantly better than Non- DGBL in academic achievement or in affective outcomes. The homogeneity test showed that educational level only moderates academic achievement. Among educational level, the effects of applying DGBL in university, senior high and junior high are significantly better than in elementary. The effects of applying DGBL in university, senior high and junior high achieved medium to large effect size level while elementary is the smallest and only achieved small to medium effect size level. Educational level does not moderate affective outcomes, meaning that different educational level will not significantly affect the result in affective outcomes.
Wang (2013) showed that educational level is not the moderator of academic achievement and affective outcomes while Liao (2010) showed the mean ES for studies conducted in preschools and colleges were significantly higher than studies conducted in elementary and secondary schools. The result of Liao (2010) is different from this research. Though the mean ES for studies conducted in preschools and colleges were significantly higher than studies conducted in elementary and secondary schools, their mean ES are not significantly different from zero, indicating that students’ achievement were not significantly better than non-DGBL in these two grade levels.
Kulik (1981) argued that individualized instruction has different effects on different people and suggested that at the lower levels of instruction, learners need the stimulation and guidance provided by a highly reactive teaching medium; at the upper levels of instruction, a highly reactive instructional medium may not only be unnecessary but may even get in the way. Vogel et al.
(2006), however, did not find differences between age groups in learning with serious games.
The result of this research is opposite to Kulik (1981) and Vogel et al. (2006).
The result of this research may cause some doubt because people might naturally think playing works best for children rather than adult. However, think twice about Piaget’s stages of cognitive development theory, the reason all comes clear.
Playing no doubt is a privileged learning experience. However, DGBL on academic achievement is in large portion an experience of formal operation, which in cognitive development theory it normally happened in elementary higher grader. Since in Piaget’s stages of cognitive development theory, each stage must be completed before proceeding to the next stage and this is not automatic, the result of this research makes sense because academic achievement is a logic cognitive learning activity. Therefore, when it comes to DGBL on academic achievement, that elementary has the lowest effect size makes sense for elementary includes all four stages of Piaget’s stages of cognitive development theory while other educational levels all achieved formal operation.
2.2.2 Sample location
The researcher found that except South America (k=2) in affective outcomes, other sample locations data showed that the effect of applying DGBL on academic achievement and affective outcomes is significantly better than Non-DGBL. The moderator homogeneity test showed that sample location
moderates both academic achievement and affective outcomes, meaning that the effect of applying DGBL on different sample location will have significant difference.
In academic achievement, Asia and North American is significantly better than Europe; in affective outcomes, Asia is significantly better than North America and Europe.
Wang (2013) showed that sample location is not the moderator of academic achievement but it is the moderator of affective outcomes. Because Wang (2013) only focused on Taiwan, the comparison between this research and Wang (2013) is impossible. Wang (2013) showed that in affective outcomes, Northern Taiwan is significantly better than Southern Taiwan.
From the result above, geopolitical contexts and cultural differences really makes a big difference. In this research, all articles gathered in Asia, Europe and South America while the data of Africa, South America and Oceania were few;
therefore, “digital divide” may exists. However, due to the reason that this research only included articles which were written in English, the conclusion is not confirmed.
2.3 Research design characteristics 2.3.1 Instrumentation
The researcher found that no matter which instrumentation is used, the learning effect of DGBL is significantly better than Non–DGBL in both academic achievement and affective outcomes. The moderator homogeneity test showed
that instrumentation moderates both academic achievement and affective outcomes, meaning that the effect of DGBL in different instrumentation will have significant difference.
In academic achievement, self-compiled test achieved medium effect size level and is significantly higher than standardized test, which achieved small to medium effect size level. However, affective outcomes shows opposite result:
standardized test achieved medium effect size level and is significantly higher than self-compiled test, which achieved small to medium effect size level.
Wang (2013) showed that instrumentation is not the moderator of affective outcomes and the instrumentation comparison in academic achievement is impossible due to sole type of instrumentation.
Observing the meta-analysis the researcher has found, none listed out what kind of instrumentation the data they collected contained. However, according to Dede et al. (2005) and Belanich et al. (2004), they did notice and argue about different type of instrumentation may affect the effect of learning outcome. This research not only proved their argument that different type of instrumentation may affect the effect of learning outcome, moreover, this research also brought out the issue to be cautious about different instrumentation may have different effect on difference domain.
2.3.2 Instructor bias
The researcher found that except CT and GT in affective outcomes, other instructor bias data shows that the learning effect of DGBL is significantly
better than Non–DGBL in both academic achievement and affective outcomes.
The moderator homogeneity test showed that instructor bias moderates both academic achievement and affective outcomes, meaning that the effect of DGBL in different instructor bias will have significant difference.
In academic achievement, CT and GT and CNT and GT achieved medium to large effect size level and are significantly higher than ST and DT. Besides, ST is significantly higher than DT. In affective outcomes, CT and GT shows the result that DGBL is not significantly better than Non-DGBL while ST achieved large effect size and is significantly higher than CNT and GT and DT. CNT and GT achieved medium effect size level, and DT achieved small effect size level.
Wang (2013) showed that instructor bias only moderates academic achievement.
CNT and GT is significantly better than ST and DT, and ST is significantly better than DT. The result of Liao (2010) also corresponded to the researcher’s research. Therefore, same teacher is better than different teacher is confirmed and control group with no teacher and game as teacher is better than different teacher is confirmed.
Sitzmann (2011) argued about the role teacher plays in comparison group is equally important as in experiment group and Kulik et al. (1983) thought that the game implementation is as supplement or as substitute will affect the role teacher plays. From the result, teacher will affect the result because different teacher may have different style and affect the result and the role teacher plays may affect the learning outcomes.
2.3.3 Experiment design
The researcher found that except POC, other experiment designs data show that the learning effect of DGBL is significantly better than Non-DGBL in both academic achievement and affective outcomes. The moderator homogeneity test showed that experiment design moderates both academic achievement and affective outcomes, meaning that the effect of DGBL in different experiment design will have significant difference.
In academic achievement, OPP is significantly higher than other kind of experiment design and achieved large effect size level. In affective outcomes, OPP and NPP achieved medium effect size level and significantly higher than PPC, which achieved small effect size level.
Wang (2013) showed that OPP is significantly better than NPP and Liao (2010) also showed that OPP is significantly better than PPC and NPP. Their results corresponded to the researcher’s academic achievement research: OPP is significantly better. From the result, OPP achieved highest ES compared to other type of research design.
Wouters et al. (2013) found that the beneficial effect of serious games is contingent on the experimental rigor: random assignment decreased the effect of
Wouters et al. (2013) found that the beneficial effect of serious games is contingent on the experimental rigor: random assignment decreased the effect of