2. Academic achievement analysis
2.2 Moderator analysis on academic achievement
The homogeneity statistic is significant on academic learning; therefore, it is necessary to examine the possible moderators. The researcher separated moderators into six categories: study characteristics, sample characteristics, research design characteristics, program characteristics, digital game categories, and digital game characteristics to perform the analysis.
2.2.1 Examination of study characteristics on academic achievement
The researcher separated study characteristics into three moderators: subject matter, publication Type and publication year and then examined possible moderators (see Table 4.2).
(1) Subject matter
The researcher separated subject matter into language, math, nature and science, social study, health and PE, computer science, and others. Research results showed that each subject matter weighted mean ES is positive and 95% CI is significantly different from zero, meaning that no matter in which subject matter,
the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because subject matter (QB = 88.60, p<.05) achieved significant level, it meant that the moderator affected academic achievement. The post hoc showed that the effects of language (ES=.84) and nature and science (ES=.81) are significantly higher than other kind of subject matters while math (ES=.27) is the smallest.
Except social study (Qw =1.49, p>.05), each subject matter Qw achieved significant level. The results showed that there are heterogeneous inside each group, meaning that unknown moderators still exist and need further analysis.
Therefore, the researcher will perform secondary moderator cross analysis.
(2) Publication type
The researcher separated publication type into journal and proceedings.
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in both journal and proceedings, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’
academic achievement.
Because publication type (QB = 1.21, p＞.05) is insignificant, it meant that different publication type will not affect the result. Both Journal and proceedings Qw are significant (p<.05), meaning that unknown moderators still
exit.
(3) Publication year
The researcher separated publication year into 1981-1985, 1986-1990, 1991-1995, 1996-2000, 2001-2005, 2006-2010, and 2011-2015. Due to the reason that the data before 2000 are few (k=7), these period will not be discussed. Research results showed that 2001-2005, 2006-2010, 2011-2015 weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in these periods, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because publication year (QB = 38.80, p<.05) achieved significant level, it meant that it is the moderator of academic achievement. Post hoc showed that 2011-2015 (ES=.57) and 2006-2010 (ES=.54) are significantly higher than 2001-2005 (ES=.33). Each publication year Qw is significant (p<.05), meaning that unknown moderators still exit.
Table 4.2
Note. *: significant at the .05 level; d+:
weighted mean effect size; k: number of articles; CI: confidence interval; #: d
+positive and 95% CI significantly different from zero ;(): post hoc.
2.2.2 Examination of sample characteristics on academic achievement
The researcher separated sample characteristics into two moderators:
educational level and sample location, and then examined possible moderators.
(1) Educational level
The researcher separated educational level into elementary, junior high, senior
high, and university. Research results showed that each weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in elementary, junior high, senior high, and university, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because educational level (QB = 46.59, p<.05) achieved significant level, it meant that educational level is the moderator of academic achievement. Post hoc showed that the effects on university (ES=.70), senior high (ES=.60) and junior high (ES=.58) are significantly better than elementary (ES=.37)
Each educational level Qw is significant (p<.05), meaning that unknown moderators still exit and further analysis is necessary. Therefore, the researcher will perform secondary cross analysis.
(2) Sample location
The researcher separated sample location into Asia, Europe, Africa, South America, North America, and Oceania. Due to the reason that the data of Africa, South America and Oceania are few, Africa, South America and Oceania will not be discussed. Research results showed that Asia, Europe and North America weighted mean ES are positive and 95% CI are significantly different from zero in sample location, meaning that in Asia, Europe and North America, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because sample location (QB =81.27, p<.05) achieved significant level, it meant
that sample location is the moderator of academic achievement. Post hoc showed that the effects on Asia (ES=.77) and North America (ES=.70) are significantly higher than Europe (ES=.26).
Asia, Europe and North America Qw are significant (p<.05), meaning that unknown moderators still exit and further analysis is necessary. Therefore, the researcher will perform secondary cross analysis.
Note. *: significant at the .05 level; d+:
weighted mean effect size; k: number of articles; CI: confidence interval; #: d
+positive and 95% CI significantly different from zero; (): post hoc.
2.2.3 Examination of research design characteristics on academic achievement
The researcher separated research design characteristics into three moderators:instrumentation, instructor bias and experiment design, and then examined possible moderators.
(1) Instrumentation
The researcher separated instrumentation into self-compiled test and standardized test. Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in self-compiled test and standardized test, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because instrumentation (QB = 13.05, p<.05) achieved significant level, it meant that instrumentation is the moderator of academic achievement, and that self-compiled test (ES=.56) is significantly better than standardized test (ES=.32). Each instrumentation Qw is significant (p<.05), meaning that unknown moderators still exit.
(2) Instructor bias
The researcher separated instructor bias into the same teacher (ST), different teacher (DT), mixture (MI), control group has no teacher involved and game as teacher in the experimental group (CNT>), control group has teacher involved and game as teacher in the experimental group (CT>), and unspecified. The MI data is none, so it will not be discussed. Research results showed that each weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in ST, DT, CNT> and CT>, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because instructor bias (QB = 84.03, p<.05) achieved significant level, it meant that instructor bias is the moderator of academic achievement. Post hoc
showed that CT and GT (ES=.73) and CNT and GT (ES=.67) are significantly higher than ST (ES= .48) and DT (ES=.27); ST is significantly higher than DT.
(3) Experiment design
The researcher separated experiment design into one-group pretest posttest design (OPP), pretest-posttest control group design (PPC), posttest-only control group design (POC), the nonequivalent pretest-posttest design (NPP), the nonequivalent posttest only design (NPO) and unspecified.
Research results showed that except POC, each weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in OPP, PPC, NPP, NPO ,the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’
academic achievement.
Because experiment design (QB = 68.55, p<.05) achieved significant level, it meant that experiment design is the moderator of academic achievement. Post hoc showed that OPP (ES=1.03) is significantly higher than other kind of experiment designs; NPO (ES=.79) is significantly higher than PPC (ES=.47) and NPP (ES=.45).
Table 4.4
Note. *: significant at the .05 level; d+:
weighted mean effect size; k: number of articles; CI: confidence interval; #: d
+positive and 95% CI significantly different from zero; (): post hoc.
2.2.4 Examination of program characteristics on academic achievement
The researcher separated program characteristics into four moderators: duration of treatment, purpose of treatment, group size in experiment group and strategy involved, and then examined possible moderators.
(1) Duration of treatment
The researcher separated duration of treatment into less than 7 days, 8~30 days, above 31 days and unspecified. Research results showed that each weighted
mean ES is positive and 95% CI is significantly different from zero, meaning that in less than 7 days, 8~30 days, above 31 days and unspecified, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement..
Because duration of treatment (QB = 793.34, p<.05) achieved significant level, it meant that duration of treatment is the moderator of academic achievement.
However, the conclusion is not confirmed because unspecified (ES=.81) is significantly higher than other duration of treatment while others do not have significant difference. Each duration of treatment Qw is significant (p<.05), meaning that unknown moderators still exit.
(2) Purpose of treatment
The researcher separated purpose of treatment into superior, complement, and replace. Research results showed that each weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in superior, complement, and replace, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement..
Because purpose of treatment (QB = 7.59, p<.05) achieved significant level, it meant that purpose of treatment is the moderator of academic achievement. Post hoc showed that superior (ES=.56) is significantly higher than replace (ES=.30).
Each purpose of treatment Qw is significant (p<.05), meaning that unknown moderators still exit.
(3) Group size in experiment group
The researcher separated group size in experiment group into individual and group. Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in both individual and group, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’
academic achievement.
Because group size in experiment group (QB = 24.94, p<.05) achieved significant level, it meant that group size in experiment group is the moderator of academic achievement and that individual (ES=.57) is significantly higher than group (ES=.24). Each group size in experiment group Qw is significant (p<.05), meaning that unknown moderators still exit.
(4) Strategy involved
The researcher separated strategy involved into both does not have strategy (BNS), both have strategies (BHS), control group has strategy but experimental group doesn’t apply any strategy (CHEN), and experimental group has strategy but control group has no strategy (EHCN).
Research results showed that excluding BHS (k=0), only BNS weighted mean ES is positive and 95% CI is significantly different from zero, meaning that if both groups have no strategy involved, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement. However, the conclusion is not confirmed because the article numbers are seriously
unbalanced (both CHEN and EHCN are below five articles). Each strategy involved Qw is significant (p<.05), meaning that unknown moderators still exit.
Table 4.5
Results of Program Characteristics Examination
Q
B k d+ 95%CIQ
wDuration of treatment 793.34* 76
1.Under 7 days 31 .41# [.34,.49] 335.06*
2.8~30 days 17 .48# [.39,.58] 262.75*
3.Above 31 days 12 .45# [.33,.56] 30.52*
9.Unspecified 16 .81# [.71,.90] 111.43*
Purpose of treatment 7.59* 76
(1> 3)
1. Superior 62 .56# [.51,.60] 624.94*
2. Complement 8 .51# [.37,.64] 145.11*
3. Replace 6 .30# [.06,.53] 15.70*
Group size in experiment
group 24.94* 76
1. Individual 66 .57# [.53,.62] 733.51*
2. Group 10 .24# [.09,.38] 34.89*
Strategy involved 73.60* 76
1. BNS 71 .53# [.48,.57] 705.99*
3. CHEN 2 2.36 [-.50, 5.21] 11.92*
4. EHCN 3 .2 [-.33,.72] 1.84
Note. *: significant at the .05 level; d+:
weighted mean effect size; k: number of articles; CI: confidence
interval; #: d
+positive and 95% CI significantly different from zero; (): post hoc.
2.2.5 Examination of digital game categories on academic achievement
The researcher separated digital game categories into action games, adventure games, fighting games, puzzle games, role-playing games, simulation games, sports games, strategy games, and unspecified. Due to the reason that the data of action games, adventure games, fighting games and sports games are few; these game categories will not be discussed.
Research results showed that each digital game categories weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in puzzle games, role-playing games, simulation games, strategy games,, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because digital game categories (QB = 79.04, p<.05) achieved significant level, it meant that digital game categories is the moderator of academic achievement.
Post hoc showed that the effects of role-playing (ES=.98) is significantly higher than simulation games (ES=.74), strategy games (ES=.46) and puzzle games (ES=.36); simulation games is significantly higher than puzzle games and strategy games.
Each digital game categories Qw is significant (p<.05), meaning that unknown moderators still exit and further analysis is necessary. Therefore, the researcher will perform secondary cross analysis.
Table 4.6
Results of Digital Game Categories Examination
Q
B k d+ 95%CIQ
wDigital game categories 79.04* 71
(5>4,6,8; 6>4,8)4. Puzzle games 34 .36# [.30, .42] 298.93*
5. Role-playing games. 6 .98# [.71, 1.24] 86.97*
6. Simulation games 21 .74# [.66, .83] 161.45*
8. Strategy game 10 .46# [.32, .61] 67.31*
Note. *: significant at the .05 level; d+:
weighted mean effect size; k: number of articles; CI: confidence interval; #: d
+positive and 95% CI significantly different from zero ;(): post hoc.
2.2.6 Examination of digital game characteristics on academic achievement
The researcher separated digital game characteristics into intrinsic vs. extrinsic, tightly linked vs. loosely linked, hard-wired vs. engines and templates or shells, reflective vs. action, synchronous vs. asynchronous, single-player vs.
multiplayer, session-based games vs. persistent-state, video-based vs.
animation-based and narrative-based vs. reflex-based and then examined possible moderators.
(1) Intrinsic vs. Extrinsic
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in intrinsic or extrinsic, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because intrinsic vs. extrinsic (QB =20.64, p<.05) achieved significant level, it meant that intrinsic vs. extrinsic is the moderator of academic achievement, and
that intrinsic (ES=.69) is significantly better than extrinsic (ES=.47).
(2) Tightly linked vs. Loosely linked
Among 76 articles, there are 74 articles studying on tightly linked and two articles studying on loosely linked. Because loosely linked only has two article, and the 95% CI is not different from zero, the comparison is impossible.
(3) Hard-wired vs. Engines and templates or shells
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in hard-Wired or Engines and Templates or Shells, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because hard-wired vs. engines and templates or shells (QB = 33.57, p<.05) achieved significant level, it meant that hard-wired or engines and templates or shells is the moderator of academic achievement, and that engines and templates or shells (ES= .61) is significantly higher than hard-wired (ES=.32).
(4) Reflective vs. Action
All 76 articles belong to Reflective. Therefore, comparison is impossible.
(5) Synchronous vs. Asynchronous
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in synchronous or asynchronous, the effect of applying DGBL on students’ academic
achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because synchronous vs. asynchronous (QB =19.59, p<.05) achieved significant level, it meant that synchronous vs. asynchronous is the moderator of academic achievement, and asynchronous (ES=.78) is significantly better than synchronous (ES=.50).
(6) Single-player vs. Multiplayer
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in single-player or multiplayer, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’
academic achievement.
Because single-player vs. multiplayer (QB =17.88, p<.05) achieved significant level, it meant that single-player vs. multiplayer is the moderator of academic achievement, and multiplayer (ES=.79) is significantly better than single-player (ES=.50).
(7) Session-based games vs. Persistent-state games
Research results showed that both weighted mean ES are positive and 95% CI are significantly different from zero, meaning that in session-based games or persistent-state games, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Because session-based games vs. persistent-state games (QB = 6.20, p<.05) achieved significant level, it meant that session-based games vs.
persistent-state is the moderator of academic achievement, and session-based (ES=.59) is significantly better than persistent-state games (ES=.48).
(8) Video-based vs. Animation-based
All 76 articles belong to Animation-Based. Therefore, comparison is impossible.
(9) Narrative-based vs. Reflex-based
Only narrative-based weighted mean ES is positive and 95% CI is significantly different from zero, meaning that in narrative-based game, the effect of applying DGBL on students’ academic achievement is significantly better than the effect of applying Non-DGBL on students’ academic achievement.
Table 4.7
Results of Digital Game Characteristics Examination
Q
B k d+ 95%CIQ
wDigital game characteristics
Intrinsic vs. Extrinsic 20.64* 76
1. Intrinsic 24 .69# [.61,.78] 218.64*
Synchronous vs. Asynchronous 19.59* 76
1. Synchronous 63 .50# [.45,.545] 559.71*
2. Asynchronous 13 .78# [.65,.91] 214.05*
Single-Player vs. Multiplayer 17.88* 76
1. Single-Player 67 .50# [.46,.55] 696.04*
Note. *: significant at the .05 level; d+: