Game-based learning verse traditional instruction on student

Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
Game-based learning verse traditional instruction on student affective
outcomes in Taiwan: A meta-analysis
Yuen-kuang Cliff Liao
Department of Education
National Taiwan Normal University, Taiwan, R.O.C
yliao@ntnu.edu.tw
Abstract
A meta-analysis was performed to synthesize existing research comparing the effects of game-based
learning (GBL) versus traditional instruction (TI) on students’ affective outcomes in Taiwan. Twenty-six studies
were located from five sources, and their quantitative data were transformed into Effect Size (ES). The overall
grand mean of the study-weighted ES for all 26 studies was 0.64. The results suggest that GBL has a greater
effect than TI on promoting student affective outcomes in Taiwan. In addition, five of the twelve moderator
variables selected for this study had a statistically significant impact on the mean ES. The results from this study
suggest that the effects of GBL are positive compared to TI. The analyses of moderator variables also provided
some valuable implications for verifying the effects of implementing game-based learning in educational
settings.
Keywords: game-based learning, attitudes, affective outcomes, meta-analysis
* Corresponding author.
28
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
1. Introduction
The educational potential of games has been
claimed by researchers and educators for quite a long
time. Piaget [1], for instance, stated that the process of
playing games could help children master the
environments that they live in and create their worlds
of imagination. Others also claimed that games could
support, reinforce and accelerate the learning process,
and support higher-order cognitive development [2],
[3], [4], [5]. Walliser [6] believed that game-based
learning stimulated critical thinking, information
gathering and sharing, and collective problem solving.
In fact, there was a fairly wide consensus in the game
study literature that the values of discovery and
achievement were fundamental to the nature of games
[7].
Researchers have also argued that game-based
learning can be more enjoyable, more interesting, and,
thus, more effective than traditional learning modes
[8], [9] ,[10]. Papastergiou [11], for example,
addressed the potential benefits of computer and video
games: (a) they can support multi-sensory, active,
experiential, problem-based learning, (b) they favor
activation of prior knowledge given that players must
use previously learned information in order to advance,
(c) they provide immediate feedback enabling players
to test hypotheses and learn from their actions, (d)
they encompass opportunities for self-assessment
through the mechanisms of scoring and reaching
different levels, and (e) they are increasingly
becoming social environments involving communities
of players. (p.603)
In addition, several studies have found that
educational games could promote students’ affective
outcomes, such as attitudes, interests and motivation
to learning or specific subject matters. Virvou,
Katsionis, and Manos [12], for example, designed
VR-ENGAGE computer game for teaching geography
to fourth grade students and examined its
effectiveness. Results showed that VR-ENGAGE was
more effective when compared to educational
software without game characteristics; in addition,
poor performing students benefited more from the
game environment than the good performing students.
Tüzün, Yılmaz-Soylu, Karakus, Inal, Kızılkaya [13]
designed a three-dimensional educational computer
game for 24 students in fourth and fifth grades to
examine their achievement and motivation. Results
indicated that students made significant learning gains,
higher intrinsic motivations and lower extrinsic
motivations by participating in the game-based
learning. Liu and Chu [14] investigated how
ubiquitous games influence English learning
achievement and motivation through a context-aware
ubiquitous
learning
environment.
Results
demonstrated that incorporating ubiquitous games into
the English learning process could achieve a better
learning outcomes and motivation than using
non-gaming method. A positive relationship between
learning outcomes and motivation was also found.
29
1.1 Purposes of study
In spite of claims regarding the potential benefits
of using games in fostering students’ affective
outcomes, research results comparing the effects of
game-based learning and traditional instruction on
students’ affective outcomes in Taiwan are conflicting.
For example, Cheng [15], Huang [16], Ke [17], Lin
[18], Wang [19], and Wu [20] all report significant
gains for game-based learning over traditional
instruction. On the other side, Chen [21], Lai [22], Lin
[23], Lo [24], and Wang [25] have found no
significant differences between game-based learning
and traditional instruction. Owing to the contradictory
evidence provided by existing research in the area, and
very little, if any, thorough quantitative synthesis of
game-based learning in Taiwan has been done, it is
important to conduct a meta-analysis to clarify the
research conclusions.
2. Procedures
The research method used in this study is a
meta-analytic approach similar to that suggested by
Hedges and Olkin [26]. The purpose of this study was
to synthesize and analyze the research on the effects of
two instructional approaches. It is important to define
these approaches so as to provide for selection of
appropriate studies:
 Game-based Learning (GBL) -- classes using
games (including non-computer games,
computer-based games , and web-based
games) as instructional approaches to teach
students, and

Traditional Instruction (TI) -- classes using
traditional classroom instruction to teach
students.
2.1 Data Sources
The studies considered for use in this
meta-analysis came from five sources. One large
group of studies came from computer searches of the
Index to Taiwan Periodical Literature System. A
second group of studies came from the National
Digital Library of Thesis and Dissertation in Taiwan
(NDLTD). A third group of studies was retrieved from
the Government Research Bulletin (GRB) of Taiwan.
The fourth group of studies was from the Educational
Document Online. The last group of studies was
retrieved by branching from bibliographies in the
documents located through review and computer
searches.
Twenty-six studies were located through these
search procedures; 23 studies came from the National
Digital Library of Thesis and Dissertation in Taiwan,
and only 3 studies were retrieved from published
journals. According to Glass, McGaw, and Smith [27],
using only published research in a meta-analysis can
inflate the mean Effect Size (ES). In their
meta-analysis on computer-assisted instruction (CAI),
Christman, Badgett, and Lucking [28] included 13
dissertations and 2 unpublished papers in an effort to
reduce the inflation of ES resulting from publication
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
bias. However, since the current study retrieved more
than 90% of its studies from dissertations/theses that
may cause the so-called publication bias, Rosenthal’s
[29] Fail-safe Number was calculated. The result
shows that Fail-safe Number was larger than the
Tolerance level, indicating that publication bias in the
present synthesis does not exist.
Several criteria were established for inclusion of
studies in the present analysis:
1. Studies had to compare the affective
outcomes (e.g., motivation, attitudes,
and interests toward learning or/and
subject matters) of GBL vs. TI on
students’ achievement.
2. Studies had to provide quantitative
results from both GBL and TI classes
so that the ES could be estimated.
3. Studies had to be retrievable from
university or college libraries by
interlibrary loan or from GRB, or
Dissertation and Thesis Abstract
System of Taiwan.
4. Studies had to use Taiwanese students
as subjects.
There were also several criteria for eliminating studies
or reports cited by other reviews: (a) studies did not
report sufficient quantitative data in order to estimate
Effect Size; (b) studies reported only correlation
coefficients -- r value or Chi-square value; (c) studies
could not be obtained through interlibrary loans or
from standard clearinghouses.
But in some cases, when more than one value was
available for use in the formula of ES, the value which
measured outcomes most correctly was selected. For
example, some studies reported both differences on
posttest measures and differences in pre-post gains,
and some studies reported both raw-score differences
between groups and covariance-adjusted differences
between groups. In such cases, pre-post gains and
covariance-adjusted differences were selected for
estimating ES.
In addition, several subscales and subgroups
were used in measuring a single outcome (e.g., those
that reported separate data by gender or grade). In
such cases, each comparison was weighted in inverse
proportion to the number of comparisons within the
study (i.e., 1/n, where n = number of comparisons) so
that the overweighing of ES of a study could be
avoided (see, for example, [31], p. 230).
2.3 Study Features Coding
Twelve study features were coded for each study
in the present synthesis. These variables are listed in
Table 1. Each of these study features was placed in
one of the following set of characteristics: (a) study
characteristics, (b) methodological characteristics, and
(c) program characteristics. The first two study
features in the study characteristics were coded so that
potential different effects for subjects with different
backgrounds could be detected. The other two study
features (i.e., type of publication and year of
publication) in the study characteristics were coded
because it is important to know how effects are related
to sources of information over time. Three study
features placed in the methodological characteristics
were coded so that effects related to characteristics of
research procedures could be detected. The last five
study features in the program characteristics were
coded because it is critical to know how effects are
related to nature and design of the primary study.
2.2 Outcome Measures
The affective outcomes measured most often in
the 26 Studies was student attitudes/interest toward
learning,
as
indicated
on
standard
or
researcher-develop affective questionnaire at the end
of the program of instruction. For statistical analysis,
outcomes from a variety of different studies with a
variety of different instruments had to be expressed on
a common scale. The transformation used for this
purpose was the one recommended by Hedges and
Olkin [26]. To reduce measurements to a common
scale, each outcome was coded as an Effect Size (ES),
defined as the difference between the treatment and
control means, divided by the pooled standard
deviation. Furthermore, when a sample size in a study
is small, Hedges’s g+ (unbiased estimate of ES) was
calculated to remove possible sample bias (Hedges
and Olkin, [26], p.81). For those studies that did not
report means and standard deviations, F values or t
values were used to estimate the ES. Also, in studies
which used one-group pretest-posttest design, in
which a control group did not exist, an alternative
approach suggested by Andrews, Guitar, and Howie
[30] was used. In their approach, the ES is estimated
by comparing the post-treatment mean with the
pre-treatment mean, and dividing by pooled standard
deviation.
In most cases, the application of the formula
given by Hedges and Olkin was quite straightforward.
Table 1: The Assignments of Study
Features in Each Characteristic
Characteristics
Study Characteristics
Variables
Grade Level
Subject Area
Type of Publication
Year of Publication
Methodological Characteristics
Instructor Bias
Sample Size
Type of Research Design
Program Characteristics
Duration of Treatment
Purpose of Instruction
Group Size for Treatment
Occasion of Treatment
Type of Game
___________________________________________
30
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
mean weighted ES was converted to percentiles,
the percentiles on students' cognitive achievement
were 74 for the GBL group and 50 for the TI group.
The overall grand median for all 26 ESs was 0.56,
suggesting that percentiles on students' achievement
were 71 for the GBL group and 50 for the TI group.
The standard deviation of 0.53 reflects the medium
variability of ESs across studies. The results indicate
that, on average, there was a medium significant effect
on affective outcome; GBL group had a significant
higher effect than TI group.
The results of this meta-analysis indicate that
GBL has moderately positive effects on students’
affective outcomes over TI in Taiwan. An effect is said
to be medium when ES = 0.5 and large when ES = 0.8
[32]. The effectiveness of GBL was also confirmed by
its 92% positive study-weighted ES values and
significant Z-value. With nearly2400 subjects included
in this meta-analysis, the generalization of this study
is considered stable. The moderateness of the effect
must be kept in mind since the overall study-weighted
mean ES of 0.64 only indicates 24 percentile scores
higher than the TI group.
The homogeneity statistics (QT = 103.35, df=25,
p<.0001), however, indicate that findings on affective
outcome were significantly heterogeneous, suggesting
that the study-weighted mean ES of 0.64 may not be
representative of the findings integrated and that other
study features may moderate the magnitude of the ESs.
A series of subgroup analyses of the moderator
variables were then conducted.
2.4 Data Analysis
For the total set of 26 studies being investigated,
Hedges and Olkin’s [26] homogeneity procedures
were employed in aggregating and analyzing the effect
sizes. Each effect size was weighted by the inverse of
its sampling variance. Thus, more weight was given to
findings that were based on larger sample sizes. The
weighted ESs were then aggregated to form an overall
weighted mean estimate of the treatment effect (d+).
The significance of the mean ES was judged by its
95% confidence interval (95% CI). A significantly
positive (+) mean ES indicates that the results favor
GBL; a significantly negative (–) mean ES indicates
that the results favor TI.
To determine whether the findings in each
dataset shared a common ES, the set of ESs was tested
for homogeneity by the homogeneity statistic (QT).
When all findings share the same population ES, QT
has an approximate distribution with k – 1 degrees of
freedom, where k is the number of ES. If the obtained
QT value was larger than the critical value, the
findings were determined to be significantly
heterogeneous, meaning that there was more
variability in the ESs than chance fluctuation would
allow [26].
Next, a series of subgroup moderate variable
analyses were conducted. A mixed effects model was
used for these analyses to model within-group
variation. A between-group heterogeneity statistic
(QBetween) was computed to test for statistical
differences in the weighted mean ESs for various
subsets of the effects (e.g., studies implemented in a
large group as opposed to a small group).
3.2 Study features analyses
able 3 presents the results of the homogeneity
statistics (QT) analysis for individual study features.
Of the 12 study features analyzed,5 moderate
variables were T significantly related to the variability
in the affective outcome. The analysis of each of the
significant study features is described in the following
section.
2.5 Coder Reliability
To obtain more reliable outcomes from coding,
the researcher of this study and two research assistants
coded the studies. Each of the two research assistants
coded half of the studies on each of the study features.
As a check for accuracy, the researcher coded each of
the studies independently. The inter-coder agreement
rate for ES calculation and study feature coding were
85.3% and 84.6%, respectively. In addition,
disagreements between the two coders were resolved
through discussion. A final agreement had to be
reached after discussion.
3.2.1 Instructor bias.
For instructor bias, studies were grouped into
two subsets, same instructor and different instructor,
denoting whether the same or different teachers taught
both the GBL and TI classes. After reviewing several
meta-analysis of media comparison research, Clark
[49] suggested that the positive effects of media
seemed to be the uncontrolled effects of instructional
method or content differences between treatments that
were compared; he concluded that effects more or less
disappeared when the same instructor delivered all
treatments. For the present meta-analysis, the mean
ES for studies using different instructors was
significantly higher than studies using sameinstructors
for treatments (QB = 13,05, df = 3, p < .05). The
findings do not seem to support Clark’s view. The
mean ESs for both subsets were all positive and
significantly different from zero (95% CI are 0.32 to
0.58, and 0.64 to 0.89, respectively), indicating that
students’ affective outcomes in GBL were
3. Results and Discussion
3.1 Overall effects
In all, 26 studies representing 2345 students
were analyzed in this meta-analysis. Table 2 illustrates
the 26 ESs derived from the 26 studies. Of the 26 ESs
included in the present synthesis, 24 (92%) of the
study-weighted ESs were positive and favored the
GBL group, while 2(8%) of them were negative and
favored the TI group. The range of the study-weighted
ESs was from -0.08 to 2.61. The mean weighted ES
(d+) was 0.64 (95% CI is 0.552to 0.721). When this
mean weighted ES was converted to a Z-value, the
Z-value was 14.83 (p < .0001). In addition, when this
31
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
Table 2: Descriptive data of 26 ESs
Author(s)
Year
g+
Chen [21]
2006
Chen [33]
2007
Cheng [34]
2009
Huang [16]
2009
Hung [35]
2006
Kao [36]
2009
Ke [17]
2006
Lai [22]
2005
Lai [37]
2007
Li [38]
2009
Lin [23]
2004
Lin [39]
2008
Lin [40]
2009
Lin [18]
2010
Liu [41]
2005
Lo [24]
2007
Pan et al. [42]
2003
Shu [43]
2007
Shyu [44]
2006
Su et al. [45]
2005
Tseng [46]
2007
Wang [25]
1998
Wang et al. [47]
2005
Wang [19]
2005
Wang [48]
2007
Wu [20]
2007
Mean ES (d+)
Standard Deviation
N of study
Homogeneity Statistic（QT）
Effect size
SE
0.351
0.277
0.435
0.244
0.895
0.291
1.288
0.263
0.683
0.373
0.730
0.313
1.428
0.185
0.064
0.172
0.549
0.129
0.518
0.359
-0.075
0.265
0.338
0.187
0.349
0.248
2.611
0.357
0.738
0.146
-0.004
0.196
0.389
0.176
0.566
0.400
0.878
0.288
0.419
0.266
0.522
0.250
0.319
0.430
0.991
0.148
0.806
0.201
0.809
0.177
0.804
0.307
0.636
0.043
95% CI
L. Limit
-0.191
-0.042
0.325
0.773
-0.048
0.117
1.066
-0.272
0.295
-0.187
-0.595
-0.029
-0.137
1.910
0.452
-0.389
0.044
-0.218
0.314
-0.102
0.031
-0.523
0.700
0.412
0.462
0.203
0.552
Z-Value
U.Limit
0.894
1.269
0.912
1.787
1.465
3.077*
1.803
4.904*
1.413
1.832
1.343
2.336*
1.791
7.723*
0.401
0.374
0.802
4.244*
1.222
1.440
0.444
-0.284
0.704
1.805
0.835
1.407
3.311
7.305*
1.023
5.065*
0.381
-0.020
0.735
2.208*
1.350
1.416
1.441
3.050*
0.941
1.577
1.013
2.085*
1.161
0.742
1.282
6.683*
1.201
4.007*
1.156
4.565*
1.405
2.622*
0.721
14.832*
N
53
69
52
70
30
44
149
138
249
16
58
116
66
60
218
104
131
26
53
58
66
24
204
107
138
46
2345
0.53
26
219.426*
*p<.05
Furthermore, the tests of Qw-statistics for studies
significantly positive compared to TI regardless of
employed pretest-posttest control group design,
whether the treatment used the same or different
nonequivalent control group design, and posttest only
instructor. The tests of Qw-statistics for the two subsets
control group design were all heterogeneous,
of studies were significantly heterogeneous,
indicating that the mean ESs may not be
suggesting that the mean ES may not be representative
representative of the findings integrated and that other
of the study's findings and that other study features
study features may moderate the magnitude of the
may influence the magnitude of the ESs.
ESs.
3.2.2 Type of research design
3.2.3 Group size for treatment
For type of research design, studies were divided into
The size of groups used in a collaborative learning has
four subsets: one group repeated measure design,
often been discussed. Several studies reported that
pretest-posttest control group design, nonequivalent
small groups function better because the members can
control group design, and posttest only control group
interact more intimately and cohesively ([50], [51]).
design. More than half of studies used nonequivalent
Harasim [52] also claimed that groups of 2–4 people
control group design. The mean ES for studies
are preferred by students for online group work for
employed posttest only control group design was
task-oriented activities. Of the 26 Studies in the
significantly higher
than
studies
employed
present synthesis, 9 (35%) of the ESs employed large
pretest-posttest
control
group
design
and
group (over 5 people) instruction for the GBL, 5 (19%)
nonequivalent control group design (QB = 33.13, df =
of the ESs used a small group (2-4 people), while 4
3, p<.05). Also, the mean ESs for studies for all four
(15%) of the ESs used individual setting; there were 8
subsets were positive and significantly different from
(31%) ESs that did not report group size. The test of
zero, indicating that students’ affective outcomes in
QB-statistics was significantly different from zero (QB
GBL were significantly higher than in TI regardless
= 13.38, df = 3, p<.05); the mean ES for studies
what types of research designs were used.
emplo yed large group size (d + = 0.86) was
32
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
Table 3: Results of Individual Study Features Analysis
Study Feture
Grade Level
Elementary
College & Adult
Subject Area
Language arts/Social work
Math/ Science/Technology
PE/Health Ed
Other
Type of Publication
Published journal
Unpublished dissertation
Year of Publication
1998-2005
2006-2010
Instructor Bias
Same instructor
Different instructor
Unspecified
Type of Research Design
One group repeated measure
Pretest-posttest control group
Nonequivalent control group
Posttest only control group
Sample size
1-49
50-99
Over 100
Duration of Treatment
0-1 month
1-2 months
Over 2 months
QBa
Kb
d+c
Study Characteristics
6.15
26
17
0.62
6
0.55
3
1.32
1.61
26
4
0.74
14
0.62
4
0.57
4
0.74
2.23
26
3
0.74
23
0.63
2.26
26
8
0.56
18
0.69
Methodological Characteristics
13.05*
26
15
0.45
8
0.76
3
0.79
33.13*
26
5
0.81
3
0.53
14
0.50
4
1.24
0.17
26
6
0.64
10
0.67
10
0.63
Program Characteristics
7.55
26
9
0.82
9
0.56
7
0.56
1
0.81
2
18
0.66
Purpose of Instruction
0.63
Replacement of traditional
Instruction
Supplement to traditional
Unspecified
Group size for Treatment
13.38*
Large group (> 5 persons)
Small group (2-4 persons)
Individual
Unspecified
Occasion of Treatment
6.44*
In class
After class
Unspecified
Type of Game
15.46*
Web-based game
game
Non-computer game
a
the between-class homogeneity statistics.
b
the total number of studies in each subgroup.
c
the weighted mean ES.
d
the 95% confidence interval for d+.
e
the within-class goodness-of fit statistics.
*p<.05
6
2
26
9
5
4
8
26
20
4
2
26
3
7
16
33
95% CId
QWe
0.52 - 0.72
0.39 - 0.71
0.93 - 1.71
59.04* Secondary
12.35*
18.8*
0.46 - 1.02
0.52 - 0.73
0.37 - 0.77
0.48 - 1.01
1.27
81.85*
17.44*
1.18
0.52 - 0.96
0.54 - 0.72
6.82*
94.29*
0.42 - 0.69
0.58 - 0.80
27.04*
74.05*
0.32 - 0.58
0.64 - 0.89
0.51 - 1.07
51.46*
38.70*
0.10
0.60 - 1.02
0.29 - 0.77
0.39 - 0.60
1.00 - 1.49
3.85
7.48*
32.03*
26.86*
0.35 - 0.92
0.50 - 0.83
0.52 - 0.73
1.09
49.21*
52.88*
0.65 - 0.98
0.43 - 0.68
0.39 - 0.73
0.41 - 1.20
38.78*
39.38*
17.63* Unspecified
0.00
0.56 - 0.77
97.33*
0.59
0.64
0.44 - 0.74
0.15 - 1.13
4.55
0.00
0.86
0.54
0.51
0.53
0.71 - 1.00
0.33 -0.74
0.34 - 0.69
0.37 - 0.69
26.18*
8.06
42.35*
13.37
0.59
0.94
0.76
0.50 - 0.68
0.68 - 1.20
0.29 - 1.22
82.61*
14.23*
0.06
1.08
0.45
0.66
0.80 - 1.35
0.29 - 0.60
0.55 - 0.77
21.94* Computer-based
10.39
55.56*
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
significantly higher than studies used small group (d+
= 0.54) and individual setting (d+ = 0.51), indicating
students’ affective outcomes were higher in a large
group instruction. These results do not quite consistent
with findings from previous studies of collaborative
learning. In addition, the mean ES for all three subsets
of studies were all positive and significantly different
from zero, indicating that students’ affective outcomes
in GBL were significantly positive compared to TI for
all types of group sizes. The tests of Qw-statistics for
studies with a small group size was homogeneous (Qw
= 8.06, p > .05), indicating that the ESs were
consistent across the findings, whereas the tests of
Qw-statistics for studies with large and individual
group sizes were significantly heterogeneous,
suggesting that the mean ES may not be representative
of the study's findings and that other study features
may influence the magnitude of the ESs.
web-based games; therefore, the result should not be
over-generalized. The mean ES for all three subsets of
studies were all positive and significantly different
from zero, indicating that students’ affective outcomes
in GBL were significantly positive compared to TI
regardless the types of games. The tests of
Qw-statistics for studies employed computer-based
games was homogeneous (Qw = 10.39, p > .05),
indicating that the ESs were consistent across the
findings, whereas the tests of Qw-statistics for studies
used web-based games and non-computer games were
significantly heterogeneous, suggesting that the mean
ES may not be representative of the study's findings
and that other study features may influence the
magnitude of the ESs.
4. Conclusion
The results from this study suggest that the
effects of game-based learning are positive over
traditional instruction on students’ affective outcomes
in Taiwan. Several moderate variables were also
analyzed to verify the effects while implement
game-based learning in the educational settings. As
many educators devote tremendous efforts with great
expectation that technology will dramatically promote
students’ affective outcomes and increase students’
academic achievement, the results of this study
provide to classroom teachers an accumulated
research-based evidence for positive outcomes of
using technology in instruction. Left unanswered is
the question of which factors truly contribute to the
positive affective outcomes. Studies of this question
will require further clarification of the exact
relationship between educational games and learning.
This meta-analysis points out only that improvements
of students’ affective outcomes are possible. That
information by itself is useful.
3.2.4 Occasion of treatment
For occasion of treatment, studies were divided
into two subsets: in class and after class. Of the 26
studies in the present synthesis, 20 (80%) of the ESs
were coded as in class and only 4 (15%) of the ESs
were coded as after class. The test of QB-statistics was
significantly different from zero (QB = 6.44, df = 2,
p<.05); the mean ES for studies coded as after class
(d+ = 0.94) was significantly higher than studies coded
as in class (d+ = 0.59), indicating students gained more
affective outcomes in an after class GBL. This is
probably because students may feel “playing games”,
as compare to “learning”, in an after class situation
that results in a greater affective outcome. The mean
ES for both subsets of studies were all positive and
significantly different from zero, indicating that
students’ affective outcomes in GBL were
significantly positive compared to TI for both
occasions of treatment. However, the tests of
Qw-statistics for both subsets were all significantly
heterogeneous, suggesting that the mean ES may not
be representative of the study's findings and that other
study features may influence the magnitude of the
ESs.
References
[1] Piaget J.,1962, “Play, Dreams and Imitation in
Childhood,” New York: W. W. Norton.
[2] Green S.C. and Bavelier D., 2003, “Action video
game modifies visual selective attention,” Nature
Vol. 423, pp. 534–537.
[3] Klabbers J., 2003, “The gaming landscape: A
taxonomy for classifying games and simulations,”
In M. Copier, M. and Raessens, J. (Eds.) Level Up
Digital Games Research Conference (p. 54–67).
The Netherlands: University of Utrecht.
[4] de Freitas S., 2005, “Review of the uptake and
embedding of digital content: Internal report,”
UK: Becta, Coventry.
[5] de Freitas S. and Oliver M., 2006, “How can
exploratory learning with games and simulations
within the curriculum be most effectively
evaluated?,” Computer & Education Vol. 46, pp.
249–264.
[6] Walliser , B., 1998, “A spectrum of equilibration
processes in games,” Journal of Evolutionary
Economics, Vol. 8, pp. 67–87.
[7] Juul, J., 2002, “The open and the closed: game of
3.2.5 Type of game
For type of game, studies were divided into
three subsets: web-based game (on-line game),
computer-based game (off-line
game), and
non-computer game. Of the 26 studies in the present
synthesis, 16 (62%) of the ESs employed
non-computer games, 7 (27%) of the ESs used
computer-based games, and only 3 (12%) employed
web-based games. The test of QB-statistics was
significantly different from zero (QB = 15.46, df = 2,
p<.05); the mean ES for studies employed web-based
games (d+ = 1.08) was significantly higher than mean
ESs of studies used computer-based games (d+ = o.45)
and non-computer games (d+ = 66), indicating
students had higher affective outcomes such as
learning motivation and attitudes toward web-based
games. Yet, there were only 4 studies that used
34
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
emergence and games of progression,” In Mayra
F. (Ed.), Computer Game and Digital Cultures
Conference Proceedings (p.323–329). Finland:
Tampere University Press.
[8] Gee, J. P., 2003, “What video games have to teach
us about learning and literacy,” New York:
Palgrave Macmillian.
[9] Malone, T., 1980, “What makes things fun to learn?
A study of intrinsically motivating computer
games,” Technical report no. CIS-7 (SSL-80-11),
Xerox Palo Alto Research Center.
[10] Prensky, M., 2001, “Digital game-based
learning,” New York: McGraw-Hill.
[11] Papastergiou, M., 2009, “Digital game-based
learning in high school computer science
education: impact on educational effectiveness
and student,” Computers & Education, Vol. 52,
No. 1, pp. 1-12.
[12] Virvou, M., Katsionis, G. and Manos, K., 2005,
“Combining software games with education:
Evaluation of its educational effectiveness,”
Educational Technology & Society, Vol.8, No.2,
pp. 54–65.
[13] Tüzün,H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y.
and Kızılkaya, G., 2009, “The effects of computer
games on primary school students’ achievement
and motivation in geography learning,”
Computers & Education, Vol. 52, No. 1, pp.
68-77.
[14] Liu, T. and Chu, Y., 2010, “Using ubiquitous
games in an English listening and speaking course:
Impact on learning outcomes and motivation,”
Computers & Education, Vol. 55, No. 2, pp.
630-643.
[16] Huang, Y., 2009, “A study of the effects of
science play instruction with assisted scientific
toys on the learning of “air” unit for the
third-grade students,” Unpublished master’s
dissertation, National Taipei University of
Education, Taipei, Taiwan.
[17] Ke, H., 2006, “A study of the effect of science
instruction with toy-playing approach on
elementary students’ attitudes,” Unpublished
master’s dissertation, National Taipei University
of Education, Taipei, Taiwan.
[18] Lin, C., 2010, “A study of motivation effects of
gesture and game-based mobile learning system
-an example for botany misconception,”
Unpublished master’s dissertation, National
Hsinchu University of Education, Hsinchu,
Taiwan.
[19] Wang, C., 2005, “A study of the effects of science
play instruction with assisted scientific toys on
fifth-grade students for their scientific literacy,”
Unpublished master’s dissertation, National
Taipei Teachers College, Taipei, Taiwan.
[20] Wu, T., 2006, “Development of a gamed-based
learning system for enhancing learner''s
motivation and self-perception in energy
education,” Unpublished master’s dissertation,
35
National Central University.
[21] Chen, L., 2006, “A study of cooperative learning
on mathematics teaching for the first-grade of
elementary school in Taipei City,” Unpublished
master’s dissertation, National Taipei University
of Education, Taipei, Taiwan.
[22] Lai, P., 2005, “The study of game teaching with
7th grade students on epistemological beliefs in
mathematics, learning motivation and academic
achievement,” Unpublished master’s dissertation,
Da Yeh University, Changhua, Taiwan.
[23] Lin. J., 2004, “A study on the effects of
incorporating game into the elementary
mathematics teaching,” Unpublished master’s
dissertation, National Taipei Teachers College,
Taipei, Taiwan.
[24] Lo, G., 2006, “A study of the effectiveness of
learning nutrition through computer game
enrichment,” Unpublished master’s dissertation,
National Taipei University of Education, Taipei,
Taiwan.
[25] Wang, I., 1998, “The effect of Adlerian group
play therapy on the improvement of elementary
school,” Unpublished master’s dissertation,
National Tainan Teachers College, Tainan,
Taiwan.
[26] Hedges, L. V. and Olkin, I., 1985, “ Statistical
methods for meta-analysis” Orlando: Academic
Press.
[27] Glass, G. V., McGaw, B. and Smith, M. L., 1981,
“Meta-analysis in social research,” Beverly Hills,
CA: Sage Publications.
[28] Christman, E., Badgett, J. and Lucking, R., 1997,
“Progressive comparison of the effects of
computer-assisted instruction on the academic
achievement of secondary students,” Journal of
Research on Computing in Education, Vol. 29, No.
4, pp. 325 – 337.
[29] Rosenthal, R.,1991, “Meta-analytic procedures
for social research (Rev.ed.),” Beverly Hills:
Sage.
[30] Andrews, G., Guitar, B. and Howie, P., 1980,
“Meta-analysis of the effects of stuttering
treatment,” Journal of Speech and Hearing
Disorders, Vol. 45, pp. 287-307.
[31] Waxman, H. C., Wang, M. C., Anderson, K. A.
and Walberg, H. J., 1985, “Adaptive education
and student outcomes: A Quantitative synthesis,”
Journal of Educational Research, Vol. 78, N0. 4,
pp. 228-236.
[32] Cohen, J., 1977, “Statistical power analysis for
the behavioral science (Revised Edition),” New
York: Academic Press.
[33] Chen, J., 2007, “The effects of the scientific game
activity on motivation toward science learning for
the 8th grade students,” Unpublished master’s
dissertation, National Taiwan Normal University,
Taipei, Taiwan.
[34] Cheng, Y., 2009, “The research of game teaching
Journal of Information Technology and Applications
Vol. 5, No. 1, pp. 28-36 2011
applied to the art appreciation curriculum in
elementary school,” Unpublished master’s
dissertation, National Hsinchu University of
Education, Hsinchu, Taiwan.
[35] Hung, H., 2006, “Design and application of a
game-based mobile guiding system,” Unpublished
master’s dissertation, National Taiwan Normal
University, Taipei, Taiwan.
[36] Kao, C., 2009, “The effectiveness of digital
game-based learning on junior high school
students' learning motivation, problem solving,
and academic achievement,” Unpublished
master’s dissertation, National Cheng Kung
University, Tainan, Taiwan.
[37] Lai, J., 2007, “The effect of E-learning in
scientific attitude of different earning style
students,” Unpublished master’s dissertation,
National Taipei University of Education, Taipei,
Taiwan.
[38] Li, C., 2009, “A action study of integrating games
into the teaching of English for junior high
students in a physical education school: A case of
the Taitung Physical Education Secondary
School,” Unpublished master’s dissertation,
National Dong Hwa University, Hualien, Taiwan.
[39] Lin, Y., 2008, “A design and study of simulation
game system：the case of learning lever principles
of elementary students,” Unpublished master’s
dissertation, National Taiwan Normal University,
Taipei, Taiwan.
[40] Lin, Y., 2009, “Learning achievement of
game-based learning on elementary school
nutrition education,” Unpublished master’s
dissertation, Leader University, Tainan, Taiwan.
[41] Liu, Y., 2005, “the effect of Interesting Scientific
Competition
on
eighth-grade
students’
achievement and attitudes toward science
learning,” Unpublished master’s dissertation,
National Taipei University of Education, Taipei,
Taiwan.
[4 2 ] Pan, I and Wang, M., 2003, “A study on the
effects of the play-based elementary science
teaching,” Journal of Taipei Municipal Teachers
College, vol. 34, pp. 157-172.
[43] Shu, F., 2006, “The action research of games
integrated into chindren's learning of mathematics
after class-an example of disadvantaged fifth
graders,” Unpublished master’s dissertation,
National Taichung University of Education,
Taichung, Taiwan.
[44] Shyu, J., 2006, “Effects of implementing the
computer games on English vocabulary learning
and attitudes,” Unpublished master’s dissertation,
National Taipei University of Education, Taipei,
Taiwan.
[4 5 ] Su, H. and Hsieh, H ., 2 00 7 , “ The study of
integrating science game into elementary school
natural science and living technology instructions
on
fourth
graders'
science
attitudes,”
Cu rricu lu m & In stru ctio n Qua rterly , vo l.
1 0 , no . 1 , p p. 111-129.
[46] Tseng, K., 2006, “Influences of singing games
applied Fulao children's songs to students' music
learning interests and achievement,” Unpublished
master’s dissertation, Taipei Municipal University
of Education, Taipei, Taiwan.
[47] Wang, W., Chou, C., Peng, H. and Yeh, I., 2005,
“ The implementation and effectiveness of
initiative game activities program on social
interaction in fifth and sixth grade students,”
Bulletin of Physical Education, vol. 38, no. 3,
pp.51-62.
[48] Wang, Y., 2007, “The effect of game teaching
applied to national defense general education in
vocational high schools,” Unpublished master’s
dissertation, National College of Physical
Education, Taipei County, Taiwan.
[49] Clark, R. E.,1983, “Reconsidering research on
learning from media,” Review of Educational
Research, Vol. 53, No. 4, pp. 445-459.
[50] Holubec, E. J., Johnson, D. W. and Johnson, R. T.,
1995, “Cooperative learning in reading and
language arts,” In Rabinowitz, M., Antonacci, P.
and Hedley, C. N. (Eds.), Thinking and literacy:
The mind at work (p. 229–240). NJ: Lawrence
Erlbaum Associates.
[51] Johnson, D. W. and Johnson, R. T., 1999,
“Making cooperative learning work,” Theory into
Practice, Vol. 38, No. 2, pp. 67–73.
[52] Harasim, L. M., 1993, “Collaborating in
cyberspace using computer conferences as a
group
learning environment,”
Interactive
Learning Environments, Vol. 3, No. 2, pp.
119–130.
Acknowledgements
This research is supported by a grant from the
National Science Council of Taiwan (grant no. NSC
96-2520-S-003-011).
36