CHAPTER Ⅰ INTRODUCTION
1.3 Research gaps
a. Few studies used affective technique to evaluate DGBL issue
Affective computing technique has become the learning research trends. But there is few studies used physiology signals recognition in DGBL issue. In this study, the affective computing
technique was used to measure the students’ learning attention.In the literature, several studies for physiology signals recognition in learning have used eye tracking technology to observing the visual attention (Dimigen, Sommer, Hohlfeld, Jacobs, &
Kliegl, 2011; Latanov, Konovalova, & Yermachenko, 2008; Lin, Imamiya, & Mao, 2008;
Schmid, Schmid Mast, Bombari, Mast, & Lobmaier, 2011), also have used EEG and Heart rate variability in ECG to measure the learning emotion(Chen & Wang, 2011; C. Zhang, Zheng, & Yu, 2009) and mental workload (Patel, Lal, Kavanagh, & Rossiter, 2011; Zhao, Zhao, Liu, & Zheng, 2012).
b. Few studies of DGBL and learning style which used eye-tracking technology were situated in the problem solving strategy.
Few studies of DGBL and learning style which used eye-tracking technology were situated in the problem solving strategy. But several studies of multimedia learning were used. For example, Tsai et al. (2012) employed an eye-tracking technique to examine students’ visual attention when solving a multiple-choice science problem. F. Y. Yang et al. (2013) investigated university learners’ visual attention during a PowerPoint (PPT) presentation on the topic of
“Dinosaurs” in a real classroom. The results showed that the earth-science majors’ students displayed higher visual attention than non-earth-science students to the text zones, but few differences were found for the picture zones. When the students viewed those slides containing scientific hypotheses, the difference in attention distributions between the text and pictures
reduced. Further analyses of fixation densities and saccade paths showed that the earth-science majors’ students were better at information decoding and integration.
Thus, eye-tracking studies have provided insights on how students pay attention to learning material. This study further explores how learning environment and style affect the science problem solving strategy of students.
CHAPTER Ⅱ LITERATURE REVIEW
In chapter 2, the literature of Digital Game-Based Learning, problem solving strategy and affective computing in learning were organized. In section 2.1, this paper described that advantage of Digital Game-Based Learning. And then, the learning attention, affective experiences, cognitive load, academic achievement and gender in Digital Game-Based Learning issue were discussed. In section 2.2, this paper summarized 4 factors at least can affect the development of problem solving strategy, included learning environments, learners’ learning style, prior knowledge and academic achievement. In section 2.3, these papers of affective computing in learning were organized. In addition, this paper found that eye movement, EEG and ECG have become the research trends, so we introduced that researches and variables of eye movement, EEG and ECG.
2.1 Digital Game-Based Learning
Digital entertainment games have become one of the most popular leisure activities globally.
The effect of Digital Game-Based Learning in promoting meaningful learning might be due to opportunities for “learning by doing” (Pannese & Carlesi, 2007).
According to T. M. Connolly et al. (2007), GBL can be defined as “the use of a computer game-based approach to deliver, support, and enhance teaching, learning, assessment, and evaluation”. Students use games to explore, discover, and question, ultimately constructing concepts and relationships in authentic contexts. These “learning by doing” and “active learning”
concepts are important constructivist principles which underlie game-based learning (Y. T. C.
Yang, 2012). Richard E. Mayer and Johnson (2010) considered that a DGBL environment should feature 1) a set of rules and constraints, (2) a set of dynamic responses to the learners’ actions, (3)
appropriate challenges enabling learners to experience a feeling of self-efficacy, and (4) gradual, learning outcome oriented increases in difficulty.
2.1.1 The advantages of Digital Game-Based Learning
Games that encompass educational objectives and subject matter are believed to hold the potential to render learning of academic subjects more learner-centered, easier, more enjoyable, more interesting, and, thus, more effective (Prensky, 2001). There is also widespread acknowledgment of the advantages that the use of games has in elementary and secondary education (Ebner & Holzinger, 2007).
Specifically, games constitute potentially powerful learning environments for a number of reasons.
Kebritchi and Hirumi (2008) identified the following five reasons for defining GBL as an effective tool for learning: (a) GBL uses action instead of explanation; (b) GBL creates personal motivation and satisfaction; (c) GBL accommodates multiple learning styles and skills; (d) GBL reinforces mastery of skills; and (e) GBL provides an interactive and decision-making context.
According to O’Neil, Wainess, and Baker (2005), computer games are useful for instructional purposes and they also provide multiple benefits: (a) complex and diverse approaches to learning processes and outcomes; (b) interactivity; (c) ability to address cognitive as well as affective learning issues; and (d) motivation for learning.
Oblinger (2004) identified the following five reasons: (a) they can support multi-sensory, active, experiential, problem-based learning, (b) they favored activation of prior knowledge
encompass opportunities for self-assessment through the mechanisms of scoring and reaching different levels, and (e) they increasingly become social environments involving communities of players.
2.1.2 Learning motivation, affective experiences, cognitive load and Digital Game-Based Learning
Many studies have investigated the effects of Digital Game-Based Learning (DGBL) on learning and motivation (Erhel & Jamet, 2013). Thomas M. Connolly, Boyle, MacArthur, Hainey, and Boyle (2012) found the use of games to teach educational content inevitably raises the question of their compatibility with deep learning. This has prompted many researchers to investigate the actual benefits of digital games, in terms of learning and motivation. For instance, Virvou, Katsionis, and Manos (2005) designed VR-ENGAGE computer game for teaching geography to fourth grade students. It was concluded that computer games could promote motivation, especially for at-risk students or for students with motivational problems. Digital Game-Based Learning provides that learners have sufficient level of curiosity to explore the learning task, sustain the learners’ motivation. A study used a attention subscale to measure the attention score when learners study by digital game (Huang, 2011).
Learning motivation is dependent of four perceptual components: attention, relevance, confidence and satisfaction (Keller, 2008). The study surveyed 264 undergraduate students after playing the Trade Ruler online game. Based on the data collected by ARCS-based Instructional Materials Motivational Survey (IMMS), a regression analysis revealed a significant model between motivational processing (attention, relevance, and confidence) and the outcome processing (satisfaction). Based on the observed ARCS scores in this study, learners started out with a successful motivational processing that consisted of a high attention level, a low relevance
level, and a high confidence level. At the end of the learning process, however, they reported a relatively low level of satisfaction (Huang et al., 2010). Derbali and Frasson (2010) investigated players’ motivation during serious game play. It is based on a theoretical model of motivation (John Keller’s ARCS model of motivation) and EEG measures, and the results showed that power spectral analysis showed EEG waves patterns correlated with increase of motivation during different parts of serious game play. Thus, this study tried to measure the attention score by EEG.
Clark et al. (2011) checked for how similar or different are the learning and affective experiences of students playing the game in two different countries (i.e., Taiwan and the United States). In the other research, Huang (2011) found that learners’ cognitive capacities were in high demand in the online gamed-based learning. Since researchers on cognitive load have concluded that an overloaded cognitive capacity can de-motivate learners. This study found the all research used questionnaire survey to measure the motivation score, the score was not an impersonal measure.
Thus, the first goal of this study is to measure the attention score by
affective computing technique and compare the attention, affective experiences and cognitive load of learners in
digital-game based and traditional learning environments.2.1.3 Academic achievement and Digital Game-Based Learning
The major reviews of digital games seeking to explore the issue of academic achievement have reported contradictory or ambiguous findings (Papastergiou, 2009). A meta-analysis of
methods 27 studies advocating game-based learning, and 3 studies supporting traditional methods of instruction. Unfortunately, the effectiveness of DGBL on students’ academic achievement is still unproven in a robust empirical research setting. Thus, the third goal of this study is to compare the academic achievement of learners in digital-game based and traditional learning environments. Vogel, Greenwood-Ericksen, Cannon-Bowers, and Bowers (2006) in a literature review based on 32 empirical studies, reported that interactive games were more effective than traditional classroom instruction on learners’ academic learning gains and cognitive skill development. Otherwise, many studies found that no differences in student learning can be found between learning environments that involve games and those without game elements (Leonard A. Annetta, Minogue, Holmes, & Cheng, 2009; Papastergiou, 2009) Wrzesien and Alcañiz Raya (2010) found sixth graders reported higher motivation and engagement levels as a result of playing a science-based game; however, there was no evidence to show that the game led to significant learning advancements over the traditional class. Base on above literature, this study wanted to compare the academic achievement of learners in digital-game based and traditional learning environments.
2.1.4 Gender and Digital Game-Based Learning
Regarding gender issues, as shown in the study, despite the fact that the boys of the sample exhibited significantly greater involvement with, liking of and experience in computer gaming outside school as well as significantly greater initial knowledge of the embedded subject matter, and greater interaction among them during the intervention, the learning gains that boys and girls achieved through the use of the game did not differ significantly. Furthermore, no significant gender differences were found in students’ views on the overall appeal, quality of user interface, and educational value of the game used. Papastergiou (2009)Data analyses showed that the
gaming approach was both more effective in promoting students’ knowledge of computer memory concepts and more motivational than the non-gaming approach. Despite boys’ greater involvement with, liking of and experience in computer gaming, and their greater initial computer memory knowledge, the learning gains that boys and girls achieved through the use of the game did not differ significantly , and the game was found to be equally motivational for boys and girls. Ke and Grabowski (2007) also found that gender did not influence the learning effectiveness and motivational appeal of games for school children. Thus, this study didn’t check the relationship between gender and Digital Game-Based Learning.
2.2. Problem solving strategy
Problem-solving is a 21st century skill which is essential for learning, work, and daily life (L. A. Annetta, 2008). Problem solving can be defined as the ability to find causes, find solutions, and avoid problems (Chan & Wu, 2007). It has been argued that digital games provide a meaningful framework for solving problems (L. A. Annetta, 2008), since students are placed in scenarios in which they must synthesize diverse information and analyze strategies, leading to a greater understanding of the causal links between decision-making behaviors (Ebner &
Holzinger, 2007). Therefore, digital games can be seen as a good tool for understanding the link between cause and effect (Kiili, 2005). Thus, apart from knowledge acquisition, game playing can also favored the development of various skills, such as critical thinking and problem-solving skills. Problem solving is one of the integral approaches to achieving effective and meaningful learning (D. H. Jonassen, 2004). Learners, while solving a problem, have to understand the
extensively applied to many subject domains such as science (Linn, Clark, & Slotta, 2003), mathematics (D. Jonassen, 2003) and design (Jermann & Dillenbourg, 2008) as a means of promoting learning in these domains.
This study found the 4 factors at least can affect the problem solving strategy:
(1) Learning environments
Digital games provide a meaningful framework for solving problems (L. A. Annetta, 2008), since students are placed in scenarios in which they must synthesize diverse information and analyze strategies, leading to a greater understanding of the causal links between decision-making behaviors (Ebner & Holzinger, 2007). Therefore, digital games can be seen as a good tool for understanding the link between cause and effect (Kiili, 2005). Although research on problem solving in DGBL has been conducted (Dickey, 2006; Robertson & Howells, 2008), its effectiveness in fostering problem solving abilities has not received sufficient attention from empirical research.
(2) Learners’ learning style
Graf et al. (2008) investigated the relationship between learning styles and working memory capacity, they demonstrated that learners with high working memory capacity tend to prefer a reflective, intuitive, and sequential learning style whereas learners with low working memory capacity tend to prefer an active, sensing, visual, and global learning style. In addition, previous research indicates that individuals with higher levels of working memory capacity perform better on learning tasks because they have more cognitive resources (Daneman & Carpenter, 1980; R.E.
Mayer, 2001) It is likely that working memory capacity affects cognitive efficiency due to the processing and storage requirements necessary to solve mental problems (Hoffman & Schraw, 2009). According these research findings, this paper want to investigate the relationship between learning style and problem solving abilities.
(3) Prior knowledge
F. Y. Yang et al. (2013) investigated university learners’ visual attention during a PowerPoint (PPT) presentation on the topic of “Dinosaurs” in a real classroom. The results showed that the earth-science majors’ students were better at information decoding and integration than non-earth-science majors’ students. The discussion points out that the interaction between types of graphics and information processing behaviors is also mediated by prior knowledge.
(4) Academic achievement
Tsai et al. (2012) employed an eye-tracking technique to examine students’ visual attention when solving a multiple-choice science problem. The results showed that successful problem solvers focused more on relevant key factors, while unsuccessful problem solvers experienced difficulties in decoding the problem, in recognizing the relevant factors, and in self-regulating of concentration.
Thus, the third goal of this study is to compare the science problem solving strategy of learners in different learning environments, learning style, major and academic achievement.
2.3. Affective Computing in Learning
Since Affective Computing was proposed, there has been a burst of research that focuses on creating technologies that can monitor and appropriately respond to the affective states of the user (Picard, 1997). Because this new Artificial Intelligence area, computers able to recognize human emotions in different ways. Why human emotion is an important research area? The latest
Eye movement, EEG and ECG have become the research trends
Some of the main physiological signals highly adopted for human emotion assessment are:
Electrocardiogram (ECG), Electromyogram (EMG), Skin Conductive Resistance (SCR), and Blood Volume Pressure (BVP). Several approaches have been found the correlation between the emotional changes and EEG signals (Chanel, Ansari-Asl, & Pun, 2007). In this study, we summarize the multi physiological feature system researches as shown in Table 1. According the table, we found the physiological signals of eye movement, EEG and ECG have become the research trends. But it not exist a system combined these signals to recognize the affective of human.
Attention and emotion effect student learning and performance
Our aim is develop a learning affective recognition system, which can get, recognize and analyze attention and emotion state when students learning. Attention can effects student learning and performance is as everyone knows. And distractor processing with emotional information also has implications for theories of attention (Srinivasan & Gupta, 2010). Thus, if human emotions are essential for human thinking and learning processes, a successful learning environment have to intensively consider this fact.
In recent years, researchers have increasingly called into question the relevance of these basic emotions to the learning process (Baker, D'Mello, Rodrigo, & Graesser, 2010). Much of what is known about achievement emotions stems from research with students participating in traditional educational settings. Positive emotions, such as enjoyment, hope, and pride, have been positively associated with intrinsic motivation, effort, self-regulation, and more sophisticated learning strategies (Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011), whereas
negative emotions such as anger/frustration, shame, anxiety, and boredom have been associated with reduced effort, lower performance, increased external regulation, and decreased self-regulated learning strategies(Daniels et al., 2009).
The Physiological input signals this study selected
The physiological input signals of eye movement, EEG and ECG were selected to input our learning affective recognition system. According the past studies, several techniques need to be combined to estimate the state of attention and emotion.
Eye movements provides information about location of attention and the nature, sequence and timing of cognitive operations (Lin et al., 2008). With the emergence of Electroencephalography (EEG) technology, learner's brain characteristics could be accessed directly and the outcome may well hand-in-hand supported the conventional test. recognize a learner's Learning Style (Rashid et al., 2011). And the arousal state of the brain (Q. Zhang & Lee, 2012), alertness, cognition, and memory (Chris Berka et al., 2004; C. Berka et al., 2007) also can be measure. Heart rate variability from ECG, has gained widespread acceptance as a sensitive indicator of mental workload(Lin et al., 2008). And positive emotions may change the HF components of HRV (von Borell et al., 2007).
Table 1.Multi physiological feature system review
According the table, we found the physiological signals of eye movement, EEG and ECG have become the research trends. But it not exist a system combined these signals to recognize the affective of human.
Emotion recognition features
Research object Reference Eye EEG ECG Facial speech SCR
emotion recognition (Kim, Ban, & Kim, 2004) ✕ ✕
neonatal seizures (Greene, Boylan, Reilly, de Chazal, & Connolly, 2007) ✕ ✕
emotion recognition (Ruffman, Henry, Livingstone, & Phillips, 2008) ✕ ✕
emotion recognition (Lin et al., 2008) ✕ ✕ ✕
visual search task (Latanov et al., 2008) ✕ ✕
emotion recognition (Q. Zhang & Lee, 2010) ✕ ✕
emotional distractors (Srinivasan & Gupta, 2010) ✕
emotion recognition (B. Yang & Lugger, 2010) ✕
emotion recognition (Murugappan, Ramachandran, & Sazali, 2010) ✕
brain computer interface (Lee, Woo, Kim, Whang, & Park, 2010)
reading process (Dimigen et al., 2011) ✕ ✕
emotion recognition (Schmid et al., 2011) ✕ ✕
learning state (Chen & Wang, 2011) ✕
driver fatigue (Patel et al., 2011) ✕
driver fatigue (Zhao et al., 2012) ✕ ✕
emotion recognition (Q. Zhang & Lee, 2012) ✕
epilepsy state (Valderrama et al., 2012) ✕ ✕
2.3.1 Eye movement variables
The eye tracking technique has been typically adopted to examine human visual attention based on the eye-mind assumption (Just & Carpenter, 1980). In general, eye fixation location reflects attention and eye fixation duration reflects processing difficulty and amount of attention (the longer the information is fixated, the more complex it is or the deeper it is processed). The fixation locations and duration reflect the individuals’ reading strategies and prior knowledge or experience (Hyönä, Lorch, & Kaakinen, 2002). Besides, scan path patterns exhibit individuals’
cognitive strategies utilized in goal-oriented tasks (Gandini, Lemaire, & Dufau, 2008). Thus, a record of eye movements provides information about location of attention and the nature, sequence and timing of cognitive operations (Lin et al., 2008).
Three eye movement parameters—fixation, saccade and scan path—are often investigated to indicate users’ cognitive behaviors (Lin et al., 2008). Fixation is defined as a relatively motionless gaze that usually lasts for 200–500 ms, during which information about a visual stimulus is extracted (Jukka, 2010). Saccades are rapid and ballistic movements of gaze between fixations with a velocity of about 500。, directing the viewer’s eye to a visual target. Information processing is suppressed during a saccade, though some peripheral information may be available.
Scan paths are defined as a sequence of fixations and saccades, indicating a movement of attention.
About an eye tracking study for solving multiple-choice science problem, the researcher tested their hypotheses and got some discussion as following (Tsai et al., 2012). (1) There is a significant difference in students’ scan sequences among factors for solving the problems between successful and unsuccessful problem solvers. Successful problem solvers tend to shift their visual attention from irrelevant factors to relevant factors, while unsuccessful problem
solvers tend to shift their visual attention from relevant to irrelevant factors and to the problem statement. It is obvious that the two groups shift attentions in opposite directions. (2) Students pay more attention to chosen options than to rejected alternatives, and tend to spend more time inspecting relevant factors than irrelevant one. Based on these studies, the strategy of solving problem will be analysis by eye movement data.
Table 2.Eye movement parameters
Parameters Research variables
Fixation (visual attention) fixation durations (Tsai et al., 2012) hot zone image (Tsai et al., 2012)
number of fixation (Schmutz, Roth, Seckler, & Opwis, 2010) Scan path (strategy) sequential analysis (Tsai et al., 2012)
qualitative analysis (Schmutz et al., 2010)
2.3.2 Brain wave variables
The EEG signal is a voltage signal that can be measured on the surface of the scalp, arising from large areas of coordinated neural activity. This neural activity varies as a function of development, mental state, and cognitive activity, and the EEG signal can measurably detect such variation. EEG is generally described in terms of its frequency band. The amplitude of the EEG shows a great deal of variability depending on external stimulation as well as internal mental states.
The relationship between EEG and emotion
The best known correlates of emotionality found with EEG involve prefrontal asymmetry.
That is, more active left frontal region indicates a positive reaction, and more active right anterior lobe indicates negative affection (Q. Zhang & Lee, 2012). The two that are most important for arousal state of brain are the alpha (8–12 Hz) and beta (12–30 Hz) frequencies.
Alpha waves are typically for an alert/relaxed mental state, while beta activity is most prominent in the frontal cortex over other areas during intense focused mental activity (Kandel, Schwartz,
& Jessell, 2000). Therefore, the beta/alpha ratio could be an indication of the arousal state of the brain. By taking arousal axis into consideration, we can sub-categorize positive and negative
& Jessell, 2000). Therefore, the beta/alpha ratio could be an indication of the arousal state of the brain. By taking arousal axis into consideration, we can sub-categorize positive and negative