Chapter Ⅴ CONCLUSION AND IMPLICATIONS
5.3 Future Study
Since Affective Computing was proposed, there has been a burst of research that focuses on creating technologies that can monitor and appropriately respond to the affective states of the user. Because this new Artificial Intelligence area, computers able to recognize human emotions in different ways. In this study, we used the affective computing technique to evaluate the performance of Digital Game-Based Learning. This is a new approach in Digital Game-Based Learning issue. We propose several issues for future studies as follow.
Increase else physiology signals to measure learning state of learners
For using affective computing technique to evaluate and compare the attention score, affective experiences and cognitive load of learners in digital-game based learning environments in this study, the attention was measured by brain wave, the affective experiences was measured by heartbeat and the cognitive load was measured by eye movement. The future
studies may increase else physiology signals to measure learning state of learners. For instance, facial expression detection (C. H. Yang, 2012), verbalization (Leony, Muñoz-Merino, Pardo, &
Delgado Kloos, 2013), pressure mouse (Leony et al., 2013).
Provide better DGBL environment
This study considered that maybe we didn’t provide the enough feature of DGBL environment, specifically for appropriate challenges (Liu et al., 2011) and full learning time (Y. T.
C. Yang, 2012). The future studies may increase the appropriate challenges enabling learners to experience a feeling of self-efficacy, and need to teach the amount of time required to fully evaluate the learning effect.
Reference
[1] American College of Cardiology/American Heart Association. (1999). Heart rate variability:
Guidelines of ambulatory electrocardiography – Part III. Journal of American College of
Cardiology, 34(3), 912-948.
[2] American Heart Association. (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043-1065. doi: doi:
10.1161/01.CIR.93.5.1043
[3] Annetta, L. A. (2008). Video games in education: why they should be used and how they are being used. Theory Into Practice, 47(3), 229–239.
[4] Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M.-T. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics.
Computers & Education, 53(1), 74-85. doi:
http://dx.doi.org/10.1016/j.compedu.2008.12.020
[5] Bakeman, R., & Quera, V. (1995). Analyzing interaction: Sequential analysis with SDIS and
GSEQ. New York: Cambridge University Press.
[6] Baker, R. S. J. d., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments. Int. J.
Hum.-Comput. Stud., 68(4), 223-241. doi: 10.1016/j.ijhcs.2009.12.003
[7] Baldaro, B., Tuozzi, G., Codispoti, M., Montebarocci, O., Barbagli, F., Trombini, E., &
Rossi, N. (2004). Aggressive and non-violent videogames: short-term psychological and cardiovascular effects on habitual players. Stress and Health, 20(4), 203-208. doi:
10.1002/smi.1015
[8] Ben Ammar, M., Neji, M., Alimi, A. M., & Gouardères, G. (2010). The Affective Tutoring System. Expert Systems with Applications, 37(4), 3013-3023. doi:
http://dx.doi.org/10.1016/j.eswa.2009.09.031
[9] Berka, C., Levendowski, D. J., Cvetinovic, M. M., Petrovic, M. M., Davis, G., Lumicao, M.
N., . . . Olmstead, R. (2004). Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset. International Journal of
Human-Computer Interaction, 17(2), 151-170. doi: 10.1207/s15327590ijhc1702_3
[10] Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T.,Olmstead, R. E., . . . Craven, P. L. (2007). EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat Space Environ Med, 78(5 Suppl), B231-244.
[11] Chan, S. M., & Wu, W. T. (2007). New problem solving ability test. Taipei, TW:
Psychological Publishing Press.
[12] Chanel, G., Ansari-Asl, K., & Pun, T. (2007). Valence-arousal evaluation using
physiological signals in an emotion recall paradigm. Lecturer Notes in Computer Science, 1, 530-537. doi: 10.1109/ICSMC.2007.4413638
[13] Chen, C.-M., & Wang, H.-P. (2011). Using emotion recognition technology to assess the effects of different multimedia materials on learning emotion and performance. Library &
Information Science Research, 33(3), 244-255. doi: 10.1016/j.lisr.2010.09.010
[14] Clark, D. B., Nelson, B. C., Chang, H.-Y., Martinez-Garza, M., Slack, K., & D’Angelo, C.
M. (2011). Exploring Newtonian mechanics in a conceptually-integrated digital game:
Comparison of learning and affective outcomes for students in Taiwan and the United States.
Computers & Education, 57(3), 2178-2195. doi:
http://dx.doi.org/10.1016/j.compedu.2011.05.007
[15] Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A
systematic literature review of empirical evidence on computer games and serious games.
Computers & Education, 59(2), 661-686. doi:
http://dx.doi.org/10.1016/j.compedu.2012.03.004
[16] Connolly, T. M., Stansfield, M. H., & Hainey, T. (2007). An application of games-based learning within software engineering. British Journal of Educational Technology, 38(3), 416–428.
[17] Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19(4), 450-466. doi:
http://dx.doi.org/10.1016/S0022-5371(80)90312-6
[18] Daniels, L. M., Stupnisky, R. H., Pekrun, R., Haynes, T. L., Perry, R. P., & Newall, N. E.
(2009). A Longitudinal Analysis of Achievement Goals: From Affective Antecedents to
(Vol. 6095, pp. 297-299): Springer Berlin Heidelberg.
[20] Dickey, M. (2006). Game Design Narrative for Learning: Appropriating Adventure Game Design Narrative Devices and Techniques for the Design of Interactive Learning
Environments. Educational Technology Research and Development, 54(3), 245-263. doi:
10.1007/s11423-006-8806-y
[21] Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A. M., & Kliegl, R. (2011). Coregistration of Eye Movements and EEG in Natural Reading: Analyses and Review. Journal of
Experimental Psychology: General, 140(4), 552-572. doi: 10.1037/a0023885
[22] Ebner, M., & Holzinger, A. (2007). Successful implementation of user-centered game based learning in higher education: An example from civil engineering. Computers & Education,
49(3), 873-890. doi: http://dx.doi.org/10.1016/j.compedu.2005.11.026
[23] Erhel, S., & Jamet, E. (2013). Digital Game-Based Learning: Impact of instructions and feedback on motivation and learning effectiveness. Computers & Education, 67(0), 156-167.
doi: http://dx.doi.org/10.1016/j.compedu.2013.02.019
[24] Fries, P., Nikolić, D., & Singer, W. (2007). The gamma cycle. Trends in Neurosciences, 30, 309-316.
[25] Gandini, D., Lemaire, P., & Dufau, S. (2008). Older and younger adults’ strategies in approximate quantification. Acta Psychologica, 129(1), 175–189.
[26] Geisler, F. C. M., Vennewald, N., Kubiak, T., & Weber, H. (2010). The impact of heart rate variability on subjective well-being is mediated by emotion regulation. Personality and
Individual Differences, 49(7), 723-728. doi: 10.1016/j.paid.2010.06.015
[27] Graf, S., Lin, T., & Kinshuk. (2008). The relationship between learning styles and cognitive traits – Getting additional information for improving student modelling. Computers in
Human Behavior, 24(2), 122-137. doi: http://dx.doi.org/10.1016/j.chb.2007.01.004
[28] Graf, S., Liu, T.-C., Kinshuk, Chen, N.-S., & Yang, S. J. H. (2009). Learning styles andcognitive traits – Their relationship and its benefits in web-based educational systems.
Computers in Human Behavior, 25(6), 1280-1289. doi:
http://dx.doi.org/10.1016/j.chb.2009.06.005
[29] Greene, B. R., Boylan, G. B., Reilly, R. B., de Chazal, P., & Connolly, S. (2007).
Combination of EEG and ECG for improved automatic neonatal seizure detection. Clinical
Neurophysiology, 118(6), 1348-1359. doi: 10.1016/j.clinph.2007.02.015
[30] Hestenes, D., & Halloun, I. (1995). Interpreting the FCI. The Physics Teacher, 33, 502-506.
[31] Hoffman, B., & Schraw, G. (2009). The influence of self-efficacy and working memory capacity on problem-solving efficiency. Learning and Individual Differences, 19(1), 91-100.
doi: http://dx.doi.org/10.1016/j.lindif.2008.08.001
[32] Holsanova, J., Holmberg, N., & Holmqvist, K. (2009). Reading information graphics: the role of spatial contiguity and dual attentional guidance. Applied Cognitive Psychology, 23, 1215–1226.
[33] Hou, H. T. (2010a). Exploring the behavioral patterns in project-based learning with online discussion: quantitative content analysis and progressive sequential analysis. Turkish Online
Journal of Educational Technology, 9(3), 52-60.
[34] Hou, H. T. (2010b). Exploring the behavioral patterns in project-based learning with online discussion: quantitative content analysis and progressive sequential analysis. Turkish Online
Journal of Educational Technology, 9(3), 52-60.
[35] Huang, W. H. (2011). Evaluating learners’ motivational and cognitive processing in an online game-based learning environment. Computers in Human Behavior, 27(2), 694-704.
doi: 10.1016/j.chb.2010.07.021
[36] Huang, W. H., Huang, W. Y., & Tschopp, J. (2010). Sustaining iterative game playing processes in DGBL: The relationship between motivational processing and outcome processing. Computers & Education, 55(2), 789-797. doi:
http://dx.doi.org/10.1016/j.compedu.2010.03.011
[37] Hyönä, J., Lorch, R. F. J., & Kaakinen, J. K. (2002). Individual differences in reading to summarize expository text: evidence from eye fixation patterns. Journal of Educational
Psychology, 94(1), 44-55.
[38] Jeong, C. A. (2003). The sequential analysis of group interaction and critical thinking in online threaded discussions. The American Journal of Distance Education, 17(1), 25-43.
[39] Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers & Education, 51(1), 279-296. doi:
http://dx.doi.org/10.1016/j.compedu.2007.05.012
[40] Jonassen, D. (2003). Designing Research-Based Instruction for Story Problems.
and Instruction, 20(2), 172-176. doi: 10.1016/j.learninstruc.2009.02.013
[43] Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.
[44] Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science:
McGraw-Hill Medical
[45] Ke, F., & Grabowski, B. (2007). Game playing for maths learning: Cooperative or not?
British Journal of Educational Technology, 38(2), 249–259.
[46] Kebritchi, M., & Hirumi, A. c. (2008). Examining the pedagogical foundations of modern educational computer games. Computers & Education, 51(4), 1729-1743. doi:
http://dx.doi.org/10.1016/j.compedu.2008.05.004
[47] Keller, J. M. (2008). An integrative theory of motivation, volition, and performance.
Technology, Instruction, Cognition, and Learning, 6, 79-104.
[48] Kiili, K. (2005). Digital Game-Based Learning: Towards an experiential gaming model. The
Internet and Higher Education, 8(1), 13-24. doi:
http://dx.doi.org/10.1016/j.iheduc.2004.12.001
[49] Kim, K. H., Ban, S. W., & Kim, S. R. (2004). Emotion Recognition System Using Short-term Monitoring of Physiological Signals. Medical & Biological Engineering &
Computing, 42(3), 419-427.
[50] Latanov, A. V., Konovalova, N. S., & Yermachenko, A. A. (2008). EEG and EYE tracking for visual search task investigation in humans. International Journal of Psychophysiology,
69(3), 140. doi: 10.1016/j.ijpsycho.2008.05.340
[51] Lee, E. C., Woo, J. C., Kim, J. H., Whang, M., & Park, K. R. (2010). A brain–computer interface method combined with eye tracking for 3D interaction. Journal of Neuroscience
Methods, 190(2), 289-298. doi: 10.1016/j.jneumeth.2010.05.008
[52] Leony, D., Muñoz-Merino, P. J., Pardo, A., & Delgado Kloos, C. (2013). Provision of awareness of learners’ emotions through visualizations in a computer interaction-based environment. Expert Systems with Applications, 40(13), 5093-5100. doi:
http://dx.doi.org/10.1016/j.eswa.2013.03.030
[53] Lin, T., Imamiya, A., & Mao, X. (2008). Using multiple data sources to get closer insights into user cost and task performance. Interacting with Computers, 20(3), 364-374. doi:
10.1016/j.intcom.2007.12.002
[54] Linn, M. C., Clark, D., & Slotta, J. D. (2003). WISE design for knowledge integration.
Science Education, 87(4), 517-538. doi: 10.1002/sce.10086
[55] Liu, C.-C., Cheng, Y.-B., & Huang, C.-W. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57(3), 1907-1918. doi:
http://dx.doi.org/10.1016/j.compedu.2011.04.002
[56] Malliani, A., Pagani, M., Lombardi, F., & Cerutti, S. (1991). Cardiovascular neural regulation explored in the frequency domain. Circulation, 84, 482-492. doi:
10.1161/01.CIR.84.2.482
[57] Marina, P. (2009). Digital Game-Based Learning in high school Computer Science
education: Impact on educational effectiveness and student motivation. Computers &
Education, 52(1), 1-12. doi: 10.1016/j.compedu.2008.06.004
[58] Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
[59] Mayer, R. E., & Johnson, C. I. (2010). Adding Instructional Features that Promote Learning in a Game-Like Environment. Journal of Educational Computing Research, 42(3), 241-265.
doi: 10.2190/EC.42.3.a
[60] Murugappan, M., Ramachandran, N., & Sazali, Y. (2010). Classification of human emotion from EEG using discrete wavelet transform. Journal of Biomedical Science and
Engineering, 3(4), 390-396.
[61] NeuroSky Inc. (2009). Brain Wave Signal (EEG) of NeuroSky, Inc. from
http://www.neurosky.com/Documents/Document.pdf?DocumentID=77eee738-c25c-4d63-b 278-1035cfa1de92
[62] O’Neil, H. F., Wainess, R., & Baker, E. L. (2005). Classification of learning outcomes:
evidence from the computer games literature. The Curriculum Journal, 16(4), 455 – 474.
[63] Oblinger, D. (2004). The next generation of educational engagement. Journal of Interactive
Media in Education, 2004(8), 1-18.
[64] Pannese, L., & Carlesi, M. (2007). Games and learning come together to maximise
effectiveness: The challenge of bridging the gap. British Journal of Educational Technology,
38(3), 438-454.
[65] Papastergiou, M. (2009). Digital Game-Based Learning in high school Computer Science
Applications, 38(6), 7235-7242. doi: 10.1016/j.eswa.2010.12.028
[67] Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36(1), 36-48. doi:
10.1016/j.cedpsych.2010.10.002
[68] Picard, R. (1997). AffectiveComputing. Cambridge, MA MIT Press.
[69] Prensky, M. (2001). Digital Game-Based Learning. New York: McGraw-Hill.
[70] Randel, J., Morris, B., Wetzel, C., & Whitehill, B. (1992). The effectiveness of games for educational purposes: a review of recent research. Simulation & Gaming, 23(3), 261.
[71] Rashid, N. A., Taib, M. N., Lias, S., Sulaiman, N., Murat, Z. H., & Kadir, R. S. S. A. (2011).
Learners’ Learning Style Classification related to IQ and Stress based on EEG. Procedia -
Social and Behavioral Sciences, 29(0), 1061-1070. doi: 10.1016/j.sbspro.2011.11.339
[72] Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., & Keltikangas-Jarvinen, L. (2008). Thepsychophysiology of James Bond: Phasic emotional responses to violent video game events.
Emotion, 8(1), 114-120.
[73] Rayner, K. (2009). The thirty fifth Sir Frederick Bartlett lecture: eye movements and attention during reading, scene perception, and visual search. Quarterly Journal of
Experimental Psychology, 62, 1457-1506.
[74] Renkl, A., Atkinson, R. K., Maier, U. H., & Staley, R. (2002). From Example Study to Problem Solving: Smooth Transitions Help Learning. Journal of Experimental Education,
70(4), 293-315. doi: 10.1080/00220970209599510
[75] Robertson, J., & Howells, C. (2008). Computer game design: Opportunities for successful learning. Computers & Education, 50(2), 559-578. doi:
http://dx.doi.org/10.1016/j.compedu.2007.09.020
[76] Ruffman, T., Henry, J. D., Livingstone, V., & Phillips, L. H. (2008). A meta-analytic review of emotion recognition and aging: Implications for neuropsychological models of aging.
Neuroscience & Biobehavioral Reviews, 32(4), 863-881. doi:
10.1016/j.neubiorev.2008.01.001
[77] Schmid, P. C., Schmid Mast, M., Bombari, D., Mast, F. W., & Lobmaier, J. S. (2011). How Mood States Affect Information Processing During Facial Emotion Recognition: An Eye Tracking Study. Swiss Journal of Psychology, 70(4), 223-231. doi:
10.1024/1421-0185/a000060
[78] Schmutz, P., Roth, S. P., Seckler, M., & Opwis, K. (2010). Designing product listing pages—Effects on sales and users’ cognitive workload. International Journal of
Human-Computer Studies, 68(7), 423-431. doi: 10.1016/j.ijhcs.2010.02.001
[79] Srinivasan, N., & Gupta, R. (2010). Emotion-Attention Interactions in Recognition Memory for Distractor Faces. Emotion, 10(2), 207-215. doi: 10.1037/a0018487
[80] Stein, P. K., & Kleiger, R. E. (1999). Insights from the study of heart rate variability. Annual
Review of Medicine, 50, 249-261. doi: DOI: 10.1146/annurev.med.50.1.249
[81] Tüzün, H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y., & Kızılkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 52(1), 68-77. doi:
http://dx.doi.org/10.1016/j.compedu.2008.06.008
[82] Tran, Y., Wijesuriya, N., Tarvainen, M., Karjalainen, P., & Craig, A. (2009). The
Relationship Between Spectral Changes in Heart Rate Variability and Fatigue. Journal of
Psychophysiology, 23(3), 143-151. doi: 10.1027/0269-8803.23.3.143
[83] Tsai, M. J., Hou, H. T., Lai, M. L., Liu, W. Y., & Yang, F. Y. (2012). Visual attention for solving multiple-choice science problem: An eye-tracking analysis. Computers &
Education, 58(1), 375-385. doi: 10.1016/j.compedu.2011.07.012
[84] Valderrama, M., Alvarado, C., Nikolopoulos, S., Martinerie, J., Adam, C., Navarro, V., & Le Van Quyen, M. (2012). Identifying an increased risk of epileptic seizures using a
multi-feature EEG–ECG classification. Biomedical Signal Processing and Control(0). doi:
10.1016/j.bspc.2011.05.005
[85] Virvou, M., Katsionis, G., & Manos, K. (2005). Combining software games with education:
Evaluation of its educational effectiveness. Educational Technology & Society, 8(2), 54-65.
[86] Vogel, J. J., Greenwood-Ericksen, A., Cannon-Bowers, J., & Bowers, C. A. (2006). Using virtual reality with and without gaming attributes for academic achievement. Journal of
Research on Technology in Education, 39, 105-118.
[87] von Borell, E., Langbein, J., Després, G., Hansen, S., Leterrier, C., Marchant-Forde, J., . . . Veissier, I. (2007). Heart rate variability as a measure of autonomic regulation of cardiac
Education, 55(1), 178-187. doi: http://dx.doi.org/10.1016/j.compedu.2010.01.003
[89] Yang, B., & Lugger, M. (2010). Emotion recognition from speech signals using newharmony features. Signal Processing, 90(5), 1415-1423. doi: 10.1016/j.sigpro.2009.09.009 [90] Yang, C. H. (2012). Fuzzy fusion for attending and responding assessment system of
affective teaching goals in distance learning. Expert Systems with Applications, 39(3), 2501-2508. doi: http://dx.doi.org/10.1016/j.eswa.2011.08.102
[91] Yang, F. Y., Chang, C. Y., Chien, W. R., Chien, Y. T., & Tseng, Y. H. (2013). Tracking learners' visual attention during a multimedia presentation in a real classroom. Computers &
Education, 62(0), 208-220. doi: http://dx.doi.org/10.1016/j.compedu.2012.10.009
[92] Yang, Y. T. C. (2012). Building virtual cities, inspiring intelligent citizens: Digital games for developing students’ problem solving and learning motivation. Computers & Education,
59(2), 365-377. doi: http://dx.doi.org/10.1016/j.compedu.2012.01.012
[93] Zhang, C., Zheng, C.-X., & Yu, X.-L. (2009). Automatic Recognition of Cognitive Fatigue from Physiological Indices by Using Wavelet Packet Transform and Kernel Learning Algorithms. Expert Systems with Applications, 36(3), 4664-4671.
[94] Zhang, Q., & Lee, M. (2010). A hierarchical positive and negative emotion understanding system based on integrated analysis of visual and brain signals. Neurocomputing, 73(16-18), 3264-3272. doi: 10.1016/j.neucom.2010.04.001
[95] Zhang, Q., & Lee, M. (2012). Emotion development system by interacting with human EEG and natural scene understanding. Cognitive Systems Research, 14(1), 37-49. doi:
10.1016/j.cogsys.2010.12.012
[96] Zhao, C., Zhao, M., Liu, J., & Zheng, C. (2012). Electroencephalogram and
electrocardiograph assessment of mental fatigue in a driving simulator. Accident Analysis