一個防災教育遊戲設計之研究

國 立 交 通 大 學

資訊科學與工程研究所

碩

士

論

文

一 個 防 災 教 育 遊 戲 設 計 之 研 究

A Study of Adventure Game Design

for Disaster Prevention Education

研 究 生：黃靖雅

指導教授：曾憲雄 博士

一個防災教育遊戲設計之研究

A Study of Adventure Game Design for Disaster Prevention

Education

研 究 生：黃靖雅 Student：Ching-Ya Huang

指導教授：曾憲雄 博士 Advisor：Dr. Shian-Shyong Tseng

國 立 交 通 大 學

資 訊 科 學 與 工 程 研 究 所

碩 士 論 文

A Thesis

Submitted to Institute of Computer Science and Engineering College of Computer Science

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master

in

Computer Science June 2009

Hsinchu, Taiwan, Republic of China

一個防災教育遊戲設計之研究

研究生: 黃靖雅 指導教授: 曾憲雄博士 國立交通大學資訊學院 資訊科學與工程研究所

摘 要

防災教育在九年一貫的教學中一直是一門重要且基礎的課程。過去學習防 災教育的概念應用通常倚賴實際的操作演習，這種方式成本較高。冒險型遊 戲(Adventure Game)提供一個虛擬的環境讓學生在其中做中學，並強調利用 觀察與判斷的能力在遊戲中達成各種關卡，因此很適合作為防災教育的學 習。然而由於冒險型遊戲通常含有多線式複雜的劇情，因此製作一個冒險型 遊戲對老師而言是一個很大的負擔。另一方面，學生在遊戲裡下的判斷以及 執行的動作皆是基於他們的知識概念，因此老師可藉由遊戲進行形成性評 量 。 在 這 篇 論 文 中 ， 我 們 提 出 一 個 Adventure Game-Based Formative

Assessment 的架構來幫助老師建構出可進行形成性評量的冒險型遊戲。在遊

戲編輯的部分，我們提出一套 adventure game script 的方法，透過互動式導 引的方式引導老師填寫我們定義的劇本，並將各個教學目標串聯出一個多線 式的遊戲劇情。劇本填寫完成後，我們的系統將會幫助老師進行實際遊戲的

建置。有了實際的遊戲後，透過我們提供的 Game Behaviour Analysis 方式可

幫助老師進行形成性評量。透過我們提出的架構，我們建構出一個高樓逃生 遊戲，並利用此遊戲對 35 位新竹市立光武國中的國一學生進行形成性評量。 我們的結果顯示，我們提出的分析方式的確可以幫助老師了解學生的認知概 念。

A Study of Adventure Game Design

for Disaster Prevention Education

Student:Ching-Ya Huang Advisor: Dr. Shian-Shyong Tseng Institute of Computer Science and Engineering

Nation Chiao Tung University

Abstract

Disaster prevention education is a basic and important course in K-12 education. Adventure games require observations and correct decisions to reach the goal, and thus can be used for disaster prevention teaching with far less costs than doing real exercises. However, making games are big burdens for teachers because of the complex cause-and-effect relations in adventure games. Another issue is that teachers want to do formative assessment in the game since learners’ behaviours in the game give teachers an insight into learners’ cognitions. In this thesis, we propose an Adventure Game-Based

Formative Assessment framework to help teachers construct assessable adventure games.

The adventure game script approach is proposed for game construction. This approach helps teachers tailor their game scenarios from different learning objectives through interactive guidance based on a pre-defined script format. After filling out the script, our system will transform the obtained information into the real game by game transformation process. Game Behaviour Analysis provides teachers with two different levels of information: conceptual level and the behaviour details level. In conceptual level, learners’ task sequences (the order of completing the tasks) will be mapped to their misconceptions right after they finish playing the game. The organized logs can be used to do more analysis such as discovering behaviour patterns in the game through behaviour mining in behaviour details level. A disaster prevention game called “Escaping the Burning Building” is constructed using our adventure game script approach. The game is played by 35 seventh grade students. The analysis results provided by our Game

Behaviour Analysis show our method can really help teachers find out learners’

misconceptions and behavior patterns in games.

Keywords: game-based learning, adventure game design, authoring method, behavior mining

致謝

首先誠摯的感謝指導教授：曾憲雄博士。在這兩年碩士生涯中，學習到 最多的就是跟隨曾教授做研究的過程，在毫無頭緒時指引我方向，在我感到 迷惘時給予當頭棒喝，並讓我學習到知識工程領域的研究方法、邏輯思考及 如何表達的能力，這將使我終生受用不盡。此外不管在學習研究方面或是做 人處事，曾教授時常與我們分享他過去寶貴的經驗，並且時時關心我們每一 位同學的進度，真的是非常感謝曾教授對我們的耐心及用心。同時也要感謝 我的口試委員，孫春在教授、賀嘉生教授、葉耀明教授和廖岳祥教授，他們 給予了我相當多寶貴的意見，讓本論文更有意義與價值。 再來要感謝的是林喚宇學長、劉怡利學姊和曲衍旭學長，這段期間內， 即使他們很忙還是願意騰出時間與我討論並給我建議，協助我修改論文和解 決系統方面的問題，在他們身上我學到了做研究該有的態度與方法，真的是 非常謝謝他們！還有實驗室的同窗們，惠君、祖淵、啟珺、士緯、世恆，在 這兩年的時光裡，和你們同甘共苦、互相扶持鼓勵，才能完成了這篇論文， 能認識你們真的很開心。還有其他在身邊鼓勵我的朋友們，雖然無法在此一 一提及，但我心裡真的非常感激有你們在我身邊。 最後要謝謝我最愛的家人，總在背後默默支持著我，為我打氣鼓勵，是 我能完成這篇論文的最大動力，真的非常謝謝你們。日後，我會更加努力地 繼續前進，不辜負你們的期望。 小雅 2008 年 7 月于新竹

Table of Content

摘 要 ... iii

Abstract ... iv

致謝 ... v

Table of Content ... vi

List of Figures ... vii

List of Tables ... viii

Chapter 1 Introduction ... 1

Chapter 2 Related Works ... 5

2.1 Authoring Approach: Script ... 5

2.2 Formative Assessment in Games ... 6

2.3 Modified Multistage Graph (MMG) Model ... 7

Chapter 3 Adventure game based Formative assessment ... 9

3.1 The adventure game script approach ... 10

3.2 Behavior pattern analysis ... 11

3.3 Comparison ... 12

Chapter 4 Adventure Game Script Approach ... 15

4.1 Game Scenario for Disaster Prevention Education ... 16

4.2 The Adventure Game Script Approach ... 17

Chapter 5 Game Behavior Analysis ... 24

5.1 Concept Status ... 24

5.2 Organized Log by MMG ... 25

5.3 Three levels of information ... 27

Chapter 6 Implementation & Experiment ... 30

6.1 Adventure Game Script approach ... 30

6.2 Game Transformer ... 31 6.3 Game display ... 32 6.4 Formative Assessment ... 35 6.5 Discussions ... 40 Chapter 7 Conclusion ... 42 Reference ... 44

List of Figures

FIGURE 1.OBJECT ORIENTED INTERACTIVE CONTENT MODEL (OOICM) ... 6

FIGURE 2.MODIFIED MULTI-STAGE GRAPH (MMG)... 8

FIGURE 3.SYSTEM ARCHITECTURE OF ADVENTURE GAME BASED FORMATIVE ASSESSMENT .... 10

FIGURE 4.THE GAME SCENARIO OF ESCAPING THE BURNING BUILDING ... 17

FIGURE 5.THE DIALOGUE ALGORITHM OF ADVENTURE GAME SCRIPT APPROACH ... 18

FIGURE 6.SCRIPT CONTENT ... 18

FIGURE 7.TASK SETTINGS OF THE ADVENTURE GAME SCRIPT ... 20

FIGURE 8.RULE RELATION SETTINGS OF THE ADVENTURE GAME SCRIPT ... 21

FIGURE 9.AN INTERACTIVE GUIDING EXAMPLE ... 22

FIGURE 10.THE MAPPING BETWEEN ADVENTURE GAME SCRIPT METHOD AND OOICM ... 22

FIGURE 11.EACH TASK IN THE GAME REPRESENTS A LEARNING OBJECTIVE ... 25

FIGURE 12.PROCEDURAL LOGS ... 26

FIGURE 13.THE FIXED-LENGTH RECORD OF EACH LEARNER ... 27

FIGURE 14.THE AUTHORING INTERFACE FOR CREATING NEW GAME ... 31

FIGURE 15.THE AUTHORING INTERFACE FOR ITEM EDITING ... 31

FIGURE 16.THE TRANSFORMATION PROCESS OF THE ADVENTURE GAME SCRIPT METHOD ... 32

FIGURE 17.THE “GOTO” ACTION OF THE GAME. ... 33

FIGURE 18.THE INVENTORY FUNCTION OF THE GAME ... 34

FIGURE 19.THE “OBSERVE”,“USE” AND “TAKE” FUNCTION OF THE GAME... 34

FIGURE 20. THE INSTANT FEEDBACK AND OTHER HINT MESSAGE OF THE GAME ... 35

List of Tables

TABLE 1.THE COMPARISON BETWEEN ADVENTURE GAME SCRIPT APPROACH AND OTHER METHODS ... 12

TABLE 2.THE COMPARISON BETWEEN BEHAVIOR PATTERN ANALYSIS METHOD AND OTHER METHODS ... 13

TABLE 3.THE NUMBER OF LEARNERS ACHIEVED IN DIFFERENT ENDINGS. ... 36 TABLE 4.THE COMMON BEHAVIOR PATTERNS OF THE “ESCAPING THE BURNING BUILDING”

GAME. ... 38 TABLE 5.THE FREQUENT BEHAVIOR PATTERNS OF THE “ESCAPING THE BURNING BUILDING”

GAME. ... 38 TABLE 6.THE SMALL GROUP BEHAVIOR PATTERNS OF THE “ESCAPING THE BURNING BUILDING” GAME. ... 40

Chapter 1 Introduction

Disaster prevention education is a basic and important course in K-12 education. Learning the knowledge of disaster prevention includes understanding the concepts of disaster prevention and the problem solving ability in the real context. These are respectively called declarative knowledge and procedural knowledge in cognitive psychology[1]. However, traditional disaster prevention education often focuses on declarative knowledge learning. Learners are difficult to learn procedural knowledge for the high cost of doing real exercises. To reduce the high cost, game-based learning happens to provide good opportunities for procedural knowledge learning since learners can learn by doing in the simulation environment. There are several types of game that are often applied for educational use. Puzzle games are often mentioned for many problem solving skills including logic and strategy, while RPG games allow learners to experience as a specific role. However, procedural knowledge for disaster prevention education emphasizes the decisions made in real situations. Adventure games require players to make attempts and decisions in the game, and different decisions may result in different endings. Therefore, adventure games are suitable for disaster prevention learning exactly.

Nevertheless, there are still some problems on previous researches[2-4] of game-based learning. Problems as teachers are hard to construct a game, especially adventure game, since game scenarios are composed of the complex cause-and-effect relations. In addition, teachers are also hard to formatively

evaluate learners' behaviors for discovering their misconceptions in the learning game even after they have the desired educational games. In this thesis, we propose an Adventure Game-Based Formative Assessment framework to help teachers construct assessable adventure games.

In game scenario construction phase, an adventure game script approach is proposed. This approach helps teachers tailor their game scenarios from different learning objectives through interactive guidance based on a pre-defined script format. On the one hand, the accomplished game scenario is composed of many tasks. Each task can be mapped to different learning objectives, and this makes the game more educational. On the other hand, the content of the filled-out script covers all the game construction information of system layer. Therefore, the desired games will be realized through our game transforming engine after the script is filled out under our guiding. This is really time-saving and easy-to-edit for teachers to make their own educational games.

Another issue is that how to assist teachers in discovering learners’ misconceptions of procedural knowledge for further remedy instructions. Formative assessments are often applied to help teachers with this. Formative assessments are part of instructions designed to provide crucial feedback for teachers and students, and the assessment results inform teachers of what has been taught well and not so well. Games are suitable for doing formative assessment since learners are immersed in the simulated environment to make decisions or attempts, and their behaviors or decisions reflect their knowledge in mind. However, even when teachers have the educational games, they are still hard to

evaluate learners. Because teachers neither can capture learners’ behaviors in the game nor has a systematic approach to analyze these behaviors. For this, we propose a method called Game Behavior Analysis. We want to provide teachers with two levels of information, which are the concept level and the behavior details level respectively for formative assessment. Task sequences are used for the information of concept level. Learners’ task sequences will be mapped to their misconceptions right after they finish playing the game. The organized logs with sufficient information are for the behavior details level. These logs can be used to do more analysis such as discovering behavior patterns in the game through behavior mining. From the analytical results, teachers will be able to find out learners’ misconceptions and other behavior details.

The evaluation criteria of our method include the effectiveness of game construction approach and the effectiveness of game behavior analysis. A disaster prevention game called “Escaping the Burning Building” is constructed using our

adventure game script approach by the teacher. The game is played by 35 seventh

grade students of Guang-Wu junior high school in Hsin-Chu city. The analysis results show our method can really help teachers find out learners’ misconceptions behavior patterns in games.

The rest of this thesis is organized as follows. Chapter 2 shows the related works of script authoring approaches and some previous researches on assessment in games. In Chapter 3, we provide an overview of our Adventure Game-Based

Formative Assessment framework. Chapter 4 describes the Adventure Game Script Approach. In Chapter 5, Game Behavior Analysis is proposed for formative

assessment in the games. And in the Chapter 6, experiments are done to verify our proposed method. Finally, conclusions and future works are given in Chapter 7.

Chapter 2 Related Works

2.1 Authoring Approach: Script

Game script is widely used in game-making field. The components of a game script include scenes, objects, characters, music and so on. Based on the structure of game scripts, we can categorize game scripts into narrative and structured. The narrative game scripts usually are for those professional game script writers. The descriptive structure of these narrative game scripts is close to human thinking, which makes it easier for people to express their game scenarios. Nevertheless, the descriptive structure is hard for system processing. There must be another transformation or human-intervening process to organize the script and obtain the structured information for system processing. Previous research [5] has proposed a <e-game> project, which is a documental approach to develop adventure games. The editor first has to fill out the descriptive script in natural language, and then ask the editor to mark up the filled-out script with the <e-game> XML language. The script thus becomes a structured document which is easy for system implementation. However, structured document is hard to understand or edit for normal users, even the author of the <e-game> project admitted that it is not easy for normal users to tag with the XML language.

For game-script authoring, our previous research has proposed an Object Oriented Interactive Content Model (OOICM) [6]. The OOICM allows user to design the high level game knowledge and we help them get their desired game.

The OOICM takes scenario construction as the first step to design RPG games. The editor can focus on scenario editing first and then add other implementation details later. As shown in Figure 1, OOICM is composed of three components which are Story Control Flow (SCF), Activity, and Scene Object (SO). Story Control Flow is a sequence of sub-goals. Activity denotes the action that SOs can perform. Some activities may achieve sub-goals in the SCF. Scene Object denotes the objects that constitute the scene of learning content. In order to complete the playing of the game, the player has to perform some activities to achieve the sub-goals in the SCF. Therefore, SCF and frames maintaining activities and objects information are applied for editors to express the game scenario, and the game will be realized by the OOICM engine. Though the authoring method is clearer than structured XML documents, it is still hard to edit when the game scenario is complex and multidimensional.

Figure 1. Object Oriented Interactive Content Model (OOICM)

2.2 Formative Assessment in Games

Tan[7] mentioned that the environment should include formative assessment methods to allow learners to monitor their learning and enable them to correct their mistakes and misconceptions, and ”Challenge Zone quiz” with a dynamic

7

assessment mechanism was provided for students for evaluating the students’

understanding of ecosystem behavior.

Shih[8] proposed a method to find out learners’ problem-solving and cognitive level by observing the playing route of each learner. However, the proposed method is non-automatic, which requires teachers to keep track of the learner’s playing route by his/her side. This is really a burden for teachers since

they have to handle a large number of learners. What is more, the method only considers the relations between tasks, without the details about each learner’s behavior patterns.

Moreno-Ger [9], in the series of the <e-game> project researches, proffered an assessment technique which allows game designers to indicate what tasks are relevant for the learning process. Learners’ behaviors in the indicated task will be recorded when playing and reported to teachers after they finish the game. Nevertheless, learners’ behavior patterns are various. Only observing the

appointed task will lack other behavior pattern details which might contain more meaningful information.

2.3 Modified Multistage Graph (MMG) Model

A Modified Multi-stage Graph (MMG) [10] model is a decision-making model of games. The model can express the effect of “different decisions,

different game flows”. All the players’ actions can be traced on the model and thus can be used to find the cause of game failures. MMG consists of four main components which are states, stages, edges and working memory. Stages are each decision branches in the game and states represent the possible choice of the stage.

Edges are used to connect different results due to different choices. The reason to be called “modified” is the use of working memory and rules attached on the

edges. Working memory can be provided for each stage’s status information while rules on the edges can be used to enable/disable some possible choices of the stage.

Figure 2 shows the modeling ability of a farming game called “Banana Farm” [10]. In Figure 2, there are several predefined actions in each stage which can detect the meaningful behaviors and inquiry process of players. For instance, in the first stage - BTS (Banana Types Selection), each state (TypeA, TypeB, TypeC) of the stage means the possible actions, i.e. choosing which types of bananas. Edges are used to connect with different decisions’ results. Besides, the working memory of MMG meets the need of global variables.

Figure 2. Modified Multi-stage Graph (MMG)

The information of MMG provides a clear high-level meaning of players’ behaviors through different paths. Therefore, MMG is suitable as our analytical model.

9

Chapter 3 Adventure game based

Formative assessment

Adventure games require observations and correct decisions to reach the goal, and thus can be used for procedural knowledge learning of disaster prevention education with far less costs than doing real exercises. What is more, learners’ behaviors in the game can give teachers an insight into learners’ cognitions. Therefore, adventure games are suitable to do formative assessments for disaster prevention education. But it is not easy for teachers to construct an adventure game because of the complex cause-and-effect relations of the adventure game. Even after teachers have the desired games, they are still hard to evaluate learners since they lack a systematic approach to analyze learners’ behaviors in the game.

We hope to help teachers design the adventure games for disaster prevention education and help them do the formative assessment in games. Therefore, we propose the adventure game based formative assessment framework as depicted in Figure 3. Once the teacher has a game scenario, he/she can tailor each learning objective to an adventure game through filling out the script under our interactive guidance. After completing the script with all the necessary information, our game-transforming engine will help the teacher construct his/her desired game. The game will be played by learners as an assessment approach. Learners’ task

concept status report, while their behaviors in the game will be logged by our MMG model. The organized logs can be applied to different behavior pattern analysis methods and get different findings.

Figure 3. System architecture of Adventure game based formative assessment

3.1 The adventure game script approach

In game construction, planning the game scenario should be the first concern when making a game. For teachers, they want to make educational games, which are often composed of different tasks of learning goals. How to connect these tasks with complex cause-and-effect relations is hard for teachers. Hence, we apply the previous OOICM as our content model since OOICM drew out the scenario construction from the whole game-making process and allow teachers

11

focus on tailoring the complex game scenario first. However, as mentioned before, the authoring method of OOICM is hard to perceive by teachers. For this issue, the Adventure Game Script approach is used. There exists a pre-defined script whose content covers all the game-transforming information of system layer and will guide teachers to edit their game scenarios in more human-thinking way through dialogues. Teachers will then compose their game scenarios through interactive guidance based on this pre-defined format script. This approach helps teachers express their games in their thinking way and allows us to get the game-transforming information of system layer. The system layer information can be used to realize the real games by our game construction engine.

3.2 Behavior pattern analysis

But even when the teachers have the desired games, it is still hard for them to do the formative assessment in games. For this, we have proposed a Game

Behavior Analysis method, which provides teachers with two different types of

information: task sequences and organized logs. Since it is the teacher who designs the game, each task in the game will represent a learning goal for assessment. Thus, learners’ task sequences can correspond to their cognitive concept levels of the combined learning objectives and formed a report right after they play the designed game. Moreover, teachers have to deal with many learners in the meantime. How to do formative assessment to many learners is another concern. For this demand, we apply previous research MMG as a model to organize procedural logs. The organized logs provide useful and sufficient

information about actions, action-taken locations, action-taken frequency and action-taken time of learners in game for more analysis. We can apply behavior mining to these logs for discovering learners’ behavior patterns.

3.3 Comparison

In this section, we compare our proposed method with other related researches. Since our adventure game based formative assessment framework includes authoring and analysis, we compare our ideas of adventure game script approach and behavior pattern analysis method with other researches.

Table 1. The comparison between Adventure Game Script approach and other methods

Game transformation Human thinking Narrative script Hard Similar

Structured script Easy Dissimilar

OOICM Easy Dissimilar

Adventure Game Script Easy Similar

As shown in Table 1, the narrative script is hard for real game transformation even it is the most similar to human-thinking. For structured scripts, it’s much easier to transform the script information into real games. However, structured scripts are hard for teachers to edit and understand. OOICM, as mentioned before, provides a new authoring method which is more understandable than structured script and is easy for game transformation. But even so, the authoring method of OOICM still contains too many technical details which are burdens for teachers. The Adventure Game Script we proposed in this thesis improves this defect of the OOICM. Teachers can fill out the script under our interactive guiding, just like someone is asking them questions. The script format and process are pre-defined

13

which can be proven for real game construction. The real games can be constructed quickly after filling out our script.

Table 2. The comparison between Behavior Pattern Analysis method and other methods

Monitoring Cost

Analysis Effectiveness Observe learning routes by teachers High High

Set checkpoints Low Low

Behavior Pattern Analysis Low High

In Table 2, the row named “Observe learning routes by teachers” is proposed by Shih[8]. Learners’ learning routes are recorded by teachers. The learning routes of learners represent their problem-solving abilities. However, it is really a load for teachers if they want to know every learner’s behaviors in the game. And the

analysis effectiveness is high since it is the teacher who sits next to the learner and records the information. Another assessment approach proposed on previous research is to set the check points at some specific tasks. This approach largely reduces the monitor cost. But learners’ behaviors are various, only observing the

appointed task will lack other behavior pattern details and thus the analysis effectiveness is low. The monitor cost of the proposed Behavior Pattern Analysis method is quite low, because there are log techniques which assist teachers in capturing learners’ behaviors in game. What is more, the logs of the Behavior

Pattern Analysis method are flexible to do more analysis. Because the analysis

method has been modulized in the framework, every action of learners can be captured regardless of what kind of game it is. The behavior information can be applied to different analysis scenarios and teachers can receive different findings.

The difficulty of the proposed Adventure Game Based Formative Assessment framework includes two parts. The problem of adventure game script approach in game construction phase is how to acquire the game-transformed information of system layer in more human-thinking way. As for the game behavior analysis method, the teachers are provided with task sequences and organized logs. How to extract meaningful information from these raw data is the difficulty of the method. Behavior pattern mining is frequently used for discovering learners’ behavior patterns. This method results in only frequent behavior patterns of most learners being discovered. However, unique behavior patterns of small groups are meaningful to teachers, since teachers have to teach students in accordance with their aptitude. In the following chapters, we will elaborate on how we overcome these difficulties through our proposed method.

Chapter 4 Adventure Game Script

Approach

According to the previous research OOICM, games can be realized through information obtained from story control flow (SCF) and frames of the objects and activities. Even so, as mentioned before, the authoring method of OOICM contains too many technical details for teachers to understand and to edit. Therefore, we apply the adventure game script approach which is a dialogue-based, interactive guiding method to acquire the necessary game-transforming information.

The script format and script-editing process are pre-defined. According to OOICM, we draw out the scenario construction from the whole game-making process. For educational games, game scenario is formed with different tasks of learning objectives. Therefore, teachers are first asked to provide tasks. Since there must be some necessary items to complete each task, item settings are required after specifying which tasks. In order to connect each task and form the whole scenario, rule settings are used. Other details for game display will also be obtained in the defined script. As a result, we define the game script format and editing procedure followed by the logic described above.

4.1 Game Scenario for Disaster Prevention

Education

We want to design a game called “escaping the burning building” for disaster

prevention education. The game scenario is shown in Figure 4, and there are 6 endings in this game. We design many tasks for the game, and different combinations of these taken tasks result in different endings.

Jumping out of the building is a mistake that often made in the fire emergency. We allow learners to jump out of the building from 6th floor, but the game will end due to the terrible injury. Calling for help is very important when the accident takes place. Therefore, there allocated a telephone to see whether learners will call for help at the first place. Another threaten in the fire scene is smoke. Therefore, learner should cover their mouth and nose with the wet towel when escaping. And in this game, it is the electrical fire and thus bubble extinguisher is useless. We will provide learners with two types of extinguisher to check whether they are aware of the environment condition and correct extinguisher-type selection. Learners should choose the correct escaping route by observing the environment. In the game, we will hint learners that it is the 5th floor (the player is now at the 6th floor) that caught fire; therefore, downstairs is full of smoke and go downstairs would end of the game since it is the fire-starting spot. Learners should go to the penthouse to wait for help. Other scenario settings are described as Figure 4.

17

Figure 4. The game scenario of escaping the burning building

4.2 The Adventure Game Script Approach

We apply the interactive dialogue approach to help teachers finish their script. The dialogue algorithm is described as Figure 5. The first phase called “Flow and Control Setting” is to develop the game scenario. In this phase, teachers first have

to design tasks of different learning objectives, and specify what items and scenes are required. After that, they can tailor the scenario flow by assigning which tasks should be accomplished or not before one specific task can be done. The second phase, “Scene and Item Manager” is used to manage the relations of items and scenes. Teachers will be asked about items located in which scene, the connective relations of scenes and so on. The third phase is for some game display settings such as pictures and locations of these pictures. Teachers would complete these settings through our interactive guiding, just like someone is asking them questions and they answer.

Figure 5. The dialogue algorithm of adventure game script approach

19

The conceptual script is shown in Figure 6. Each part of script consists of different settings. In task settings, teacher has to define what task they want to put in the game and what task is the final task. In item settings, teacher has to set some usage constraints. For instance, whether the item can be taken by the player and put in player’s inventory. And in scene settings, the contained item and

locations of these items will be assigned. Task relation settings help teachers connect each task through the cause-and-effect relation, and design the effects after each task is done. The teacher can thus focus one event at a time, and get the whole complex, inter-related scenario after they fill out the script.

Example 1: Use Adventure Game Script approach to edit the “escaping the burning building” game

In the example, we want to design an adventure game about escaping the burning building for disaster prevention education. In the “Flow Control Settings” phase, first we will have to assign what kind of tasks we want to allocate in this game, such as “use the extinguisher at fire”. The filling script and the current

game scenario are shown on the upper side and the lower side respectively in Figure 7.

After that, we have to indicate what items and scenes are necessary for each task. For instance, “fire” and “extinguisher” are needed for the task “use the extinguisher at fire”, and scenes like “hallway” and “bathroom” are added since the “fire” is located in the “hallway” and the “extinguisher” is in the “bathroom”.

Figure 7. Task settings of the adventure game script

Rule settings will be guided steps by steps: For each task, specify what tasks have to be done before and what will happen to items or scenes after the task is completed. For example, the teacher will specify the task “take the extinguisher”

must be achieved before the task “use the extinguisher at fire”. And after achieving the task “use the extinguisher at fire”, the item “fire” should disappear. The rule settings and the game scenario after setting the rules are shown in Figure 8.

21

Figure 8. Rule relation settings of the adventure game script

In the second phase “Scene and Item Manager”, scene relations (such as “Hallway” is in the east side of the “bathroom”) and items locate in which scenes (such as “extinguisher” is in the “bathroom”) are pointed out. And the last phase

of the adventure game script is about game display details. Pictures of items and scenes will be uploaded and their locations of the screen will also be indicated. Figure 9 shows a whole process of the interactions between editors and the authoring system.

Figure 9. An interactive guiding example

23

The script content obtained from these three phases covers all the acquired information of OOICM: Story control flow consists of nodes and connectors which respectively represent tasks and the cause-and-effect relations between tasks. This information can be acquired through our “Flow Control Setting” phase

as shown in Figure 10. Other information maintained by frames in OOICM includes the actions in tasks and some objects (items and scenes) settings of tasks. These settings can also be obtained in our “Scene and Item Manager” phase and “Display” phase. And according to the OOICM research, the information

maintained in story control flow and frames is adequate for a real game construction. Therefore, after filling out the script, we can get all the necessary game-transforming information of the system layer.

Chapter 5 Game Behavior Analysis

Once the games in teachers’ mind can be realized, teachers would be able to

do formative assessment to learners through the games. In this chapter, we will illustrate how to provide teachers with learners’ behavior information in the

games.

5.1 Concept Status

Each task in the game is elaborately designed by the teacher and can be mapped to some learning objectives in procedural knowledge. Figure 11 shows an example of this. In this example, the task “get the powder extinguisher” is used to

diagnose whether learners are aware of fire categories and the corresponding extinguisher types, while another task “use the wet towel to cover mouth and nose” checks whether learners know how to prevent from being choked in the fire

scene. Therefore, learners’ task sequences (the order of completing each task) can correspond to their cognitive concepts and formed a concept status report right after they play the designed game.

25

Figure 11. Each task in the game represents a learning objective

Example 2:

Suppose the task sequence of the learner, John in the “escaping the burning building” game is: <usePhone, …, coverMouthAndNose, …, takeElevatorOut >

We can interpret that John has the concept of calling 119 for help at once in the fire emergency, and John also knows wet towel can be used to prevent him from being choked. But John doesn’t seem to know it is the last choice to jump out of the building when in the burning building.

5.2 Organized Log by MMG

5.2.1 Procedural Logs

Learners’ actions will be logged and organized by MMG model. MMG is

script. Learners’ action sequence and action frequency can thus be traced by

MMG and become organized logs, as shown in Figure 12. Logs of each learner can represent one transaction. Transactions of all learners can be applied to behavior mining for discovering behavior patterns. The behavior patterns of a student can be interpreted as the student’s learning levels or whether this student

can apply what he/she has learned.

Figure 12. Procedural Logs

5.2.2 Behavior feature vector

If we want to cluster learners, first we have to build up a mapping from organized logs to fixed-length records. To get fixed-length records, we divide each action log into three stages which are action logs in earlier stage, action logs in middle stage and action logs in later stage. The record is shown as Figure 13, also consists of three stages. Each stage of the record will list all the actions defined in the script, and the vector value is determined according to the action logs of the corresponding stage. The vector value will be 0 if the learner did not

27

do this action in this stage. The value 1 means the learner has done this action once in this stage, while the value 2 represents frequent actions of this stage. This mapping provides a stable approach to obtain the general situation of learners’ actions in the game. The general situation can be the input for clustering though it might lose some detailed information.

Figure 13. The fixed-length record of each learner

5.3 Three levels of information

Assume that there is a teacher who wants to do formative assessment to his students through the constructed game and expects to receive different levels of information. As mentioned before, learners’ behaviors in the game can be interpreted as their cognitions. Since we already have the procedural logs of learners in the game, data mining techniques are often applied to explore data in search of consistent patterns and produce relevant analysis results. Behavior pattern mining is frequently used for discovering behavior patterns. Different information needs will require different mining scenarios. If teachers want to know most learners’ behavior patterns, standard behavior pattern mining is used.

If teachers feel like to know the behavior patterns of each type learners, clustering is often applied.

We provide three types of patterns for teachers by behavior pattern mining: there are patterns with high support, almost all the learners will have these

patterns and are often due to game settings; we call this type of patterns “common behavior patterns”. “Frequent behavior patterns” are patterns found in most

learners filtering out the common behavior patterns. Patterns in each small group of learners are called “small group behavior patterns”. Small group behavior

patterns are aimed to provide teachers with unique behavior patterns of each group.

5.3.1 Common behavior patterns

Some actions frequently appeared in most learners’ logs simply because

these actions are part of the scenario settings or due to this type of game. We call this type of actions “common behavior patterns”. For instance, if there locates a

telephone in the middle of the scene and this telephone is the only item that can be used in this scene, then it is expectable that “use phone” will be frequent actions in most learners’ logs. However, this type of information is slightly meaningful

for teachers. We have to inform teachers of this common behavior patterns for focusing on more meaningful analysis.

For discovering this common behavior patterns, first we have to apply sequential pattern mining for all learners. The high support patterns (close to 100%) in frequent sequence patterns can be interpreted as common behavior patterns.

29

Most learners’ behavior patterns will be the biggest concern for teachers. We call this type of behavior patterns “frequent behavior patterns”. The sequential

pattern mining is again applied to all learners. As mentioned above, common behavior patterns are less meaningful for teachers. Hence, the frequent behavior patterns will exclude the common behavior patterns from the frequent patterns of the sequential pattern mining result. The frequent behavior patterns can be used to analyze what most learners will do in the game and what these actions mean.

5.3.3 Small group behavior patterns

While most learners’ behavior patterns are significant, the behavior patterns

of small groups contain more precise information for teachers. This type of information, “small group behavior patterns” as we called, is also critical since

teachers would like to teach students in accordance with their aptitudes.

For discovering small group behavior patterns, first we cluster students with similar behavior patterns. Fixed length record is necessary when doing clustering. We use actions, time segment and action frequency as behavior features to transform learner’s behavior sequences into fixed length behavior feature vector

as mentioned above.

After doing clustering to all the learners, there formed several small groups. And the behavior patterns of each small group can be obtained through filtering out the common behavior patterns from the sequential pattern mining result of the small group. Teachers then can find out what unique behavior patterns in each group and remedy these misconceptions.

Chapter 6 Implementation & Experiment

6.1 Adventure Game Script approach

Now that we have the game scenario in our mind (as described in the previous chapter) the adventure game script approach will help us express our game story by filling out the script. The authoring interface of the proposed method is depicted as Figure 14 and Figure 15. As we can see in the figure, each setting is shown in one tab with clear step indications. The setting processes are quite identical, which makes it effort-saving for teachers to edit after they have run through one setting. Teachers can add new tasks, items and indicate the relations of these tasks following by our guidance. They can switch back to any previous tabs to modify their settings before, and they can also delete the settings if they want. However, deletion may have great effects on other tasks or items, since the tasks and items are weaved tightly. For this problem, our system will warn teachers of the influenced scenario settings when delete a task, item, or a relation.

31

Figure 14. The authoring interface for creating new game

Figure 15. The authoring interface for item editing

6.2 Game Transformer

After the script is filled out, the real game will be realized by our transformation process as described in Figure 16. The filled out script will form an XML document and the XML file will be parsed into different runtime

maintainable objects. The task relations will be parsed as rule facts, which are the input of the inference rule. The display of the real game is developed by these runtime maintainable objects including items and scenes. Whenever the learner interacts with the game, one of the rules will be matched, and the game will give response to the action according to the defined rule facts by inference engine. Consequently, each action of the learner will be responded and “interactivity”, one of the key elements of educational games, is thus reached.

Figure 16. The transformation process of the adventure game script method

6.3 Game display

The game will be realized through our game construction engine after the necessary information is obtained. The game is portrayed as Figure 17 to Figure 20. Learners are able to move to different scenes (as shown in Figure 17), put obtainable items in the inventory (as shown in Figure 18), observe scenes and

33

items, and use items at other items or learner himself (as shown in Figure 19). Each action will be given feedbacks by some descriptions or item variations as the teacher designed (as shown in Figure 20), thus learners can totally immerse in the environment and make attempts. Their behaviors will reflect some truly response in the scenario.

Figure 17. The “goto” action of the game.

Display which scenes can be reached next.

Figure 18. The inventory function of the game

Figure 19. The “observe”, “use” and “take” function of the game.

Inventory

Observe scenes or items, Use items at other items

35

Figure 20. the instant feedback and other hint message of the game

6.4 Formative Assessment

After constructing the real game, the game is applied to 35 seventh-grade students of Guang-Wu junior high school in Hsin-Chu city. The experiment is to verify whether the game can really help teachers find out learners’ misconceptions for disaster prevention.

6.4.1 Concept Status

As mentioned before, the concept status of learners can be immediately reported (as shown in Table 3). If we start from observing the endings of all learners, we can find that only one student safely escaped and 23 students failed the game from falling out of the building. This can be initially interpreted as most students do not have enough correct concepts to get out and even have the

Instant Description Feedback Currrent Location Items

incorrect concept of jumping out of the building. Four learners had chosen the correct escaping route, but they did not know it is important to cover their mouth and nose to prevent from choked. Two learners had also chosen the correct route, but got serious choked for neither calling for help first nor covering their mouth and nose. There was one learner choosing the wrong escaping route without considering the environment condition. Two learners ended game by taking the elevator, which is a totally mistake when in the fire scene.

Table 3. The number of learners achieved in different endings.

If we observe the number of learners achieved each task of the game (as described in Figure 21), more detailed information can be discovered. We can find out that 20 learners did not cover their mouth and nose in the fire scene, and there are still 10 students using the bubble extinguisher (wrong type), without noticing it is an electrical fire. But it also points out that most students have called 119 for help and know it is more secure to touch the door before opening it.

37

Figure 21. The number of learners achieved in each task.

6.4.2 Three levels of information

From concept status, the teacher can get general information about what concepts these learners have. Nevertheless, if we want to know the exact actions learners done in the game, we have to do the behavior pattern mining and provide the teacher with three levels of information.

Common behavior patterns

The sequential pattern mining is applied to all learners first, and we can notice from the frequent sequence patterns (as shown in Table 4) that the “GotoScene” actions are the most frequent. This might due to our game style: the

player has to constantly changing from one scene to another when moving around. Consequently, we can regard these “GotoScene” actions as common behavior

Table 4. The common behavior patterns of the “escaping the burning building” game.

Frequent behavior patterns

Since we have realized that the “GotoScene” actions are common behavior patterns, we can filter all the “GotoScene” actions from the sequential pattern

mining results and obtain the frequent behavior patterns in Table 5.

Table 5. The frequent behavior patterns of the “escaping the burning building” game.

From inspecting the frequent behavior patterns, we can detect that most students were not aware of the fire category and took the wrong type of extinguisher. More than 86% of students have taken the towel and have also moistened it. Nevertheless, “use the wet towel” does not show up as frequent

patterns. This might be understood as there are still some learners not knowing the concept of covering mouth and nose right away in the fire scene.

39

Small group behavior patterns

We cluster the 35 students into 3 clusters through K-means. The cluster1 involves 12 learners while the cluster2 contains 21 learners, and the cluster3 consists of 2 learners.

From the action sequence of each cluster centers, we can get a general picture of the difference of these three clusters. From the viewpoint of game endings, we can discover that learners in the cluster1 and cluster3 tend to choose the correct escaping route while learners in cluster2 tend to fall out for jumping out of the building. Learners in cluster1 and cluster3 are likely to do more actions than learners in cluster2. Although cluster1 and cluster3 both reach the penthouse, learners in cluster3 did more correct actions such as “use wet towel to cover mouth and nose” and “feel the door before open it”.

The small group behavior patterns of cluster1 are listed in Table 6. The red-colored patterns are the frequent behavior patterns. From the patterns, we can tell that students in this cluster tend to do more actions than most other learners. They seemed to know the fire should be put out by the powder extinguisher, but they still took the bubble extinguisher. And the pattern “use the bubble extinguisher to put out fire” also shown in the frequent sequence patterns. This

can be explained as learners in cluster1 did not really know which extinguisher they should use, and put out fire simply by trial and error.

Table 6. The small group behavior patterns of the “escaping the burning building” game.

6.5 Discussions

From the conceptual level information, we can get the general picture of all learners. We surprisingly discover that most learners have the concept of touching the door before open it when in the fire scene. Although most learners can put out the fire with the correct type of extinguisher, most learners still have taken the bubble extinguisher without considering it is the electrical fire. Nevertheless, we notice that fewer learners than we expect have the concepts of calling 119 for help immediately. And from the endings of the played game, we find that 66% of learners have jumped out of the building, which is really a big mistake when in the fire scene.

And when we have insights into the behavior details of learners, first we notice that the moving actions are frequently shown in the results. This might due to our game design, and thus these patterns can be filtered out when analyze. For most learners, we observe that both “use the bubble extinguisher at the fire” and

41

“use the powder extinguisher at the fire” are discovered. This can be explained as

most learners put out the fire simply by trial and error, without the proper concepts. Another big finding is that most learners are aware of taking the wet towel, but “using the wet towel” pattern does not show up in the final frequent

behavior sequence. We can infer that most learners sometimes have taken all the stuff they need, but they don’t know what these for or when to use them.

Clustering method is applied for finding more information about each small group. The result shows that there are some learners who did few actions, and these learners form a cluster. From the mean center of the cluster, we find that these learners just moved around in the game and tended to jump out of the building. Teachers can apply more correct concepts about fire emergency to these learners.

Chapter 7 Conclusion

Adventure games are suitable learning contents for disaster prevention education. However, making games are big burdens for teachers because of the complex cause-and-effect relations in adventure games. Another issue is that current games lack for the critical essence of online learning - assessment. Therefore, we propose an Adventure Game-Based Formative Assessment framework to help teachers construct assessable adventure games.

Flow logics are the kernel of adventure games, and the flow is the process of decision making. Hence, we provide an Adventure Game Script approach, an online authoring script method to help teachers combine tasks in the game with different learning objectives and tailor these tasks into complex game flows simply through filling out the pre-defined Adventure Game Script. After filling out our adventure game script, our system will transform the obtained information into the real game by our game transformation process. This is really time-saving and easy-to-edit for teachers to make their own educational games.

Another issue is that teachers want to do formative assessment. Games are suitable for doing formative assessment since learners are immersed in the simulated environment to make decisions or attempts, and their behaviors or decisions reflect their knowledge in mind. However, even when teachers have the educational games, they are still hard to evaluate learners. For this, we propose a method called Game Behavior Analysis. Task sequences and organized logs are

43

provided as the analytical raw data. Learners’ task sequences will be mapped to

their concept status right after they finish playing the game. The organized logs with sufficient information can be used to do more analysis such as discovering behavior patterns in the game through behavior mining. These analytical results will feed back to teachers and help them to assess the learning effects.

The experiment results have verified our proposed framework. The game “escaping the burning building” can really be realized by our game transformer

after filling out the script. And the game is played by 35 seventh-grade students of Guang-Wu junior high school in Hsin-Chu city. The concept status report can really provide a general picture of learners’ misconceptions. The organized logs

are applied to behavior mining for behavior patterns in the game, and we provide teachers with three levels of information. Common behavior patterns are the patterns due to game scenario settings, while frequent behavior patterns are the patterns that reflect most learners’ behaviors. And small group behavior patterns

are unique patterns for each small group of learners.

[2, 3, 6, 7, 11-18]

Reference

1. Neisser, U., Self Knowledge and Psychological Knowledge: Teaching

Psychology From The Cognitive Point of View. American Psychologist

Association, 1975.

2. Howland, K., Good, J., and Robertson, J., A Learner-Centered Design

Approach to Developing a Visual Language for Interactive Storytelling, in ACM IDC 2007 Proceedings: Games.

3. Goolnik S., Roberson, J., and Good, J., Learner Centered Design in the

Adventure Author Project. International Journal of Artificial Intelligence

in Education. 2006.

4. Riedl, M.O., and Young, R.M., From Linear Story Generation to

Branching Story Graphs. IEEE Computer Graphics and Applications

Special Issue on Interactive Narrative, 2006.

5. Moreno-Ger, P., Martinez-Ortiz, I., and Fernández-Manjón, B., The

<e-Game> project: Facilitating the Development of Educational Adventure Games, in Cognition and Exploratory Learning in the Digital age. 2005.

6. Wu, D.C., Lin, H.Y., Tseng, S.S., Weng, J.F., and Su, J.M., Constructing a

Role Playing Interactive Learning Content Model, in Ninth IEEE International Symposium on Multimedia 2007 - Workshops. 2007.

7. Chen, C.M., Chen, M.C., and Li, Y.L., Mining Key Formative Assessment

Rules based on Learner Profiles for Web-based Learning Systems, in Seventh IEEE International Conference on Advanced Learning Technologies. 2007.

8. Shih, J.L., and Su, H.Y., Analyzing Children's Cognitive Activities in

Digital Problem-solving Learning Games “William Adventure”: An In-depth Case Study, in The 16th International Conference on Computers in Education 2008.

9. Martínez-Ortiz, I., Moreno-Ger, P., Sierra, J.L., and Fernández-Manjón, B.,

Production and Deployment of Educational Videogames as Assessable Learning Objects. Springer-Verlag Berlin Heidelberg 2006.

10. Cho, L.H., The Design of Collaborative Role-Playing Learning Scheme. 2008, National Chiao Tung University.

11. RPG Maker. [cited; Available from: http://www.enterbrain.co.jp/tkool/RPG_XP/eng/.

12. Wu, D.C., Constructing an RPG-like Interactive Learning Content Model. 2007, National Chiao Tung University.

13. Desurvire, H., and Wiberg, C., Evaluating Real Time Strategy Game

Player Experiences and Playing Styles, in Evaluating User Experiences in Games. 2008.

45

14. Tan, J., and Biswas, G., Simulation-Based Game Learning Environments:

Building and Sustaining a Fish Tank, in The First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning. 2007.

15. Lee, M.W., Chen, S.Y., and Liu, X., Mining Learners' Behavior in

Accessing Web-Based Interface. Technologies for E-Learning and Digital

Entertainment, 2007: p. 336-346.

16. Magerko, B., Story Representation and Interactive Drama, in 1st Artificial

Intelligence and Interactive Digital Entertainment Conference. 2005.

17. Chuang, T.Y., and Chen, W.F., Effect of Computer-Based Video Games on

Children: An Experimental Study, in The First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning. 2007.

18. Cheong, Y.G., Kim, Y.J., Min, W.H., Shim, E.S. and Kim, J.Y., PRISM: A