Who Performs Better When Learning with Business Simulation Games?
A Case Study of a College General Course in Taiwan
Yu-Hui Tao
1Kung Chin Hung
2 1Department of Information Management
National University of Kaohsiung
700 Kaohsiung University Road, Nan-Tzu District
Kaohsiung, Taiwan, R.O.C.
ytao@nuk.edu.tw
2Top-BOSS International Corp.
3F., No.19-13, Sanchong Rd., Nankang Distric
Taipei City , Taiwan, Taiwan, R.O.C.
kc@top-boss.com.tw
Abstract: - Previous studies on business simulation games (BSGs) have concluded that performance may not be the primary benefits of using BSG because of mixed results. This study aims to understand what kind of student profile may perform better in classroom learning with BSGs. A case study of 43 students in a general college course in Taiwan indicates that knowledge and skill do matter, and participation and tacit learning preference contribute well to performance. However, students who have an auditory learning preference and are highly motivated may not like BSG learning in the classroom. Also, more dimensions of variables are needed to increase the explaining power of performance scores in regression analyses.
Key-Words: - Business simulation game, Performance, Perception, Regression, Student profile
1 Introduction
Business simulation game (BSG) usage reached 97.5% of all AACSB member schools in the United States more than a decade ago [2]. In early studies, the relationship between BSG learning performance and perception presented mixed results. Thus, recent research studies on BSG usage has been shifted to non-performance perspectives such as attracting the attention of students and helping them focus in class. Prensky [9] claimed that games provide an indispensable motivational condition for students’ learning, and thus students may be more interested in choosing courses that incorporate BSGs in the instructional activities. Therefore, BSG has been promoted as a popular means of informal learning [5], as well as for use in formal learning [8, 10]. Based on these three reasons, the responses of students to classes using simulation games is a critical indicator of the widespread use of simulation games in higher education.
Extant BSG literature focuses on business major students. However, business operations and management is also considered general knowledge for non-business major students or working professionals interested in accumulating business knowledge and experience through learning by doing. Business operations and management knowledge for non-business major students is especially critical
nowadays when many universities wish to culture their students through extensive and in-depth general education in addition to thestudents’majorsubjects. From this perspective, an investigation to explore the relationship of student performance and student profile is desirable, and this is the objective of the current study. In the remaining sections of this article, background literature related to the research objective are presented before the research methods are introduced. These are followed by the data analyses, discussions, and conclusions.
2 Background Literature
2.1 Business simulation games in Taiwan
Taiwan has two major BSG providers, Top-Boss Corporation (http://www.top-boss.com.tw) and Pitotech Corporation (http://www.pitotech.com.tw). As revealed in Faria [2], most simulations used in UK universities are developed in-house while most games used in the US are published simulations. In Taiwan, Pitotech mainly sells imported simulations while Top-Boss develops many games in-house. Top-Boss offers self-developed products, such as Business Operations Simulation System (BOSS), Marketing Winners, Distribution Master, Beer
Games, and Retail Expert, as well as imported games such as Magnus, which was developed by Singapore National University. These games are as intuitive as their names; for example, BOSS is a general business operations game, Beer Games is a well-known MIT beer game (http://beergame.mit.edu/), and Retail Expert is a convenience store game. Pitotech, on the other hand, offers imported games such as Virtual Business Management, Virtual Business Retail, and Virtual Business Sport games, which have a smaller overall market share in Taiwan. Other related vendors, such as APEX International Corporation (http://www.apex.com.tw) which offers the Virtual Stock Exchange game, are not appropriate candidates for this study aiming at business management.
In recent years, Top-Boss Corporation has successfully promoted its products to higher education institutes through free trials, workshops, training camps, and sponsorships for national BSG competitions. These BSGs are team-based and competition-oriented; hence, BOSS or other games areincorporated in theschools’curriculum aspartof various group competition activities. These activities are usually repeated several times in a row and several days apart due to the round of game play nature. For the students, the required parameters in many Web-based forms need to be entered in each round, thus graphical reports or visual performance data are available for the users to increase users’ usability and learning interests during the games. For the teachers, they use these games throughout most of the semester. The rationale behind the different expertise required for business decision making in the games are used to illustrate the knowledge and applications of targeted course subjects in class [10].
2.2 Research on business simulation games
Kiili [6] proposed a problem-based gaming model that distinguishes the learning process into elements of strategy formation, active experimentation, game world observation, and reflection. During this process, single-loop learning is formed if the player goes from reflection to active experimentation without forming new strategies, or double-loop learning is formed if reflections are followed by forming new strategies.
Kraiger et al. [7] classified several broad categories of learning outcomes, including skill-based, cognitive, and affective outcomes. Skill-based learning outcomes address technical or motor skills; cognitive learning outcomes include three subcategories, namely, declarative knowledge, procedural knowledge, and strategic knowledge; affective learning outcomes refer to attitudes.
However, these three categories of outcomes do not have to be consistent. Anderson and Lawton [1] followed up their previous studies and confirmed the failure to find a relationship between simulation performance and students’ attitude toward the simulation. Consequently, they called for further effort in finding the missing link between performance and measurement items.
An interesting contrast is that Wellington and Faria [13] confirmed that good simulation performers are consistently good by examining high- versus low-rank performers in two consecutive rounds of simulation competition. Although no explanation was found for this research finding, it thus seems to indicate that GPA, beginning attitude toward the simulation exercise, and team cohesiveness are not significant differences.
Tao et al. [11] proposed an integrated model fusing information system (IS) theories and education theories because previous studies rarely focused on the technological perspective of BSGs. In general, the perceptions of the empirical students provideadequateevidenceforTaiwan’steachersto adopt or continue using BSGs.
3 Research Design
Data collection design is first described and then the variable selection is briefly justified. These are followed by the statistical methods used in this study.
3.1 Data collection design
Based on the suggestion of Tech and Murff [12], a general course for teaching three different complexity levels of BSGs, BOSS (total enterprise simulation), Retail Expert (single-player small game), and Beer Game (multiple-player small game) from TOP-BOSS, is offered in a general class to undergraduate students in a university in Taiwan. Each game is designed in reference to the problem-solving game model and is taught in sequence as follows: introducing the game, practicing by simulated game competition, writing a group thought-sharing report, and gaming competition [6].
The final grade calculation is divided into attendance, individual tests, group reports, and competitions, which can be used as dependent variables in this analysis stage. The objective is to focuson “learning by doing” forstudentswith or without a management background. Students learn most of the knowledge by playing the games and collecting needed information over the Internet.
Some tips can also be learned from winners who share their wining experience at each practice and formal competition, as well as from the thought-sharing reports summarizing necessary knowledge, tactics, and strategies for playing the games.
A self-reported questionnaire is distributed at the end of the class to collect the data as described in Section 3.2.
3.2 Variable selection
Students divided by gender, management major, previous experience of management courses, and previous experience of BSG are individually tested for their differences on performance scores. The justification is briefly described below.
Gender has been associated with learning performance and has been investigated in the BSG context in previous studies [14]. Since previous business experience plays an important role on BSG performance [14] and the sample students come from five different colleges (i.e., Management, Science, Engineering, Society and Humanities, and Law), business major, previous experience of management courses, and previous experience of BSG are used in thisstudy to reflectstudents’business experience.
BSG is an online simulation-based game usually played together by a group of students. Thus, it is natural to assume that the BSG performance of students will be influenced by whether or not the students like to play online games. Feinstein et al. [3] distinguished simulation from simulators. In their study, simulators are sometimes called an iconic model because of their visual, auditory, and kinesthetic representations of a real system for training purposes. Furthermore, factors such as visual, kinesthetic, structure, learning with others, tactile, motivation and persistence, belonging to Dunn and Dunn’s PEPS learning style model, were used by Hawk and Shah [4] when comparing learning style between undergraduate business students in American universities and international universities.
The class activities are designed to be team-work basis, which includes preparing the thought-sharing reports, playing two of the three game, and learning by the students themselves for management knowledge necessary to play the games. Thus, students’participation [15] in group discussion and information searching off class are assumed to contribute to the BSG performance. Wellington and Faria [13] compared students’ simulation
performance and found that the sample groups were significantly different in GP.
3.3 Statistical methods
Three different statistical methods are used in this study. First, descriptive analysis is used to profile the sample students; second, t-test is used to test the performance of two groups of students with different characteristics; third, linear regression is used to identify the selected variables contributing to different performance scores collected in Section 3.1.
4 Data Analysis
4.1 Sample profile and descriptive statistics
Thegeneralcourse“BusinessSimulation Gamesand Competition” was offered to all undergraduate students in a national university in Taiwan. Over 150 students pre-registered for this course, but only 51 were officially approved by the online registration system. Forty-three of the 51 students completed the questionnaire at the end of the class.
Table 1 lists the means and standard deviations (S.D.) of the performance scores and measurement items. A brief summary of the sample profile is sketched as follows: 67.4% of the students were male and only 25.6% of the students had experiences using BSGs in previous classes. Students majoring in Engineering, Management, Science, Law, and Humanities displayed a distribution of 37.2%, 30.2%, 18.6%, 9.3%, and 4.7%, respectively. In terms of average scores in the previous semester, the majority of students fall into the 80s (55.8%) and 70s (34.9%), while very few fall into the 50s, 60s, and 90s.
4.2 Data analysis
Four individual characteristics, namely, major, gender, previous experience of management courses, and previous experience of BSG, are tested against final score. A t-test comparison of students divided by these four characteristics is shown in Table 2. Although the means for the positive-answer (yes) group students are higher and have less standard deviations, only Management major and previous experience of BSG have mild impacts on student class scores at the p=0.1 level. In other words, students majoring in management or have BSG experiences may potentially perform better than their counterparts.
Item Mean/S.D. Range
Final score (100%) 82.74/8.19
Competition score (50%) 76.77/11.18
Sharing report score (15%) 82.17/11.80
Test score (20%) 85.97/3.29 Attendance score (15%) 95.60/9.68 0-100 Online game 6.37/1.94 Visual Learning 7.40/0.93 Auditory Learning 5.97/1.40 Kinesthetic learning 7.29/1.13 Wants structure 6.30/1.46 Tacit learning 7.05/1.17
Learning with others 6.60/1.29
High motivation 5.63/1.31
High persistence 5.67/1.24
Participating group discussion 6.37/1.00
Searching information after class 5.35/1.45
9-point Likert scale from 1 representing very disagree and 9 representing very agree
Table 2. T-test analysis between students groups for class score
Variables Mean/S.D. Nyes Mean/S.D. Nno t Sig.
Management major? 85.23/2.83 13 81.67/9.48 30 1.847 0.069
Female? 83.86/4.60 14 80.42/12.76 29 0.977 0.345
Management course before? 84.47/3.33 22 81.09/10.85 21 1.395 0.175
BSG before? 85.09/3.05 11 81.94/9.24 32 1.683 0.100
at 0.1 level
Table 3. Regress Analysis Results for Intention to Continuance Usage
Mean performance (Scores) Independent
Variable Final Competition Sharing Test Attendance
Visual learning -0.504*** -0.388*/-0.356* -0.355* -0.316*
Auditory learning - - 0.319*
Kinesthetic learning -
-Tacit learning 0.291 -/0.436* - 0.562***
Wants structure -/-0.469* -0.481**
Learning with others -
-High motivation -0.375 -/0.662* - -1.148*** High persistence - -/-0.169* - -0.745* Online game - - -0.341* Participating group discussion 0.440* 0.285/0.487** 0.524** Searching information after class 0.339*
GPA last semester 0.319
F 3.779* 3.844*/3.702** 4.327* 4.620** 4.849**
Adjusted R square 0.264 0.155/0.343 0.097 0.259 0.465
***P< 0.001 ** P< 0.01 * P< 0.05 P= 0.1
style of a student in the Taiwan context. Variables in learning style may belong to different dimensions. This study adopts the 12 variables presented in Table 3. Backward stepwise linear regress is used to identify significant variables that contribute to five different performance scores.
Several observations can be derived as follows: First, visual learning is the most significant variable contributing negatively to all but the test performance scores. The feelings of visual, auditory, and kinesthetic learning do not resemble online games or simulators because BSG is more of a text-based simulation than a simulator. Accordingly, students with high visual learning preference may find it negatively associated with BSG performance scores in most cases, except in the test score. It is surprising to see a negative influence on performance scores for highly motivated students, which may imply that they are not used to this new method of “learning by doing”through gamecompetition.
Second, in the aspect of positive influence, participating group discussion had a significant positive influence on student performance in three out of five performance scores. Tacit learning effectively reflects the nature of BSG as a problem-based gaming model that involves strategies. Both meet the original expectation of this research.
Third, test score is indeed very different from the other scores in that three variables—online game, searching information after class, and GPA of last semester—uniquely contribute to test score, but not to others. This implies that test scores may not be appropriately evaluated by the BSG competition activities, and vice versa. Teachers need to be very careful in matching the course objective with its evaluation method.
Fourth, only the adjusted R-square of the attendance score borders the high level while the rest are at a low or insignificant level. This implies the necessity for more hidden dimensions of variables to increase the explaining power of regression analysis on BSG performance score.
5 Conclusion
We have several findings on who may perform better with BSG learning activities based on the above analysis and discussion. First, certain knowledge (management major) and skill (previous BSG experience) do matter in terms of student performance. Second, participating group discussion and tacit learning are positive factors in influencing student performance, which matches the learning-loop and reflection conceptsin Kiili’s[5]
problem-based gaming model. Third, students with auditory learning preference and high motivation may not be used to the BSG type of learning-by-doing style. Finally, more hidden variables and dimensions of variables are yet to be identified to increase the explaining power of regression analysis for determining the contributing factors of BSG performance scores in future research.
References:
[1] Anderson, P. H. and Lawton, L. Simulation performance and its effectiveness as a PBL problem: A follow-up study, Developments in Business Simulation & Experiential Exercises, Vol. 34, 2007, pp. 43-49.
[2] Faria, A. J. and M. Schumacher. Business Simulation games: Current usage a ten year update, Developments in Business Simulation & Experiential Exercises, Vol. 11, 1996, pp. 220-225.
[3] Feinstein, A. H., Mann, S. and Corsun, D. L. Charting the experiential territory: Clarifying definitions and uses of computer simulation, games, and role play. Journal of Management Development, Vol. 21, No. 10, 2002, pp. 732-744.
[4] Hawk, F. F. and Shah, A. J. Using learning style instruments to enhance student learning, Decision Sciences Journal of Innovative Education, Vol. 5, No. 1, 2007, pp. 1-19. [5] Kapp, K. Gadgest, games and gizmos: Informal
learning at Nick.com.
http://karlkap.blogspot.com, 2006.
[6] Kiili, K. Foundation for problem-based gaming, British Journal of Educational Technology, Vol. 38, No. 3, 2007, pp. 394-404.
[7] Kraiger, K., Ford, J. K., and Salas, E. Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, Vol. 78, No. 2, pp. 311-328
[8] Lim, C. P. Spirit of the game: Empowering students as designers in schools. British Journal of Educational Technology, Vol. 39, No. 6, 2008, pp. 996-1003.
[9] Prensky, M. Digital game-based learning. Computer in Entertainment, Vol. 1, No. 1, 2003, pp. 21-21.
[10] Prensky, M. Students as designers and creators of educational computer games: Who else? British Journal of Educational Technology, Vol. 39, No. 6, 2008, pp. 1004-1019.
[11] Tao, Y.-H., Cheng, C.-J. and Sun, S.-Y., What influences college students to continue using business simulation games? The Taiwan experience, Computers & Education, Vol. 53, No. 3, 2009, pp. 929-939
[12] Tech, R. and Murff, E. R. T. Are the business simulations we play too complex, Developments in Business Simulation & Experiential Exercises, Vol. 35, 2008, pp. 205-211.
[13] Wellington,W. J. and Faria, A. J. An investigation of the relationship between simulation play, performance level and recency of play on exam scores, Developments in Business Simulation & Experiential Exercises, Vol. 18, 1991, pp. 111-114.
[14] Wellington, W. J. and Faria, A. J. Are good simulation performers consistently good, Developments in Business Simulation & ExperientExercises, Vol. 22, 1995, pp.5-11. [15] Whiteley, T. R. and Faria, A. J.. A study of the
relationship between student final exam performance and simulation game participation. Simulation & Gaming, Vol. 20, 1990, pp. 44-64.
