www.elsevier.com/locate/ijedudev
Developing a networked VRML learning system for health
science education in Taiwan
Chien Chou
a,*, Chin-Chung Tsai
b, Hsiu-Fei Tsai
aaInstitute of Communication Studies, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu, 30010, Taiwan, ROC bCenter for Teacher Education, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu, 30010, Taiwan, ROC
Abstract
This study discusses applying virtual reality (VR) and Virtual Reality Modeling Language (VRML) to promote health science education in Taiwan. It first describes the needs of health science education in Taiwan, and the advantages of using computer technology in health science teaching and learning. A networked desktop VR-based system and course-ware entitled “Travelling with Our Food” were developed for health science learning. The design of the course, the development of the system (platform and software), and expert-based and user-based evaluations are reported. Evalu-ation results, research issues, and possible future work are also discussed. 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Virtual reality; Web-based learning; Health science education curriculum
1. Introduction
Knowledge of health science is quite important throughout our lives. However, formal health science education in Taiwan is only provided to third (9-year-old) to seventh (13-year-old) graders, and to tenth graders (16-year-olds). After that, stu-dents basically receive health information on their own, for example from mass media (books, news-paper, magazines, television, etc.).
The need for formal as well as informal health science teaching and learning has become more and more important over the past 10 years, par-tially because of Taiwan’s economic development (Jiang, 1997). People realize that they want not
* Corresponding author.
E-mail address: cchou@cc.nctu.edu.tw (C. Chou).
0738-0593/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 7 3 8 - 0 5 9 3 ( 0 0 ) 0 0 0 0 3 - 1
only to survive, but also to have well and happy lives. Hence, besides the health information pro-vided in traditional text books and mass media, the use of computer technology may provide an alter-native instructional mode for health science teach-ing and learnteach-ing. For example, students can select health science information based on their needs and interests. In addition, using the multimedia capacity of computer technology, textual infor-mation can be supplemented with 2-D or 3-D graphics, video, and animation. Students are thus not only more motivated to learn on their own, but also they are better able to visualize health science concepts. This study described an innovative attempt to develop a networked VRML learning system about the human digest system for Taiwan college students.
2. Health science education and health education
The major purpose of this study is to help stu-dents acquire health science knowledge about the digestive system. That is, scientific information or factual knowledge about the digestive system is the main instructional content. Hence, the term “health science education” is used in this study, instead of “health education” since the purpose of health education generally is to present practical knowl-edge about health care to prevent one from con-tracting diseases. We believe that for mature stu-dents, health science knowledge could help them obtain or further understand relevant information of health education. For example, a scientific understanding of the digestive system may reveal some guidance about how to take care of the diges-tive system. Although there may be some linkages between health science knowledge and health behaviors, we do not assume that the acquisition of health science knowledge or even health education knowledge will necessarily affect students’ health behaviors.
3. Virtual reality for education
Recent developments in virtual reality (VR) have made it a more advanced computer tech-nology, and attracted increasing amounts of atten-tion among researchers in many educaatten-tional disci-plines. Chou et al. (1997) defined VR as a technological, functional, experiential, and psycho-logical concept. From the technopsycho-logical perspec-tive, VR is usually a collection of advanced com-puter technology components and equipment, such as high-speed computers, head-mounted displays (HMD), motion-sensing data gloves and so on. This view is also the most popular and well-known. From the functional perspective, VR can be seen as a tool to provide real-time interactivity between computer and users. Through the dynamic display of computer graphics, users can interact with virtual objects and information in the VR environment. From the experiential and psycho-logical perspectives, VR is a tool that allows users to experience tele-presence. Tele-presence refers to
“the extent to which one feels present in a com-puter-mediated environment, as well as in the immediate physical environment” (Chou et al., 1997; p. 224). According to these perspectives, we can define VR as a concept or tool consisting of computer-technology-based artificial worlds filled with computer-generated images that respond to users’ movements and inputs, and which allows users to experience a mediated sense of presence. Johnson et al. (1998) proposed that virtual-reality environments for education typically fall into three categories: networked text-based virtual environments, desktop virtual reality, and immers-ive virtual environments. Networked text-based virtual environments are commonly known as MUDs and MOOs (Bolter, 1997), and support real-time interactive use among users in distributed and remote locations. Users basically interact with each other through typing words into their personal computers. Desktop virtual reality provides 3D multimedia simulations users can enter and explore. Users share the same virtual worlds through networking from remotely located per-sonal computers (Jern, 1997). Immersive virtual environments employ high-end laboratory equip-ment, such as workstation computers, head-mounted display, and data gloves (Burdea and Coiffet, 1994), and users are required to go to lab-oratories and wear the equipment to interact with the virtual worlds. Although these environments provide the greatest degree of immersion, and thus may engage students’ attention and enhance learn-ing, the cost—of hardware as well as software— limits their applications and popularity. A compari-son of the three categories of virtual reality, in terms of their medium forms, playback platforms, platform/development costs, number of parti-cipants allowed, and immersive effects, is listed in Table 1.
Thus, VR is a learning tool that helps make abstractions more concrete (Byrne, 1993), and so helps students internalize their learning (Regian et al., 1992). In the study of Chou et al. (1997), struc-tural analysis curriculum was presented in a three-floor hexagonal learning center. By using HMD and a 3D mouse, students could “touch” and “grasp” virtual objects in the learning center, deter-mine the structural stability of various structures,
Table 1
Comparison of the three categories of virtual reality Categories of virtual reality
Text-based virtual world Desktop virtual reality Immersive virtual environment Medium forms Text only Dynamic 3D graphics, sound, Dynamic 3D graphics, sound
video video
Playback platform Personal computer with Personal computer with High-end computer, e.g. SGI network ability network ability ONYX/RE2, with extra
equipment HMD, data glove, 3D mouse, headphones, etc.
Playback platform cost Low Low to medium Extremely high
Content development cost Low Medium High
Immersive effects Low Low to medium High
Users participated Multiple users, sometimes a Multiple users, sometimes a Usually single users, sometimes large number at the same time large number at the same time more than one depending on
the system design
visualize the internal forces on diagrams, and observe structural deformations.
VR provides multiple or alternative represen-tations of the real world. Participants and virtual objects are not constrained by physical realities or practicalities. Loftin et al. (1993) developed a vir-tual physics laboratory to help college students understand Newtonian and quantum physics. Stu-dents in the lab were allowed to freely change the physical positions, speeds, forces and displace-ments of virtual objects.
VR also makes it possible to represent unreach-able “lost worlds”. In a study by Sanders and Gay (1996), an ancient Greek farmhouse was re-built on the Web, allowing students to observe it in three dimensions. Using hyperlinks and a database incor-porated into the Web site, students used their own desktop computers not only to navigate through the building, but also to connect to relevant infor-mation about past civilizations.
In a study by Johnson et al. (1998), a virtual fantasy island was built for 6- to 10-year-old stu-dents to construct and cultivate simple virtual eco-systems. Students could observe a dormant volcano on the island, help tend gardens, put sunglasses on the sun to block sunshine, pull clouds closer to increase the rainfall, and even speed up the ecosys-tem’s natural processes.
4. The use of virtual reality and constructivism
The use of virtual reality for education is also consistent with the merits of constructivist theory. Constructivism is a relatively new educational paradigm that is receiving considerable special attention among science educators (Staver, 1998). It asserts that learners should be viewed as cogni-tive subjects engaged in the process of accogni-tive knowledge construction, and that such a process is person and relevant to subjects’ existing knowl-edge structures (von Glasersfeld, 1993). Conse-quently, constructivist-oriented learning environ-ments emphasize students’ prior knowledge and focus on challenging existing misconceptions they may have that are at odds with accepted scientific views. Prior research has shown that students at every grade level have various misconceptions concerning biology and health science (Wandersee et al., 1994). Constructivism also highlights student autonomy and encourages teachers to conduct learner-centered instructional activities (Tsai, 1998). The use of VR for education, clearly, could further the aims of constructivist-oriented learning environments. First, VR provides relatively highly authentic representations of the instructional con-cepts (compared to other instructional media such
as textbook pictures) that may easily challenge stu-dents’ existing misconceptions and then promote students’ “conceptual changes” (Dole and Sinatra, 1998). Moreover, student autonomy is assured because students must actively interact with VR materials to maintain the learning process. Finally, students can freely travel in VR environments and acquire information of interest to them, which cre-ates a student-centered learning environment.
5. VR courseware design and development
The health science virtual reality system presented in this study belongs to the desktop vir-tual reality category. It allows students to use per-sonal computers to explore simulated environments containing pre-designed objects and information. We employed desktop and Web-based virtual-reality environment because distributing infor-mation over the Internet allows more users to access it at the same time. Students can use their own computers to access the health information at various locations and times. From the content developer’s viewpoint, the creation, revision and expansion of Web-based virtual objects and infor-mation costs less and is easier to work with than equivalent immersive virtual worlds. Although this kind of VR application is usually limited in size and complexity, and lacks high-quality immersive “realism”, the following situations in Taiwan may make its educational potentials promising and easy to realize.
1. In recent years, the government (e.g. the Minis-try of Education and the National Science Coun-cil in Taiwan) eagerly promotes computer edu-cation and network-based learning not only at the college level, but also across K-12 schools. 2. Taiwan is famous for her computer industry. The computers and network facility are widely available and accessible to almost all schools and individuals.
3. The VRML system is constructed on the Inter-net so that anyone who has Inter-network facility, Chinese operating system, and a VRML browser (i.e. Cosmo Player 2.0) could use this system.
In short, the aim of this project was to create an integrated and interactive network of linked virtual worlds that can be used as supplements for formal as well as informal health education curriculum.
The courseware content of this study was infor-mation and 3-D aniinfor-mation about health science, in particular, the human digestive system. Some sys-tematic procedures for designing instruction (Dick and Carey, 1990) were adapted to construction of the courseware “Travelling with Our Food”. The target learners were identified as non-health-science-major college students, and the secondary target learners were high school students. The course content hierarchy included Digestive Pro-cesses, General Organization, Mouth or Oral Cav-ity, Pharynx, Esophagus, Stomach, Pancreas, Liver, Small Intestine, and Large Intestine, as shown in Fig. 1. The users followed the food—an apple to be digested by the system—as it traveled from the mouth to the rectum.
The platform used in this study to develop the pilot system and courseware included a Pentium II-266 personal computer (128 MB RAM, 6.4 GB hard disk, Windows NT) for the server side. The client side, i.e. the students running the courseware needed at least Pentium 133s personal computers with at least 32 MB of RAM, and Netscape Com-municator 4.0 or Internet Explorer (IE) 4.0 brow-sers. The software used to develop the courseware included 3D Studio Max, the commercial package Cosmo World 2.0 for writing VR Modeling Langu-age (VRML), and Hypertext Markup LanguLangu-age (HTML) in organizing the graphics, text, and audio on the World Wide Web. Table 2 lists the hardware and software used to develop and play back the VR learning course.
Each topic of the course presentation (see Fig. 1) was divided into two parts on the screen. Fig. 2 shows a sample screen—the Small Intestine— from the course. On the left part of the screen, a 3D graphic written in VRML allowed students to not only view from any direction but also to enter the small intestine by using navigation tools on the menu bar at the bottom of the screen. On the right part of the screen, text and/or 2D graphics were presented for detailed health science information. Students could click on underlined and colored hot
Fig. 1. The course content hierarchy.
Table 2
Hardware and software used to develop and play back the VR learning environment
Computer platforms
Pentium II-266 personal computer Development software
3D Studio Max Cosmo World 2.0
Virtual Reality Modeling Language (VRML) Hypertext Markup Language (HTML) Display platform
Pentium 133 or up
Netscape Communicator 4.0 or IE 4.0 browser Cosmo Player 2.0 (to present 3D VRML graphics)
Fig. 2. A sample screen—small intestine—from the pilot health science system, including a VRML window for 3D graphics (on the left), and textual and 2D graphics (on the right).
keys to read more detailed information presented in movable pop-up boxes, as illustrated in Fig. 3.
6. VR courseware evaluation
A formative evaluation was conducted to exam-ine the usability and instructional effectiveness of the system and the courseware. Flagg (1990) stated that formative evaluation is one of the most critical steps in the development of learning materials. The main purpose of the formative evaluation is to help designers of this system during its early develop-ment stages improve its quality and cost-effective-ness and thus increase the likelihood that the final system will achieve its goal and objectives.
Fig. 3. Advanced information is presented in a movable pop-up box such as the one in the center of the screen. Two major evaluation approaches were adopted:
expert-based and user-based (Sweeney et al., 1994). The expert-based evaluation has the evalua-tors using the system to determine whether the sys-tem matches predefined or generally-accepted design criteria. The user-based evaluation involves one or more users completing one or more tasks in an appropriate environment. Reiser and Kegel-mann (1994) suggested that in most formative evaluation cases, education professionals, subject matter experts, media specialist and so on are rec-ommended as the individuals invited as experts to evaluate the computer-assisted instructional materials.
Therefore, five experienced experts were involved in the first evaluation, including one com-puter-assisted training specialist, one human-inter-face designer, one medical specialist, one 3D graphics artist, and one VRML application devel-oper. The method used in expert-evaluation was thinkaloud or talkaloud method, obtaining infor-mation from experts about their moment-to-moment opinions and comments of the system while they were using it. The major advantage of
this method is simplicity, in terms of the small amount of expertise required (Dix et al., 1993). Sweeney et al. (1994) stated that this method might be the only source of data on the cognitive process involved in using the system.
During the one-to-one evaluation, experts were asked to navigate through the courseware and speak aloud their opinions, feeling, concerns, con-fusions, suggestions and so on. For example, the graphics artist suggested that the 3D graphics pres-entation should find the optimum balance between “realism” and “good looking”. He thought the 3D graphics in this system seemed too “slick and clean”. The computer-assisted training specialist suggested that detailed and comprehensive instruc-tions should be included in the first Web page, especially for those who had no experience of VRML 3D graphics. In-depth interviews were also conducted after they finished their evaluations, dur-ing which they were encouraged to discuss specific suggestions for revision. For example, the human-interface evaluator suggested a guided tour for those who were not familiar with the VRML navi-gation. The learning system and materials were
then revised according to the evaluation results and suggestions. Fig. 4 shows the approach, goal, method, and participant in the formative evaluation for this study.
Nichols (1997) claimed that formative evalu-ation is basically performed in a local setting where students and developers can be brought together to complete their evaluatory tasks. In this study, 12 target college students from National Chiao Tung University in northern Taiwan participated in the user-based formative evaluation. Voluntary stu-dents were recruited from networks by posting on the NCTU campus electronic bulletin board system (BBS). Since the health science courseware would be presented on computer networks in the future, it was felt it was more appropriate to recruit campus network users as our samples. Sweeney et al. (1994) suggested that objective performance data might be collected by asking users to complete
Fig. 4. The approach, goal, method, and participant in the formative evaluation employed in this study.
tests, as well as subjective opinions on the system by surveying users. Reiser and Kegelmann (1994) also stated that the evaluation of a learning system is incomplete without a report on student learning performance. Therefore, a written comprehension test and a survey questionnaire were conducted after each student finished the journey. The test consisted of two question types: multiple choice and fill-in questions. The questions concerned the textual information, the 2D graphics, and the 3D VRML graphics provided in the course. Fig. 5 shows one of the fill-in questions.
The test sheet was presented in Microsoft Word; therefore, colorful pictures from the course were included as stimuli, as shown in Fig. 5. Students were impressed and responded positively to this computerized multimedia test. The test scores indi-cated that students acquired basic knowledge of the digestive system from the course, especially from
Fig. 5. One example of fill-in questions, with correct answers, number and percentage of students who answered correctly.
some 2D graphics and 3D VRML graphics. For example, 10 of the 12 students correctly answered questions on organs’ relative locations (e.g. Where are pancreatic juice and gallbladder juice injected into the small intestine? (1) after the ileum, (2) before the duodenum, (3) after the jejunum, (4) after the cecum). However, about one-third of stu-dents did not correctly answer questions based on factual knowledge (e.g. What is the average length of the human small intestine? (1) 2–3 m, (2) 4–5 m, (3) 6–7 m, (4) 8–9 m). These results also imply that students learned more spatial knowledge from the VRML instructional materials than from those provided in HTML, which mainly presented fac-tual knowledge.
A questionnaire designed to survey students’ attitudes towards the course was conducted upon completion of each student’s tryout. More than two-thirds of the students responded that their learning motivation was enhanced by this system, and that they were willing to visit the web site again. One student wrote:
I never dreamt of travelling inside our body. I thought it must be a bloody and disgusting experience...Here I think it is more like a com-puter game, an interesting educational game... Students considered that the major advantage of the VRML course was that the presentation was attractive, interactive and informative. The major disadvantages were its slow speed and the ease of getting lost in navigation. One student wrote:
The animation is very beautiful to attract my attention. And the whole course is very edu-cational and rich in information (maybe too rich for me to finish reading)...I am so eager to navi-gate through every part of the digestive system. However, the computer was too slow to respond to my commands, especially when I got lost in the longest small intestine...
When asked how to solve the problem, two stu-dents suggested a “guided tour” option for those
who did not want to move independently, and one student added a “speed change” option so that they could decide to move quickly or slowly.
Students were asked whether they experienced the immersive effect of VRML navigation. Nine of the 12 students answered that they had not. They reported that they also paid attention to other tex-tual and graphic information presented in the course, and they were fully aware of the outside environment, for example, other people moving around in the laboratory.
When the VRML presentation (3D dynamic graphics) and the HTML information (text and 2D graphics) were compared, seven students liked the VRML presentation more because it was interest-ing and interactive; two liked the HTML presen-tation more because it was richer and more read-able; three students expressed no preference.
Students were asked to suggest educational applications in which VRML could be employed. A variety of answers were collected, including: navigation within computer hardware, archaeolog-ical fieldwork, nuclear power operation, virtual museums, aquariums, exploring how birds live, playing with Chinese dragon, and so on. One stud-ent’s statement seemed to summarize their ideas:
...anything that you cannot easily touch, grasp, or observe in your daily lives, or something that never exists in the real world. To me, VR frees my imagination and physical constraints, taking me to anywhere in any form at any time.
7. Suggestions and conclusions
Based on the experience of developing the VRML health science learning system, and the results of its formative evaluation, this study yielded the following conclusions and suggestions for both the system design and course develop-ment.
1. 3D graphics presented in VRML more mot-ivated students to learn the content than the text-only information. Students reported that they were highly interested in navigating around and
inside the digestive organs. On the other hand, the text provided rich and advanced information that supplemented the graphics presentation. 2. Students reported that the system and the course
provided high interactivity by which the system exactly responded to their inputs and com-mands. The interactivity engaged students in the course presentation and made them become more active in the entire learning journey. That is, the journey provided a constructivist-oriented learning environment that promoted students’ active knowledge construction.
3. After analyzing students’ background infor-mation and researchers’ observations on site, it was found that the difference in users’ prior 3D graphics or game-playing experiences affected their attitudes and learning performance. It was observed that students with more prior 3D graphics or game-playing experiences mastered the control and navigational tools more quickly, and thus reduced their time on the task. 4. Based on the experiences of course development
and evaluation in this study, it is concluded that virtual reality is a feasible alternative instruc-tional tool for teaching concepts, especially con-cepts for which visualization is required. This finding is also supported by Sanders and Gay (1996), Chou et al. (1997) and Johnson et al. (1998). It is suggested that virtual reality can be used to introduce a variety of learning topics that are not easily observed in daily life or the physical world.
5. It is further suggested that full advantage of 3D displays must be taken, as well as the flexible-scale capacity of VR should be used to demon-strate learning objects. The inside-organ journey in this study is a good example, and was much appreciated by the students.
Based on the prototype VR courseware for health education presented in this study, the fol-lowing four questions are proposed for future research.
1. The response speed needs special consideration. The possible solutions are (1) using high-speed computers when playing back the course; and/or (2) using complicated shapes and patterns
spar-ingly in the 3D graphics. The former solution can be easily applied because Taiwan is famous for computer products and the cost of computer products keeps dropping. If computer perform-ance, in terms of speed, can be improved, the use of complicated shapes, patterns and colors can be less constrained.
2. In this study, students reported that they did not feel the “immersion” as claimed by most VR developers. As stated above, highly immersive VR usually requires high-end equipment such as head-mounted displays, headphones, and motion-sensing data gloves. Networked desktop virtual reality applications such as the one developed in this study, on the other hand, save on equipment cost but do not provide highly immersive VR environments. The question as to whether the immersion effect is a critical factor for VR learning warrants more research. 3. How to balance realism and aesthetics of the
graphical presentation caused ongoing argument throughout the entire design and development process. When developing the organs’ appear-ances, designers had different ideas about the colors and textures of the images. Some design-ers argued that realism is the first priority for a learning system while others argued that aes-thetics should be considered in order not to scare learners away. How to balance the realism and aesthetics of graphical presentations is itself an issue for research.
4. How can getting lost while navigating the VR 3D graphics be avoided? Students reported that sometimes they did not know their locations, where they should go, and how to get there. A “guided tour” seemed a possible solution to minimizing this problem, as requested by one student. Hence, a guided tour was preset by the course developer by using a series of lines through the digestive organs for students to fol-low. An experiment to assess the effectiveness of this guided tour is in progress and warrants more research.
In this study, a Web-based learning course— Travelling with Our Food—was developed to examine the feasibility of applying VR to health information learning. It was found that VR 3D
graphics presentations attract students’ attention, allowing them to walk through the graphics, and thus motivating them to become more engaged in the learning materials. Thus, combined with text and graphics in traditional computer-assist learning (CAL), Web-based VR’s potential brings us a new tool to deliver learning materials and improve research, not only on the technologies themselves, but also by using VR as a science instruction and learning strategy. This course can also be promoted to users other than college students.
The further development of this VR system will integrate some health care information about the digestive system into its courseware. For example, information about some common diseases or can-cers of the digestive system, how to take care of the digestive system, and healthy diet habits will be added to appropriate parts of the courseware. By this way, the linkages between health science knowledge and health education will become more explicit. It is expected that the coverage of this courseware will be extended and it could then enhance students’ health science learning, assist health education, and possibly affect their health behaviors.
Acknowledgements
An early version of this article was presented at the Annual Conference of the National Association for Research in Science Teaching (NARST) in Boston, USA on March 30, 1999, and was nomi-nated for the 2000 NARST Outstanding Paper Award. This project was supported by the National Science Council in Taiwan, Project Number NSC87-2511-S-009-007-ICL.
References
Bolter, J.D., 1997. Virtual reality and the redefinition of self. In: Strate, L., Jocobson, R., Gibson, S.B. (Eds.), Communi-cation and Cyberspace: Social Interaction in an Electronic Environment. Hampton Press, Cresskill, NJ, pp. 105–119. Burdea, G., Coiffet, P., 1994. Virtual Reality Technology. John
Wiley, New York.
Technology Laboratory Technical Report #R-93-6, Human Interface Technology Laboratory, Seattle, WA.
Chou, C., Hsu, H.-L., Yao, Y.-S., 1997. Construction of a vir-tual reality learning environment for teaching structural analysis. Computer Applications in Engineering Education 5 (4), 223–230.
Dick, W., Carey, L., 1990. The Systematic Design of Instruc-tion, third ed. Scott, Foresman and Company, Glenview, IL. Dix, A., Finlay, J., Abowd, G., Beale, R., 1993.
Human–Com-puter Interaction. Prentice-Hall, New York.
Dole, J.A., Sinatra, G.M., 1998. Reconceptualizing change in the cognitive construction of knowledge. Educational Psy-chologist 33, 109–128.
Flagg, B.N., 1990. Formative Evaluation of Educational Tech-nologies. Lawrence Erlbaum, Hillsdale, NJ.
Jern, M., 1997. Information visualization enables interaction with large datasets on the Web. In: Earshaw, R., Vince, J. (Eds.), The Internet in 3D—Information, Images and Inter-action. Academic Press, San Diego, CA, pp. 223–236. Jiang, Y.Y., 1997. The survey and development research of
col-lege students’ health message demands on World Wide Web—A case study of NCTU students. Unpublished master thesis of National Chiao Tung University, Hsinchu, Taiwan. Johnson, A., Roussos, M., Leigh, J., Vasilakis, C., Barnes, C., Moher, T., 1998. The NICE project: Learning together in a virtual world. On-line available at: [www.ice.eecs.uic.edu/ ‘nice/NICE/PAPERS/VRAIS/vrais98.2.html]
Loftin, B., Engelberg, M., Benedetti, R., 1993. Applying virtual realities in science education: a prototypical physics labora-tory. In: Proceedings of IEEE Symposium on Research Frontiers in Virtual Reality, pp. 67–74.
Nichols, G.W., 1997. Formative evaluation of Web-based instruction. In: Khan, B.H. (Ed.), Web-based Instruction. Educational Technology, Englewood Cliffs, NJ, pp. 369– 374.
Reiser, R.A., Kegelmann, H.W., 1994. Evaluating instructional software: A review and critique of current methods. Edu-cational Technology Research and Development 42 (3), 63–69.
Regian, W., Shebilske, L.W., Monk, J.M., 1992. Virtual reality: An instructional medium for visual-spatial task. Journal of Communication 42 (4), 136–149.
Sanders, D., Gay, E., 1996. VRML-based public education— An example and a vision. On-line available at: [www.crmlsite.com/dec96/sport2.html]
Staver, J.R., 1998. Constructivism: Sound theory of explicating the practice of science and science learning. Journal of Research in Science Teaching 35, 501–520.
Sweeney, M., Maguire, M., Shackel, B., 1994. Evaluating user– computer interaction: A framework. International Journal of Man–Machine Studies 38, 689–711.
Tsai, C.-C., 1998. Science learning and constructivism. Curricu-lum and Teaching 13, 31–52.
von Glasersfeld, E., 1993. Questions and answers about radical constructivism. In: Tobin, K. (Ed.), The Practice of Con-structivism in Science Education. AAAS, Washington, DC, pp. 23–38.
Wandersee, J.H., Mintzes, J.J., Novak, J.D., 1994. Research on alternative conceptions in science. In: Gabel, D.L. (Ed.), Handbook of Research on Science Teaching and Learning. Macmillan, New York, pp. 177–210.
