• 沒有找到結果。

Development of design criteria and evaluation scale for web-based learning platforms

N/A
N/A
Protected

Academic year: 2021

Share "Development of design criteria and evaluation scale for web-based learning platforms"

Copied!
19
0
0

加載中.... (立即查看全文)

全文

(1)Development of design criteria and evaluation scale for web-based learning platforms Cheng-Mei Hsua, Yu-Chu Yehb and Jen Yenc a. Graduate School of Design, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliu, Yunlin 640, Taiwan b. Institute of Teacher Education, Research Center for Mind, Brain & Learning, National Chengchi University, 64 Chih-nan Road, Section 2, Wenshan, Taipei 116, Taiwan. c. Department and Graduate School of Visual Communication Design, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliu, Yunlin 640, Taiwan. Abstract Standardized and objective design criteria for evaluating web-based learning platforms can effectively distinguish the quality of a platform and, therefore, contribute in improving web-based learning outcomes. This is a two-phase study, in which Delphi technique and heuristic evaluation were employed in the first phase to develop the evaluation criteria and scale of web-based learning platforms; in the second phase, questionnaire survey, real online evaluation, and experts' analyses were used to analyze the reliability and validity of the developed scale. Contributions of this study include (1) providing an example of developing evaluation criteria for web-based learning platforms via a standardized procedure; (2) developing a reliable and valid scale for evaluating web-based learning platforms; and (3) establishing a basis for guiding and evaluating the design of a web-based learning platform, as well as enhancing the quality and development of a web-based learning environment. Relevance to industry This study provides objective criteria for designing a web-based learning platform. It also provides an evaluation scale using a standardized development procedure. The results of this study could contribute to enhancing the quality of a web-based learning environment. Keywords: e-learning; Web-based learning platform; Design criteria; Evaluation criteria; Evaluation scale; Standardization. 1. Introduction.

(2) As the Internet continues to expand and develop, e-learning offers a completely new teaching mode. This new mode has become the new standard and norm in the circle ([Allen and Allen, 2002] and [Horton, 2000]). Therefore, the entire technological structure of digital learning requires a good learning platform to establish a complete digital learning environment (Roberts and McInnerney, 2006). Effective design criteria can help users to evaluate and improve the quality and development of the web environment ([Keith, 2003] and [Waterhouse and Rogers, 2004]); a standardized procedure contributes to the development of such criteria (Industrial Development Bureau, Ministry of Economic Affairs, 2004). Many studies have proposed the design criteria for web-based learning platforms ([Hsu and Cheng, 2005], [IST, 2003], [Ou, 2000] and [Sarapuu and Adojaan, 1998]). These criteria, however, were not developed based on a strong theoretical basis of learning or a holistic approach in which multiple criteria are considered; moreover, the development procedures of these criteria were not clearly described. This study, therefore, attempts to propose effective design criteria for web-based learning platforms and develop an evaluation scale for web-based learning platforms via an elaborative and standardized procedure.. 2. Principles of designing a web-based learning platform 2.1. Integrating learning and instructional theories A good web-based environment design should consider learning theories and methods (Chen and You, 2001). The theory of Constructivist Learning Environment emphasizes that the learning environment should provide related cases, information resources, cognitive tools, communication tools, and scaffolding which help students acquire an integrated set of cognitive skills ([Jonassen, 1999] and [Yazici et al., 2001]). Meanwhile, the meaningful learning theory builds on constructivist learning theory and stresses that learning is active, situated, goal approaching, and issue-centered ([Barab and Duffy, 2000], [Jonassen, 2002] and [Peck et al., 1999]). Furthermore, the theory of Computer Supported Collaborative Learning (CSCL) aims at providing both an authentic environment and rich resources to solve the problem of limited human working memory, which is critical for improving learning outcomes. It has also been suggested that integrating instructional theory for proof and practical application is the most important factor for successful web-learning management (Hong, 1999). Finally, anchored instruction emphasizes the importance of acquiring knowledge from a real scenario or discussion and encourages the sharing of real-world experience, which is in line with the goals of web-based learning. Therefore, theory-based design criteria and evaluation scales for web-based learning platforms should enhance meaningful learning, integration of cognitive skills, effective web-learning management, and sharing of real-world experience. These elements are, therefore, integrated into the criteria and scale developed in this study. 2.2. Considering multiple criteria.

(3) Many researchers ([Becta, 2005], [Bostrom et al., 1990], [Khan, 1997], [McGreal, 1998], [Rasmussen and Davidson-Shivers, 1998], [Swisher, 1994], [Wang, 2000], [Wu, 2000] and [Yu, 2002]) have identified that web-based learning construction and design should consider multiple criteria, including curricula, learning styles, interactive design, and multimedia application. For example, Tung (2003) declared that course content, student participation, learning interactivity, and technical support influence web-learning outcomes. In the same vein, Khan and Vega (1997) claimed that course objective(s), course interactivity, and content quality are the three most important evaluation criteria for web-course effectiveness. Moreover, it was pointed out that course content must be appropriate, organizational, and facilitative to material application in learning environment planning (Yue, 2003). Standardization is also an important criteria that must be considered. A standardized e-learning model can provide learners abundant integrated resources to communicate and share with others. For instance, the Sharable Content Object Reference Model (SCORM), an international and universal e-learning model, provides standardized materials for communication across learning platforms and, therefore, fosters reusable learning content in a sharable framework (Lin, 2004). Keith (2003) also pointed out that different distance learning systems should establish an exchange mechanism to use a common format for packaging, sharing, and browsing so that online learning resources could be reusable, accessible, durable, interoperable, adaptable, and affordable. To make e-learning materials usable in different countries, Zeng et al. (2003) created the Teaching Materials Markup Language (TMML), in which international course standard and local material features were employed. Moreover, the materials could be easily repackaged, exchanged, and reused. With regard to course design, Waterhouse and Rogers (2004) addressed nine categories of course policy to give the instructor and students a clear understanding of a smooth-running course of e-learning. The main categories include student perusal of course information via e-mails, student grade and privacy protection, e-mail response rules, instructor participation in student discussions, and accessible instructional software. Therefore, multiple criteria derived from a holistic perspective, especially the viewpoint of a standardized web-based learning model, must be considered when designing a web-based learning platform. Such criteria not only improve web-based learning quality and facilitate the sharing of instruction resources, but also enhance course usefulness and increase new technology acceptability. 2.3. Purposes of this study The main purposes of this study were (1) to propose objective design criteria for web-based learning platforms from a holistic perspective; and (2) to develop a reliable and valid scale for evaluating web-based learning platforms based on the developed design criteria via a standardized procedure. In.

(4) addition, via careful evolving process, this study attempted to set up an example on how to develop valuable design criteria, and an effective tool for evaluating web-based learning platforms.. 3. Method 3.1. Research design This is a two-phase study and both qualitative and quantitative methods were employed. Specifically, the first phase used the Delphi technique and the heuristic evaluation method to develop the design criteria and their corresponding indicators for web-based learning platforms. The second phase, based on the results of phase 1, employed a questionnaire survey, real online evaluation, and experts' analyses to develop an evaluation scale for web-based learning platforms as well as to examine its reliability and validity. The Delphi technique obtains forecasts from an independent expert panel over two or more rounds. Normally, an administrator provides an anonymous summary of the experts' forecasts and their reasons for them after each round. The process stops when experts' forecasts change slightly between rounds, and final round forecasts are combined by averaging ([Rowe and Wright, 1999] and [Rowe and Wright, 2001]). The Delphi technique keeps individual responses anonymous so that social influences are minimized ([Huang, 1987], [Huang, 2003] and [Wang, 1998]). On the other hand, in the heuristic evaluation method, experts individually examine, evaluate, and communicate their opinions, which are combined to form a design decision or complete the design evaluation (Nielsen, 1993). 3.2. Procedures and participants 3.2.1. Phase 1. The researchers first collected test items from relevant questionnaires to collect design criteria. Then, the researchers purposely sampled qualified experts from university websites in Taiwan to conduct the Delphi technique. Based on professions and academic publications, 10 information management, design, and education scholars were selected. All of them possessed Ph.D. degrees and had rich experiences in instructional website construction or Internet instruction. Sufficient information about the study was provided when inquiring on willingness to participate, and a gift was given when the survey was finished. Meanwhile, the revised data were communicated via anonymous mails and, finally, these data were integrated to represent experts' viewpoints. The Delphi technique round lasted for 1 month and it took 5 months to compile the data into initial design indicators. Then, this study used the heuristic evaluation method to refine the initial design indicators with regard to their appropriateness and distribution. Two professors specialized in developing distance.

(5) instruction and two senior practical engineers specialized in designing web-based learning platforms were purposely sampled and each of them was paid US$ 50. Finally, the results were developed into the Evaluation Criteria for Web-based Learning Platform (EC-WLP). The EC-WLP was a five-point Likert scale with ratings ranging from “not very important” to “very important”. 3.2.2. Phase 2. In phase 2, five college students were asked to provide suggestions to revise the EC-WLP. Then, a questionnaire survey was conducted to examine the internal-consistency reliability and construct validity of the EC-WLP via Cronbach's α analysis and factor analysis. One hundred and fifty-eight college students were randomly selected from two universities in the middle and south of Taiwan. After the reliability and validity were confirmed, a real online evaluation of three learning websites was conducted via the EC-WLP. The response options of the EC-WLP were then changed from the degree of importance to satisfaction to meet the needs of evaluation. Specifically, the response options were anchored from “very dissatisfactory” to “very satisfactory”. One hundred and eighty college students, including design and nondesign students from four universities in the north, middle, and south of Taiwan, were randomly sampled to evaluate the practical value of the EC-WLP. The selected websites were GEPT (http://www.gept.org.tw/), NSCU (http://cu.nsysu.edu.tw/), and PCSCHOOL (http://www.pcschool.tv/). GEPT is a language training and testing center, NSCU is a famous university in Taiwan, and PCSCHOOL is a private e-learning service. These learning websites had varied constructs, which are good for exploring different design styles and features. The GEPT included audio and the PCSCHOOL provided video instruction. Except for the absence of audio and video, the properties of NSCU were similar to those of the other two websites. Moreover, all the websites were similar in frameworks and they allowed users to obtain guest probation. Finally, three experts in web-based learning platforms were purposely sampled to analyze whether the online evaluation results were consistent with the website properties. Again, the experts were paid US$ 50 each.. 4. Results 4.1. Development of design criteria and the EC-WLP Two hundred and seventy-five test items were collected from five relevant questionnaires, namely, the Digital Learning Material Quality Standard (Industrial Development Bureau, Ministry of Economic Affairs, 2004), Iris Magical Digital Learning Form (Chiu, 2003), Student Online Learning Result Survey (National Taipei University of the arts-e college, 2003), DOVILES Manual (Open University of Hong Kong, 2004), and Evaluation of Learning Platform Technologies (IST, 2003). Collectively, these instruments contain design criteria in instruction, techniques, platform.

(6) interfaces, and function operations found in business, academic, and institutional studies in different countries. The broadness of these resources ensures that the developed criteria in this study can be universally applied. Ninety-seven items were selected and combined into the initial design indicators. Then, this study employed the Delphi technique and standardized scale compilation procedure to screen and revise these indicators. During the five rounds of revision, the revised data were gathered prior to Delphi technique application to ensure expert agreement before the next round. The main tasks in each round were as follows: (1) for the first round: integrating similar indicators and revising the denomination, order, and wording to prevent subjective or inducible content; (2) for the second round: reaching consensus for disagreements or ambiguous viewpoints and deciding the order of indicators; (3) for the third round: continuing to delete, add, or change the debatable indicators and wording as well as the order of indicators; (4) for the fourth round: continuing to revise the indicators; and (5) for the fifth round: continuing to refine the wording. The process stopped at the fifth round because the consensus among participating experts had been reached. Specifically, 16 indicators were deleted in the first round; two indicators were deleted in the second round and, at this time, four dimensions of indicators – instructional strategy, teaching material, learning tool, and learning interface – were decided. Moreover, 13 indicators were deleted in the third round and 10 indicators were deleted in the fourth round. Finally, 56 indicators remained and were compiled into the initial version of the EC-WLP. To further refine and analyze the EC-WLP with regard to the appropriateness of each indicator and its content structure, the heuristic evaluation method was conducted. Specifically, two academic educator experts refined the dimensions of instructional strategy and teaching material, and two senior practical engineer experts refined the dimensions of learning tool and learning interface. After three rounds of mutual communication and integration via e-mails, 40 indicators were included in the EC-WLP and grouped into categories. The distribution of categories in the dimension of instructional strategy, teaching material, learning tool, and learning interface were 6, 6, 5, and 5, respectively (see Table 1 for the distribution of categories and indicators). Table 1. Content structure of the EC-WLP Dimension Category. Indicator. Instructional strategy 1. Instructional goal. Three indicators (indicator 1, 2, 8).

(7) Dimension Category. Indicator. 2. Evaluation. One indicator (indicator 3). 3. Assistance. One indicator (indicator 4). 4. Teaching. One indicator (indicator 9). 5. Communication. One indicator (indicator 10). 6. Other strategies. Three indicators (indicator 5, 6, 7). Teaching material 1. Accuracy. Two indicators (indicator 11, 12). 2. Paragraph division. One indicator (indicator 13). 3. Appropriateness. Four indicators (indicator 14, 16, 17, 20). 4. Range scheme. One indicator (indicator 15). 5. Clear topic. One indicator (indicator 18). 6. Systematicness. One indicator (indicator 19). 1. System tool. Two indicators (indicator 21, 28). 2. Facilitation teaching. Four indicators (indicator 22, 23, 24, 29). 3. Linking function. Two indicators (indicator 25, 26). 4. Usability. One indicator (indicator 27). 5. Navigation design. One indicator (indicator 30). Learning tool. Learning interface 1. Text. Two indicators (indicator 31, 32). 2. Image. Two indicators (indicator 33, 34). 3. Animation. Two indicators (indicator 35, 36).

(8) Dimension Category 4. Video. Indicator Two indicators (indicator 37, 38). 5. Overall interface design Two indicators (indicator 39, 40) View Within Article. 4.2. Reliability and validity analyses of the EC-WLP A questionnaire survey was conducted to analyze the reliability and validity of the EC-WLP in this study. The retrieved questionnaires were 158 and the valid questionnaires were 150. Factor analysis extracted four factors, namely, learning interface, teaching material, instructional strategy, and learning tool. The total variance explained was 67.497%. Although indicator 1 was in factor 2, its factor loading in factor 3 (0.371) was acceptable (see Table 2). Based on the theoretical structure confirmed by experts aforementioned, we decided to keep this item in factor 3. Cronbach's α coefficients for the EC-WLP and the four factors were 0.977, 0.945, 0.946, 0.926, and 0.924, respectively, which clearly indicated that the EC-WLP is reliable. Table 2. Rotated component matrix Indicator. F1. 32. The words frequently convey information. 0.812. 31. The texts can be clearly read. 0.774. 34. The graphics and text complement and support comprehension improvement. 0.758. 33. The images clearly communicate information. 0.746. 38. The video transmission is smooth and does not lag. 0.718. 39. The interface design is pleasing and artistic. 0.699. 36. The animation design increases learning desire. 0.675. 35. The animation design clearly communicates information. 0.675. 37. The video quality is clear and good. 0.628. F2. F3. F4.

(9) Indicator. F1. 40. The interface design is creative. 0.594. F2. 13. The teaching material paragraph is clear. 0.736. 12. The teaching material is objective. 0.734. 15. The teaching material scheme is appropriate and materials correlate 17. The teaching material quality is appropriate and meets learners’ capabilities 19. The teaching material organizational structure is clear and systematic 16. The teaching material quantity is appropriate and meets learners’ needs 14. The teaching material induces learning motivation 20. The teaching material cases and situations meet learners’ cognitive abilities. F3. 0.703. 0.694. 0.693. 0.691 0.683 0.649. 11. The teaching material is accurate. 0.648. 18. The teaching material unit topic is clear and definite. 0.624. 1. It clearly indicates the instruction goal. 0.537. 0.371. 5. It provides cases and situations to improve students' understanding. 0.762. 6. It applies various learning facilitation medias. 0.741. 7. It applies novel and challenging strategies to increase motivation. 0.740. 9. It effectively integrates learners’ past learning experience and knowledge.. 0.735. 4. It provides Frequently Asked Question (FAQ). 0.687. 8. The presented content is correct in its instructional goal. 0.684. F4.

(10) Indicator 10. It provides learner communication and interaction opportunities (e.g. online discussion). F1. F2. F3. F4. 0.630. 3. It assigns evaluation practice for the class. 0.571. 2. It indicates knowledge and techniques to be learned. 0.496. 22. It provides search functions. 0.684. 24. It provides learning records. 0.669. 25. The menu linkage displays normally. 0.633. 28. It provides quick error instruction. 0.609. 21. It provides practical learning tools (e.g. online notebook). 0.603. 23. It provides related software for downloading. 0.591. 29. It provides the mechanism to ask for systematic manager help. 0.577. 30. The navigation is clear and easily understood. 0.568. 27. It provides learner process management. 0.559. 26. The category is appropriate. 0.532. Percentage of variance. 19.327 18.817 15.959 13.394. Cumulative %. 19.327 38.144 54.103 67.497. Note: Extraction Method, Principal Component Analysis; Rotation Method, Varimax with Kaiser Normalization. View Within Article. The means of indicators in the EC-WLP ranged from 3.38 to 4.15 and the standard deviations ranged from 0.872 to 1.072, which indicated that the participants regarded the indicators as important. Moreover, correlation analysis found that the four dimensions and their indicators were significantly correlated, rs = 0.700–0.784, ps < 0.01..

(11) 4.3. Application of the EC-WLP The EC-WLP was employed to evaluate three websites via a questionnaire survey to examine its practical value in this study. The retrieval rate of valid questionnaire was 96%. Cronbach's α coefficients of the three websites were 0.871, 0.786, and 0.863, respectively. Correlation analysis found that the four dimensions of the three websites were significantly correlated, rs = 0.360–0.649, ps < 0.01. Therefore, the satisfaction degree of instructional strategy, teaching material, learning tool, and learning interface was mutually influenced. Moreover, comparisons of means found that the nondesign students had higher scores in learning tool and learning interface than the design students. Descriptive statistics of the three websites in the four dimensions are depicted in Table 3 and Fig. 1. Table 3. Descriptive statistics of the three websites GEPT. NSCU. PCSCHOOL. Mean SD. Mean SD. Mean SD. Instructional strategy 3.55. 0.380 3.53. 0.426 3.59. 0.347. Teaching material. 3.60. 0.407 3.69. 0.871 3.59. 0.437. Learning tool. 3.46. 0.446 3.46. 0.528 3.56. 0.409. Learning interface. 3.35. 0.478 3.40. 0.492 3.57. 0.515. View Within Article. Full-size image (27K) Fig. 1. Curve diagram satisfaction of the three websites. View Within Article.

(12) The analyses made by experts found differences among the three websites selected and these results were consistent with those found in the real online evaluation by students. Specifically, the findings were as follows. (1) Instructional strategy: the PCSCHOOL was the best in terms of media style and interface instruction. (2) Teaching material: the teaching material in the NSCU was rich and its content was ranged orderly from simple to complex. This website also provided related graphics and materials for downloading. (3) Learning tool: the PCSCHOOL outperformed the others in learning tool. Its menu and navigation design was clear and easily operated; however, the color of its menu did not change while moving over buttons. On the other hand, the buttons on NSCU website were not grouped appropriately under one menu. (4) Learning interface: the text and image in the PCSCHOOL was clear and in good quality, providing delicate pictures and literary compositions.. 5. Discussion 5.1. Development of the design criteria and the EC-WLP The main purpose of this study was to develop objective criteria for designing web-based learning platforms as well as develop a reliable and valid scale for evaluating such platforms via cross validation of qualitative and quantitative methods. The analytical results of this study suggested a four-dimensional design criteria and the developed EC-WLP had satisfactory reliability and validity. The four-dimensional design criteria found in this study were instructional strategy, teaching material, learning tool, and learning interface. The high Cronbach's α coefficient of the EC-WLP and the significant correlations among its four dimensions indicated that the evaluation indicators in the EC-WLP have good internal consistency. The analytical results in this study also demonstrated that the developed design criteria have satisfactory content validity, construct validity, and external validity. This study employed Delphi technique to gather the design indicators and then invited four experts to refine the EC-WLP as well as analyze its content structure. The results clearly indicated that the EC-WLP has good content validity. Moreover, this study employed factor analysis to examine the construct validity of the EC-WLP. The extracted factors and its distribution of indicators were almost the same as those confirmed by experts and the variance explained was 67.497%. Finally, the external validity of the EC-WLP was evidenced by the finding that the results of experts' analyses lent support to those of students' online evaluation. Briefly, the findings in this study strongly supported the idea that the EC-WLP is an effective and reliable tool for evaluating web-based learning platforms. Studies show that instructional theory integration for practical application is critical for successful web-based learning system management (e.g. [Barab and Duffy, 2000], [Chen and You, 2001], [Hong, 1999], [Jonassen, 2002], [Jonassen, 1999], [Peck et al., 1999], [Yazici et al., 2001] and [Wen, 2002]). The findings in this study strongly suggested that the EC-WLP is developed based on important learning and instructional theories. For example, the meaningful learning theory was.

(13) presented in the dimensions of instructional strategy (indicator 1, 5, 7, 8 and 9) and teaching material (indicator 19); the scaffolding theory was displayed in the dimensions of instructional strategy (indicator 4 and 6) and learning tool (indicator 27–33); the anchor instruction and CSCL theories existed in multiple dimensions (indicator 5, 7, 12 and 28). The theory of constructivist learning environment was illustrated in all indicators of the dimensions of learning tool and learning interface. Apparently, the proposed design criteria and the developed EC-WLP in this study were derived from an elaborate theoretical framework. In addition, the effectiveness of these criteria and the EC-WLP were proven by students' online evaluation of three platforms. Therefore, the developed design criteria and the EC-WLP in this study are valuable for academic and practical use with regard to rating web-based learning platforms. Tan et al. (1999) claimed that most evaluation scales present evaluation items directly and without a standardized development procedure. This study emphasized systematic and continuous process of compilation. Whether in developing design indicators via the Delphi technique and heuristic evaluation method, or in revising the indicators via a careful compilation process, the thorough and systematic procedure employed in this study established a good example for developing effective evaluation criteria of web-based learning platforms. Specifically, this study not only developed objective design criteria and effective evaluation scale, but also served as a model for constructing an evaluation scale through a standardized process. However, this study found that the participants' satisfaction in instructional dimension (instructional strategy and teaching material) was higher than that in technical dimension (learning tool and learning interface) across the three websites selected. The result lent support to the argument that instructional website design should focus on learning rather than on bright and dazzling design; it is important to embed instructional theory into the website design and consider educational demands such as learning, instruction, and curriculum to facilitate meaningful learning (Chen and You, 2001). 5.2. Differences among web-based learning platforms This study evaluated three instructional websites. It was found that the PCSCHOOL was evaluated as the best platform with regard to instructional strategy, learning tool, and learning interface. It is reasonable because, compared to the other two platforms, the PCSCHOOL adds audio instruction, provides a more user-friendly menu and navigation design, and better quality texts and images. However, the NSCU outscored the others in teaching material for its rich and well-structured teaching materials. These findings, again, supported the argument that a well-designed online learning platform should be characterized by embedded learning and instructional theories (Chen and You, 2001), organized course content (Yue, 2003), skillful use of multimedia (Khan, 1997), and good accessibility (Keith, 2003). 5.3. Differences between design and nondesign students.

(14) This study found that design students had lower satisfaction in learning interface and learning tool dimensions than nondesign students. This finding is in line with the argument that design students are sensitive to images and color and, therefore, tend to look for perfection ([Lee, 2000], [Lin, 2000], [Tzeng, 1987], [Tzeng, 2002] and [Yang et al., 2005]). Hence, compared to nondesign students, design students were more critical and not easily satisfied with the interfaces in terms of their aesthetic design. Moreover, the web-based learning tools were developed for general education courses rather than for specific course in the field of design; functions were therefore more emphasized than aesthetic design. These reasons may explain why the design students were not as satisfied as nondesign students in learning tool dimensions.. 6. Conclusions and suggestions The importance of developing valid design criteria and tools for evaluating web-based platforms was greatly emphasized. This study developed objective design criteria and an effective evaluation tool for web-based learning platforms from a holistic perspective via an elaborative and standardized development procedure. The main processes in this study included establishing the initial design criteria, refining the criteria, developing the EC-WLP, undertaking reliability and validity analyses of the EC-WLP, and conducting online evaluation via the EC-WLP. Based on this elaborative evolving process, four dimensions of design criteria with 40 indicators were proposed and an evaluation scale with good reliability and validity was developed in this study. Moreover, the standardized development procedure in this study provided a good example for developing an effective evaluation tool for web-based learning platforms. Therefore, the findings in this study could contribute to enhancing the quality of designing web-based learning platforms as well as the effectiveness of web-based learning. Good design criteria require continuous revision and update to meet environment changes. Therefore, a database should be constructed to collect online evaluation results via the EC-WLP to help update its scale norms and design criteria and undertake comparison of cross-cultural differences, thus, increasing its value for practical use. In addition, this study found differences between design and nondesign students in evaluating web-based learning platforms. Future studies should compare students across disciplines to develop a platform that meets the needs of students with varied backgrounds.. References Allen and Allen, 2002 M.W. Allen and M. Allen, Michael Allen's Guide to e-learning, John Wiley and Sons, Hoboken, NJ (2002)..

(15) Barab and Duffy, 2000 S. Barab and T.M. Duffy, From practice fields to communities of practice, Theoretical Foundations of Learning Environments, Lawrence Erlbaum, Mahwah, NJ (2000), pp. 25–55. Becta, 2005 Becta, An Introduction to Learning Platforms, British Educational Communications and Technology Agency (2005) Available from: http://www.becta.org.uk/corporate/publications/documents/LEA_advice_learning_platform.pdf. Bostrom et al., 1990 R.P. Bostrom, L. Olfman and M.K. Sein, The importance of learning style in end-user training, MIS Quarterly 14 (1) (1990), pp. 101–109. Full Text via CrossRef Chen and You, 2001 W. Chen and M. You, Internet mediated design course: the construction of the Internet to assist design course, Design Research 2 (2001), pp. 109–115. Full Text via CrossRef | View Record in Scopus | Cited By in Scopus (0) Chiu, 2003 Chiu, H., 2003. A Study on the Implementation of Instructional Website Assistance by Concept Mapping. Unpublished master thesis, National Changhua University of Education, Taipei, Taiwan. Hong, 1999 M. Hong, E-learning, Hua-Cai, Taipei (1999). Horton, 2000 W. Horton, Designing Web-based Training: How to Teach Anyone Anything Anywhere Anytime, John Wiley and Sons, Hoboken, NJ (2000). Hsu and Cheng, 2005 C. Hsu and I. Cheng, The development and design of teaching platform: taking Blackboard and NSCU Cyber University as examples, Journal of Educational Research and Development 1 (1) (2005), pp. 177–206. Huang, 1987 Z. Huang, Curriculum Evaluation, Sh-Ta, Taipei (1987). Huang, 2003 Huang, R., 2003. The Study of Professional Technological Competence Possessed by Vocational High School Graduates Majoring in Graphicarts Communications. Unpublished master thesis, National Taiwan Normal University, Taipei, Taiwan..

(16) Industrial Development Bureau, Ministry of Economic Affairs, 2004 Industrial Development Bureau, Ministry of Economic Affairs, Quality Standards in e-learning (2004) Available from: http://www.elearn.org.tw/eLearn/download/sp3/93. IST, 2003 IST, Evaluation of Learning Platform Technologies – A Contribution to the UNIVERSAL Project (2003) Available from: http://nm.wu-wien.ac.at/lehre/seminar/s2001/LearningPlatformTechnologies.pdf. Jonassen, 1999 D.H. Jonassen, Designing constructivist leaning environment, Instructional Design Theories and Models: A New Paradigm of Instructional Theory, Lawrence Erlbaum Associates, Mahwah, NJ (1999), pp. 215–239. Jonassen, 2002 D.H. Jonassen, Designing Constructivist Learning Environments (2002) Available from: http://www.coe.missouri.edu/. jonassen/courses/CLE.. Keith, 2003 Keith, H., 2003. A Scorm-based Content Aggregation and Presentation System. Unpublished master thesis, Tamkang University, Taipei, Taiwan. Khan, 1997 B.H. Khan, “Web-based Instruction (WBI): What is it and Why is it?” Web-based Instruction, Educational Technology Publication, Englewood Cliffs, NJ (1997) pp. 6–7. Khan and Vega, 1997 B.H. Khan and R. Vega, Factors to Consider When Evaluating a Web-based Instruction Course: a Survey. Web-based Instruction, Educational Technology Publication, Englewood Cliffs, NJ (1997) pp. 375–378. Lee, 2000 Lee, E., 2000. The Study of Information-seeking Behavior: A Case of the Students of the Design College of Shih Chien University. Unpublished master thesis, Tamkang University, Taipei, Taiwan. Lin, 2000 Lin, R., 2000. A Study on the Learning Attitude of Students of Design Department. Unpublished master thesis, National Taiwan University of Science and Technology, Taipei, Taiwan. Lin, 2004 Lin, J., 2004. The Design and Implementation of the Scorm Conformed Software for Concept Map. Unpublished master thesis, National Taiwan University of Science and Technology, Taipei, Taiwan..

(17) McGreal, 1998 R. McGreal, Integrated distributed learning environments (idles) on the Internet: a survey, Educational Technology Review spring/summer 9 (1998), pp. 25–31. National Taipei University of the arts-e college, 2003 National Taipei University of the arts-e college, Student Online Learning Result Survey (2003) Available from: http://elearn.tnua.edu.tw/project/database/survey_st.html. Nielsen, 1993 J. Nielsen, Usability Engineering, Morgan Kaufmann, San Francisco, CA (1993). Open University of Hong Kong, 2004 Open University of Hong Kong, DOVILES Manual (2004) Available from: http://www.ouhk.edu.hk/doviles/index.html. Ou, 2000 Y. Ou, A preliminary study of evaluating quality on the web, Journal of Basic Education 13 (2000), pp. 219–238. Peck et al., 1999 K.L. Peck, D.H. Jonassen and B.G. Wilson, Learning With Technology: A Constructivist Perspective, Merrill, Upper Saddle River, NJ (1999). Rasmussen and Davidson-Shivers, 1998 K.L. Rasmussen and G.V. Davidson-Shivers, Hypermedia and learning styles: can performance be influenced?, Journal of Educational Multimedia and Hypermedia 7 (4) (1998), pp. 291–308. Roberts and McInnerney, 2006 T.S. Roberts and J.M. McInnerney, The course is online: why aren't the students learning?, International Journal of Instructional Technology and Distance Learning 3 (9) (2006), pp. 17–28. Rowe and Wright, 1999 G. Rowe and G. Wright, The Delphi technique as a forecasting tool: issues and analysis, International Journal of Forecasting 15 (4) (1999), pp. 353–375. Abstract |. PDF. (135 K) | View Record in Scopus | Cited By in Scopus (99) Rowe and Wright, 2001 G. Rowe and G. Wright, Expert opinions in forecasting: the role of the Delphi technique, Principles of Forecasting: A Handbook for Researchers and Practitioners, Kluwer, Norwell, MA (2001), pp. 125–144. Sarapuu and Adojaan, 1998 Sarapuu, T., Adojaan, K., 1998. Evaluation Scale of Educational Websites. ERIC Document Reproduction Service No. ED427733..

(18) Swisher, 1994 K. Swisher, American Indian learning styles survey: an assessment of teacher knowledge, The Journal of Educational Issues of Language Minority Students 13 (1994), pp. 59–77. Tan et al., 1999 D. Tan, C. Chen and M. Lai, The life style scale of ICP Taiwan database reliability and validity test, reconstruction and cross-sectional comparison with a marketing view, Web Journal of Chinese Management Review 2 (7) (1999), pp. 69–80 Available from: http://cmr.ba.ouhk.edu.hk/cmr/oldweb/n9/981105.html. Tzeng, 1987 J. Tzeng, The research of the designer learning development, Ming Chuan Design Education, 130 (1987). Tzeng, 2002 Tzeng, I., 2002. Divergent Thinking Detection and Analysis for Students of Industrial Design. Unpublished master thesis, National Chiao Tung University, Hsinchu, Taiwan. Tung, 2003 Tung, H., 2003. A Study on the Critical Factors of the Performance of e-learning Application on Education from the Learning Theory Perspective. Unpublished master thesis, Ming Chuan University, Taipei, Taiwan. Wang, 1998 Y. Wang, An application of curriculum evaluation using Delphi method, Education Resources and Research 25 (1998), pp. 43–46 Available from: http://www.socialwork.com.hk/artical/educate/gz11.htm. Wang, 2000 Y. Wang, A study on factors influencing computer attitude and learning performance – a field experiment of web page design in high school, Journal of Information Management 7 (1) (2000), pp. 1–106. View Record in Scopus | Cited By in Scopus (0) Waterhouse and Rogers, 2004 S. Waterhouse and R.O. Rogers, The importance of policies in e-learning instruction, The IT Practitioner's Journal Educause Quarterly 27 (3) (2004) Available from: http://www.educause.edu/pub/eq/eqm04/eqm0433.asp. Wen, 2002 Wen, F., 2002. The Influence of Web-based Collaborative Environment on Learning Performance. Unpublished master thesis, Shu Te University, Kaohsiung, Taiwan..

(19) Wu, 2000 Wu, C., 2000. Comparing the Student's Learning Performance Between the Web-based Teaching and the Conventional Teaching – a Case of Word 2000. Unpublished master thesis, Tamkang University, Taipei, Taiwan. Yang et al., 2005 M. Yang, F. Chen and M. You, The exploitation of difficulties of industrial design student and the implications for career guidance to design education, Journal of Design 10 (2) (2005), p. 63. Yazici et al., 2001 Yazici, S., Boyle, T., Khan, T., 2001. Towards a Multimedia Learning Environment for Object Oriented Design. Paper Presented at the LTSN-ICS 2nd Annual Conference. Available from: http://www.ics.heacademy.ac.uk/Events/conf2001/papers/Yazici.htm. Yue, 2003 X. Yue, Online English Teaching Resource and Support (2003) Available from: http://edtech.ntu.edu.tw/epaper/920810/prof/prof_1.asp. Yu, 2002 Yu, S., 2002. The Effect of Hypermedia Structure and Learning Style on Learning Achievement About Hypermedia. Unpublished master thesis, National Dong Hwa University, Hualien, Taiwan. Zeng et al., 2003 X. Zeng, J. Su, W. Cai and D. Su, A Study of Standardization of Teaching Materials in e-learning (2003) WISCS 2003. Available from: http://kbc.ntcu.edu.tw/teaching/2003summer/onlinetest/WISCS2003/B1-1.pdf..

(20)

參考文獻

相關文件

Constant review and flexibility in fine-tuning and re-engineering the strategies

e-Learning activities that support the teaching of Reading Courseware platforms for Reading Comprehension.. Practise downloading eBooks

Improve macro-average F-measure: 0.333 → 0.511 Five-fold cross validation for better thresholds Threshold T j = average of five thresholds. Tuning threshold significantly

• Information retrieval : Implementing and Evaluating Search Engines, by Stefan Büttcher, Charles L.A.

In this chapter, the results for each research question based on the data analysis were presented and discussed, including (a) the selection criteria on evaluating

For Experimental Group 1 and Control Group 1, the learning environment was adaptive based on each student’s learning ability, and difficulty level of a new subject unit was

The main objective of this system is to design a virtual reality learning system for operation practice of total station instrument, and to make learning this skill easier.. Students

A decision scheme based on OWA operator for an evaluation programme: an approximate reasoning approach. A decision scheme based on OWA operator for an evaluation programme: