The development of computerized two-tier diagnostic test and remedial learning system for elementary science learning

(1)

The Development of Computerized Two-tier Diagnostic Test and Remedial

Learning System for Elementary Science Learning

Ah-Fur Lai

Department of Computer Science, National Chiao Tung University, Taiwan

lai@tmue.edu.tw

Abstract

The main purposes of this study were to develop the web-based concept diagnostic and remedial system (Dr system for short), and conduct an elementary science learning experiment for evaluating its applicability.

1. Introduction

In science learning, abstract concepts are difficult to be understood, so many misconceptions were found in students’ mental model and affect the learning performance. How to diagnose the students’ misconception is an important and complex task in traditional classroom or e-learning environment. There are many diagnostic methodologies for science misconception, including concept mapping, interview, paper and pencil test, two-tier diagnostic test. To avoid the shortcoming of the p & p test and take advantage of interviewing and concept-mapping, two-tier diagnostic test was proposed by Treagust in 1985 [1][2]. The two-tier diagnostic test question is comprised of two tier items. The first tier of each item is a multiple choice question related to prepositional statement and it is part of the concept map as well. The second tier of each item consists of a multiple choice set of reasons for the answer related to the first tier item [1][2]. Most of the two-tier diagnostic test is administered by means of traditional paper & pencil-based method. In the traditional instructional context, after the learners’ misconception being detected, they will be provided the remedial instruction by their teachers. In the e-learning environment, diagnosing the students’ misconception and remedying the learners’ alternative concept are still two big tasks. Bringing testing, diagnosis and remedy instruction together makes e-learning valuable and meaningful. Nevertheless, few research pay attention to this work. This study employs the information technology to integrate the two-tier diagnostic test and remedial

learning task to develop an e-testing system called Dr system (that is, Dr system can diagnose the learners’ weakness on-line and offer the web-based prescriptive materials similar to the diagnostic behavior of a human doctor), and conducts a learning experiment to evaluate its applicability.

2. System Framework

To offer the integrating operation for testing, diagnosing, and remedial learning function, this study employed three-tier conceptual framework to design the Dr system, depicted as Figure 1. Each question in the test item pool contains two-tier diagnostic test items, the mapping digital remedial learning content, and the related concept structure. The concept structure plays the critical role of connecting the diagnostic test item and remedial learning content. In the 2-tier test development, the authors ought to create the concept structure or ontology for both test items and digital materials in advance. Afterward, when the authors create the test item or upload the digital content into the Dr system, they should specify its related sub-concept of item or content.

Figure 1. 3-tier conceptual framework for Dr system In the on-line diagnostic testing procedure, the learners take the first tier question retrieved from diagnostic test item pool shown as Figure 2. Next, they will respond to the second tier question designed for explaining the reason of answer of the first tier item shown as Figure 3. If the learners do not respond to both questions correctly, they are diagnosed to have misconception in their mind. The response patterns and

(2)

behaviors of the test-takers will be recorded in the student model. As soon as the students finish taking the diagnostic test, the system will generate the diagnostic report and propose the suggestive learning path for remedying their alternative concept or weakness. The students can accept the web-based remedial multimedia material immediately or later.

Figure 2. Example of first tier test

Figure 3. Example of second tier test

The Dr system consists of concept structure editor, 2-tier diagnostic test item management subsystem, prescriptive multimedia material management subsystem, report generating subsystem, remedial learning subsystem, and on-line testing subsystem, show as Figure 4. The study adopted ASP.NET and MS-SQL Server to develop the Dr system. In order to exchange the test items and digital materials with other systems, the system was designed to conform to the international XML-based standard including QTI and SCORM.

Figure 4. System architecture of Dr system

3. Evaluation Result

Based on the electro-magnet concept, the study developed the 2-tier diagnostic test items for conducting the science learning experiment. The participants are 57 sixth graders from Taipei city and

31 experienced elementary science teachers. The evaluation inventories adopted 5-point Likert scale (5=strongly agree, 1=strongly disagree). The evaluation result indicates that the teachers show high appraisal toward the Dr system shown in Table 1. And the students had positive attitude toward the system and learning approach as well.

Table 1. The evaluation result Group Subsystem /

awareness Mean deviation Std. Item management 4.52 .53 Report generating 4.56 .55 Online testing 4.44 .64 Remedial content 4.43 .63 Teaching usefulness 4.46 .62 teacher Total 4.48 .60 Interface 4.02 .95 affinity 4.10 .98 Learning usefulness 4.6 .87 studen t Total 4.07 .94

4. Conclusion

The two-tier diagnostic test provides a structured method for diagnosing the student’s misconception more precisely and it is suitable for large amount of survey. The abstract content can be visualized or consolidated by means of multimedia technology that learners can modify their mental representation using constant interaction to acquire the utterly correct concept or rule out the misconception. This study adopted the theory of two-tier diagnostic test and multimedia learning to implement the Dr system which integrates the testing, diagnosing and remedy learning on web-based environment.

This study applied the Dr system on elementary science learning. The experimental result indicated that both the teachers and the students showed high appraisal toward Dr system. Nevertheless, the development of two-tier diagnostic test item is hardworking and lengthy for traditional or on-line environment. It is suggested that the test item pool be designed by team work or virtual community and be shared through the web-based platform.

5. References

[1]Treagust, D. F. (1988). Development and use if diagnostic tests to evaluate students’ misconceptions in science.

International Journal of Science Education, 10(2), 159-169.

[2]Treagust, D. F. & Haslam, F. (1987). Diagnosing secondary students’ misconceptions of photosynthesis and respiration in plants using a two-tier multiple choice instrument. Journal of Biological Education, 21, 203-211.