System Design

With the above discussion, it is obviously mastery learning is an effective and practical teaching approach. To relieve teacher’s burden and provide sufficient and convenient environment to student, a web-based mastery learning system is presented in this thesis. The mastery learning process is shown in figure 4-1. In his system, the

process is a cyclical approach which consists of unit lecturing, formative assessment, remedial learning, and advanced learning. In traditional mastery learning process, remedial learning usually takes about 2 times. If any student does not achieve mastery level after second remedial learning, the process is terminated. With the capability of networked system, remedial learning can repeat as many times as it needs. The reason is remedial learning is done by networked system instead of teacher. One of many advantages about Networked system over traditional approach is networked system is more persistent than human beings (Fogg, 2003). The remedial learning may repeat as long as student does not achieve mastery level. The approach also realizes what Carroll mentioned student’s chance of perseverance and learning. The chance of students’ perseverance and learning can be enhanced through this system; therefore they can achieve course objectives. In this approach, the main purpose of networked technology is to let learning process proceeds smoothly. In traditional classroom teaching activities, there is time limitation. It is hard to allocate suitable time and location for student to interact with teacher or peers after class. With the aids of networked and information technology, student can participate remedial learning beyond time and location boundary.

Figure 4-1: mastery learning process

Unit

Remedial learning has two dimensions. One is learning activity and another is formative assessment. Formative assessment not only assesses student’s mastery level, but also diagnoses student’s weak point and insufficient part. Therefore, formative and diagnostic aspects are taken into consideration while designing test items. By way of test items on diagnostic assessment, and inspecting student’s answers, then student’s learning problem can be revealed. Once identifying student’s learning problem, suitable or appropriate remedial material can be presented to each individual student.

Each test item of diagnostic assessment addresses an atomic concept. In this manner, student’s cognitive process in learning the subject domain can be easily identified.

Taking common fraction problem in basic mathematics domain as example, its solving process consists of several atomic concepts. In order to correctly solve this problem, the student should have knowledge of these atomic concepts. If there is misunderstanding on any atomic concept, then it will lead to incorrect answer. While designing test item of diagnostic assessment, teacher identifies all atomic concepts of a specific problem domain, then design a test item for each atomic concept. If student can correctly answer this item, that means he/she has understood it. Furthermore, if student correctly answer all test items relate to a specific problem domain, that means he/she has achieve mastery level for this learning unit or domain. Formative assessment can be in traditional way or web-based. System presented in this thesis adopts a mixed model (Lin, Kuo, Yuan, 2004). In this model, activity such as learning or assessment may be traditional way or web-based relies on practical consideration.

If teacher prefers traditional way, then assessment is in paper and pencil form. After collecting and grading student’s answer, if any student does not achieve mastery level, teacher will select suitable remedial material on the system, and asks student to take learning. The preferable way is in web-based form. Teacher puts all test items into

system. At the end of each lecturing unit, students are requested to take formative assessment on the web. System will automatically select remedial learning material for students, if they do not achieve mastery level. No matter formative assessment is in traditional form or web-based form, further formative assessment after remedial learning is on the web. Test items for any formative learning after remedial learning are randomly selected by system. Each item has accompanied with a hint or further explanation. If student’s answer is incorrect, system will present the related hint to student. This will make student gain immediately assistant.

Figure 4-2: system architecture

This system consists of database, user interface, and mastery learning subsystems as shown in figure 4-2. Database subsystem has two contents, one is for student’s profile and learning portfolio, another is learning material and test item bank. User interface subsystem also has two forms. One is student’s interface which allows to learn, assess, discuss with classmate, build own learning portfolio and communicate with teacher.

Another is teacher’s interface which allows to enter student’s profile, build learning material, build test item bank, set rules for remedial learning, view student’s learning portfolio and communicate with students. The functionalities of user interface is summarized in figure 4-3. Mastery learning subsystem is core of this system. It

User interfac

e

Internet

Database Mastery

learning subsystem

Student 1 Student n Teacher

provides a platform to let student and teacher perform mastery learning process.

Figure 4-3: user functionality 4.4 Experiment Design

In order to evaluate effectiveness and functionality of this system, an experimental has been conducted on an Institute of Technology located in northern Taiwan. The experimental course was principle of electrical circuitry. Students attended this class were junior and fail on previous study. This kind of classes sometimes is called summer school which means open in summer vacation. Totally, 24 students attended this class in which one was female and others were male. Because of these students had studied same subject but fail on final score, it’s more suitable to introduce mastery learning into this class. As per definition of mastery learning, first step is to determine what student should learn. Then, divide learning material into several lecturing units. In this experiment, it is divided into 12 units. For each unit, teacher prepared lecturing material, test items of formative assessment, remedial learning material, and hint for every test item. Normal lecturing was on classroom, assessment was web-based form, and further as well as remedial learning was also through web-based system.

Each unit started with unit teaching. At the end of teaching, student was requested to perform web-based formative and diagnostic assessment. It this experiment, mastery level was defined as correctly answering 85% of test items. If student achieved this level, he/she has mastered this unit, and was encouraged to take further advanced study. In contrast, if student did not achieve this level, he/she was requested to attend remedial learning. The remedial learning material was not exactly same as unit lecturing material. The reasons are (1) in different lecturing approach (traditional and web-based), the lecturing material should be in different form to let student learned most, (2) because of student did not accomplish last learning, the learning material should be amended, (3) the remedial learning material should focus on what student did not understand, in other word did not answer correctly. As mentioned above, diagnostic assessment is part of formative assessment, student’s weak point or misunderstanding can be identified through test items. It is teacher’s responsibility to clarify a test item should link with which course concept, and what remedial material should be presented if student gave incorrect answer. System presented in this thesis provides a convenient and systematic way to let teacher accomplishes this task.

Student who did not get mastery, should arrange and allocate time to attend remedial lecturing after class. Student could attend formative assessment after remedial learning. If student still did not get mastery, the above remedial process should be repeated until student gets mastery. System provides another two features to assist student perform remedial learning. Student can discussed with classmates through discussion board or contacted teacher while encountering difficulty on remedial learning. To encourage students who have gotten mastery share acquaintance with classmate, system will give extra score to these students. For some students who want to apply scholarship or get high rank, this strategy will provide motivation. With

regarding to contact teacher, this system provide a more effective way. Student raised problem on the system, and system will deliver this message with predefined channel such as email and SMS (Short Message Service), etc. set by teacher. The purpose is to let teacher gets this message and provides assistance to student as soon as possible.

Teacher can set a learning deadline on this system. If there is any student does not get mastery over this deadline, system will automatically inform teacher. Then, teacher may contact student to provide necessary assistance or teaching.

Figure 4-4: remedial learning

Figure 4-5: formative assessment

Figure 4- 6: result of assessment

Figure 4-7: hint for test item 4.5 Summary

At the end of course, a satisfactory survey was conducted. The questionnaire and the result are shown in table 4-1. In summary, about 94% of students claimed this system has positive effect on their learning. 95% of students claimed this system changed their learning attitude toward more positive. These results agreed on Fogg’s viewpoint that computing system may be used as a persuasive technology to change people’s attitude. With regarding to provide instant and specific hint to each item student gave wrong answer, about 91% of students thought it gave great help. This is a strength part of web-based mastery learning. In traditional approach, it is hard to accomplish especially if such students are not a few. All of students spent more time on learning.

The average time increased on learning is about 2.5 hours per week. These extra hours do not squeeze learning time of other courses. This is another evidence that student’s attitude has been changed and willing to spend more time on learning. At the end of course, not all 24 students got mastery, two of them fail in final score. The possible

reasons are as follows. (1) This was a summer school class, students were fail at last semester, that means some of them may have serious learning problem which may not be cured by mastery learning. (2) As stated previously, people indeed have difference on ability, and are prone to different achievement. (3) Although computing system can be a persuasive tool, but not all students can be persuaded. Persuasive effect of this system may not work for some of students.

Several issues need to be pointed out. Whether student gets mastery is checked by formative assessment. Besides, if student does not get mastery, then should attend remedial learning. Therefore, formative assessment should have functionality of diagnostic assessment. How to design a suitable and adequate test items to diagnose student’s real learning difficulty, so that suitable remedial learning material can be presented to student is an important issue. For each subject, domain experts or teachers may form a task force to design lecturing units, test items and remedial learning material.

Secondly, the modality of learning material on the web may affect student’s learning performance. Fogg’s study has confirmed this point (Fogg, 2003). In Fogg’s research, attractive material is more persuasive than unattractive one. This is a critical issue on network- or web-based learning system. Same as first point, if teacher or domain expert can spend more effort to design learning material, it may be more effectiveness for student’s learning.

Finally, teacher may aware student’s learning situation in face to face lecturing circumstance, and necessary intervention or assistance may be applied instantly. In this manner, student may follow teacher’s lecturing step as close as possible. In web-based learning circumstance, it lacks in such facility or channel. How to provide a mechanism to aware student’s learning status on the web in real time manner, it is an

issue need to be addressed.

1 Teacher said every student learns almost everything if time is sufficient and has proper attitude. It gives me confident to my learning.

28.6% 52.4% 19.0% 0.0% 0.0%

2 I feel this system has positive effect to my

learning. ^{33.3% 52.4%} 9.5% 4.8% 0.0%

3 In this learning model, I spent more time

on learning. ^{42.9% 47.6%} 9.5% 0.0% 0.0%

5 In this learning model, I feel more active

on learning. ^{33.3% 42.9%} 9.5% 14.3% 0.0%

6 If I were positive and active as now, I

should not attend this summer school. ^{23.8% 28.6%} 42.9% 4.8% 0.0%

8 I intend to spend more time on learning at

networked learning environment. ^{52.4% 38.1%} 9.5% 0.0% 0.0%

9 In networked learning environment, it makes me more convenient to access learning material

23.8% 47.6% 23.8% 4.8% 0.0%

10 In web-based assessment, system promptly gives me hint for incorrect answer. It provides great positive impact on my learning.

47.6% 28.6% 14.3% 9.5% 0.0%

11 In web-based learning, it is as teacher is

always on my side. ^76.2% 9.5% 14.3% 0.0% 0.0%

12 In summary, I like this kind of learning

very much. ^{19.0% 52.4%} 28.6% 0.0% 0.0%

Time spent on learning per week. 2.5 hours in average 1: very disagree, 2: disagree, 3: no comment, 4: agree, 5: much agree

Chapter 5

Learning Management and Portfolio Package

In this chapter, a web-based course management system is introduced which allows teachers to manage course-related affairs such as, course announcement, grade processing, and learning status of each student. The system also facilitates communication between teacher and student, and provides real time learning feedback messages in a real-time manner which allows students to check their own learning status and make necessary adjustments. At the end of the course, both students and teacher can pack their own portfolio files which contain score trends, lecturing content, the course syllabus, discussion content, collected files, homework, self assessment, and teacher’s comment. The system completely records every detail related to student’s learning, then packs these details in an organized structure.

Students may then carry their own portfolios to the next learning stage. This is the optimal way to evaluate student’s learning progress and level. In order to improve accountability and reliability of portfolio files, this system adopts MD5 to construct an on-line verification mechanism.

5.1 Introduction

A broad range of teaching activities and various teaching materials are used within a school system. If their effects are to be maximized, a fair amount of time and energy will be needed by the teachers, students and administrative personnel in order to achieve that goal. The ways of enhancing learning activities with less time and energy, by taking advantage of new information and communication technologies, is becoming a more desired goal as those technologies become a part of everyday life. In a traditional learning environment, interactivities between teacher and student are

usually done through conversations in the class, or written comments and grades on an exam paper. To get a picture of how well they are doing, students look at their exam scores and compare them with other students’ scores. It is just about the only way to get a rough idea of how well he or she has done. If the work of providing scores is to be done for every single test and piece of homework, the teacher has to repetitively do the same job over and over. More over, if grades from previous exams were not provided, and temporary information from previous learning activities not preserved, the time and energy spent by the teacher to reference them would diminish the reference value which they should have.

In the web-based learning environment of today, when faced with the situation above, many teachers would put information on the Internet for reference. Since anonymity is uncontrollable on the Internet, it means that the teacher can only post data which can be viewed by everyone and he/she is therefore not able to advise each student based on his or her achievements. If the teacher was to advise a single student, e-mail would be a traditional way of doing it, but this would create a heavy load and create problems of content management on the teacher’s side. The possibility of incorrect data being sent is very high because of the quantity of e-mail the teacher needs to manage.

Recent advances in network based lecturing have resulted in the wide-spread availability of web-based course management system for use in all educational endeavors. A web-based course management system is responsible for following major tasks:

z Delivery learning content in a convenient and effective way z Manage student affairs,

z Manage instructional content,

z Track student progress, and

z Give student learning status in real time manner.

In response to the increasing demand of this type of management work required by teachers, some commercial companies provide services to do just that. The most popular commercial course management systems are WebCT and Blackboard. WebCT is an e-learning systems with a highly flexible e-learning environment developed in US. WebCT includes the following tools: Instructor Tools, Student Tools, Presentation Tools, Communication Tools, and Assessment Tools. Yip (2004) uses WebCT as a platform and tool to teach course online, and found that students generally prefer online teaching to the traditional classroom lecturing because they enjoy the various way to communicate with the instructors, tutors, and fellow students. Mclean &

Murrell (Mclean and Murrell, 2002) described that WebCT became an integral part of a problem-based learning, student-centered curriculum introduction in 2001 at a school of medicine. In this implementation, WebCT might serve four possible purposes: communication, resource delivery, the quiz/assessment, and student tracking. Blackboard is another famous course management system provider based in US. As the information shown on website, Blackboard product consists of following tools and features: Course Management, Content Authoring, Adaptive Content Release, Syllabus Builder, Teaching and Learning Tools, Assessments and Surveys, Grade book, and Reporting and Performance Dashboard.

Commercial products such as WebCT and Blackboard are powerful enough to fulfill most of institutes. As Siew & Shepherd (2002) mentioned that systems such as Blackboard and WebCT already exist and provide much of functionality. Why people don’t simply use them? The reason simply is WebCT and Blackboard are both commercial products, and thus expensive. In addition, people will gain the maximum

flexibility by building their own system which is built up with their know-how and experiences. There are a set of tools for students and lecturers available. But it lacks on the assessment and learning activity recording facilities.

Yet, there are some commercial web sites which provide convenient tools to assist teachers take care of student’s score, especially for teachers whose institute does not provide such web-based facility and expect to manage student’s affair with the power of web. MyGradeBook (2000), a subsidiary of Pearson Education, is such a company.

It provides password access to your grade book to keep parents informed and involved in motivating students. For students, a structured view of upcoming and past assignments, along with scores recorded to-date, helps to keep them on track throughout the grading period. Reduce the frequency and time spent in one-on-one parent conferences and notes.

Instead of commercial products, many educators are seeking open-source solutions.

The most popular open source software is ATutor (2000) and Moodle (2004). ATutor is an Open Source Web-based Learning Content Management System (LCMS) that emphasizes on accessibility and adaptability. Administrators can install or update

The most popular open source software is ATutor (2000) and Moodle (2004). ATutor is an Open Source Web-based Learning Content Management System (LCMS) that emphasizes on accessibility and adaptability. Administrators can install or update