行政院國家科學委員會專題研究計畫 成果報告
子計劃二：『個人化學習資訊蒐集、分析與推薦模組』之研 究(3/3)
計畫類別： 整合型計畫
計畫編號： NSC92-2520-S-216-001-
執行期間： 92 年 08 月 01 日至 93 年 07 月 31 日 執行單位： 中華大學資訊工程學系
計畫主持人： 曾秋蓉 共同主持人： 張鈿富
報告類型： 完整報告
處理方式： 本計畫可公開查詢
中 華 民 國 93 年 9 月 16 日
行政院國家科學委員會補助專題研究計畫 ■ 成 果 報 告
□期中進度報告
智慧型個人化網路學習、測驗與診斷服務平台之研究 - 子計 畫二： 『個人化學習資訊蒐集、分析與推薦模組』之研究(3/3)
計畫類別：□ 個別型計畫 ■ 整合型計畫 計畫編號：NSC-92-2520-S-216-001
執行期間： 92 年 8 月 1 日至 93 年 7 月 31 日
計畫主持人：曾秋蓉 共同主持人：
計畫參與人員：
成果報告類型(依經費核定清單規定繳交)：□精簡報告 ■ 完整報告
本成果報告包括以下應繳交之附件：
□赴國外出差或研習心得報告一份
□赴大陸地區出差或研習心得報告一份
□出席國際學術會議心得報告及發表之論文各一份
□國際合作研究計畫國外研究報告書一份
處理方式：除產學合作研究計畫、提升產業技術及人才培育研究計畫、
列管計畫及下列情形者外，得立即公開查詢
□涉及專利或其他智慧財產權，□一年□二年後可公開查詢
執行單位：中華大學
中 華 民 國 93 年 9 月 15 日
智慧型個人化網路學習、測驗與診斷服務平台之研究 - 子計畫 二：『個人化學習資訊蒐集、分析與推薦模組』之研究(3/3) A Personalization Engine for a Network-based Learning Environment
計畫編號：NSC-92-2520-S-216-001
執行期間： 92年 8月 1日至 93年 7 月 31日 計畫主持人：曾秋蓉
執行單位：中華大學
Abstract
With the rapid growth of computer and network technologies, the development of on-line computer-assisted learning systems has become an important and challenging issue.
In this project, we focus on the influence of multiple sources of personalization information, such as individual differences and learning styles, on the performance of learning science courses, from which a platform for helping teachers in developing adaptive subject materials for science courses is proposed. To determine the initial learning styles of the students, the Keefe questionnaire is employed in our approach. To precisely reflect the actual learning style and talent of each student, the interactions and learning results of each student are analyzed when adjusting the subject materials. An experiment was conducted to evaluate the performance of our approach, by employing one-way ANOVA and t-test to analyze the test results on three groups of students using different adaptive learning approaches. The analysis results show the novel approach is helpful in improving learning performance and is worthy of further study.
Keywords: computer-assisted learning, adaptive learning, learning style, distance education, science education
1. Introduction
With the recent rapid advances in computer and network technologies, educational researchers have developed methods, tools and environments for computer-assisted learning.
When used for educational purposes, hypermedia systems are particularly suitable for providing a higher degree of control, such that training or learning performance can be enhanced immensely by identifying the personal characteristics of students and adapting subject contents and presentation to better suit their needs.In additions, a hypermedia system can offer more than predefined learning paths by selecting different nodes in different orders, thus individual students produce a multitude of paths through the subject materials [1][20][28].
Several researches have already addressed the importance of adaptive learning, either in traditional instruction or in computer-assisted instruction. In studying the effect of adaptive learning in science courses, most CAI researches often focus on the determination of difficulty levels, learning paths and learning styles of subject material, whilst the interactions between personalization information, learning styles and adapting subject material are seldom taken into consideration.
In this project, a Multi-Source Adaptive Learning (MSAL) system is proposed. MSAL can assist instructors to construct adaptive subject materials for science courses, by taking personalization information and learning styles into consideration. To evaluate the performance of our approach, an experiment was conducted by employing one-way ANOVA and t-test to analyze the test results on three groups of students using different adaptive learning approaches. The analysis results show that the novel approach is helpful in improving learning performance in science courses, and is worthy of further study.
2. Relevant Work
When considering individual differences in adaptive learning, not only the cognitive levels but also the learning styles of students have to be considered. In the following subsections, various relevant studies addressing adaptive learning and learning styles are given.
2.1 Adaptive learning
Snow and Farr suggested that sound learning theories are incomplete or unrealistic if they do not include a whole person view, integrating both cognitive and affective aspects [26], which implies that no educational program can be successful without due attention to the personal learning needs of individual students. A single approach to instruction whether traditional or innovative, simply fails to do the job [10]. Russell suggested that educators should identify and acknowledge learning differences and make maximum use of the available technology to serve them accordingly [25].
In 1965, an Individually Guided Education (IGE) program was proposed for organizing and delivering educational experiences from teams that studied on how people learn and how to personalize instruction process. In 1994, the Comprehensive Application of Behavior Analysis to Schooling (CABAS) program was proposed [14]. CABAS schools are self-correcting and self-sustaining, and incorporate the science of teaching into every aspect of schooling. The Fred S. Keller School located in Yonkers, New York, is one of the schools participating in the program. The School functions as a cybernetic system of education in which the individualized instruction of each student influences the behavior of the entire education community.
Brusilovsky suggested using adaptive hypermedia to support individual learning [3].
The idea of adaptive hypermedia is to adapt the course content for a particular learner based on the profile or records of the learner. According to Brusilovsky’s approach, the adaptive
hypermedia system should satisfy three criteria: (1) it should be a Hypermedia or hypermedia system, (2) it should have a user model, and (3) it should be able to adapt the hypermedia using the user model [3].
Paolucci addressed the importance of individualization in hypermedia that any strategy should be adaptive and personalized [23]. To insure personalization, adaptive hypermedia systems should be capable of diagnosing and identifying each student’s misconceptions.
Based on the conception, Lo et al. developed a Hypermedia-based English Learning system for Prepositions (HELP), which provides non-native speakers of English learning diagnosis and remedial instruction according to their assessment results [18].
To provide effective adaptive learning on computer networks, it is important to understanding the on-line learning behaviors of students. In 1998, an on-line learning behavior diagnosis system was proposed [6]. Several parameters were defined and recorded to describe the on-line behaviors of students, such as idle time, response time, effective learning time, ineffective learning time, and login time. Based on those parameters, the system can detect several learning attitudes of students, such as “concentration”,
“willingness” and “patience”, and hence the course contents can be adapted to meet individual requirements.
It can be seen that, with the popularization of the World Wide Web, the use of hypermedia in learning has attracted the attentions of many researchers from the fields of computers and education. Therefore, it is worth studying how to effectively develop adaptive learning systems on the World Wide Web environment [19].
2.2 Learning Style
Learning style is a concept that followed the research from a cognitive perspective starting in the 1960's. Numerous writers have addressed the concept of learning styles and the various ways they are measured [2] [9][16].
Keefe [12] described learning style as both a student characteristic and an instructional strategy. As a student characteristic, learning style is an indicator of how a student learns and likes to learn. As an instructional strategy, it informs the cognition, context and content of learning. Learning style is a consistent way of functioning that reflects the underlying causes of learning behavior [10].
Learning problems frequently are not related to the difficulty of the subject matter but are associated with the type and level of cognitive process required to learn the material [11].
Gregorc and Ward claimed that if educators want to successfully address the needs of the individual they must understand what “individual” means and adjust their teaching styles to meet the learning styles of students [5].
Talmadge and Shearer have proved the existence of learning styles. Their study shows that the characteristics of the content of a learning experience are a critical factor affecting relationships that exist between student characteristics and instructional methods [27]. Reiff indicated that learning styles influence how students learn, how instructors teach, and how they interact [24]. Keefe asserts that perceptual style is a matter of student choice, but that preference develops from infancy in a subconscious way [10].
The knowledge to the features of student preferences is helpful for developing more flexible learning environments. For example, students with the visual-audiolearning style have greater recall of concepts that are presented visually [1]. All students can benefit from a responsive learning environment and from the enhancement of their learning skills [12].
Using one teaching style or learning style exclusively is usually not conducive enough to a successful educational program [1]. Additionally, research has shown that there is little relationship between overall college achievements and learning style [28], yet they indicated the possible relationships between learning style and performance in specific subject areas.
For example, there is evidence that individual cognitive learning styles are related to programming ability in novice programmers [2][20]; moreover, Larkin-Hein showed the
critical role that a learning style approach could play in terms of physics and engineering education [17].
The Learning Style Profile (LSP) provides educators with a well validated and easy-to-use instrument for diagnosing the characteristics of an individual's learning style. It also provides an overview of the tendencies and preferences of the individual student [12].
Several studies have reported that students learn in different ways, depending upon many personal factors and everyone has a distinct learning style [21][22].
There have been several models for defining and measuring learning styles proposed, such as the following: Kolb’s experiential learning [16], Dunn’s PEPS (Productivity Environmental Preference Survey)[4], and James Keefe’s four-fold framework [10]. The James Keefe four-fold framework aids in the classification of various learning style conceptualizations. This framework categorizes learning style instruments as measures of cognitive, affective or physiological styles, or as comprehensive instruments designed to measure several types of learning styles. It measures the strength of twenty-four skills across independent scales as follows:
Cognitive Styles
Cognitive styles of learning include those aspects of the brain, which perceive meaning and interact with the world, and are often used to predict student achievement on standardized tests. Students who are aware of their cognitive strengths can use them to assist in weak areas.
The following table shows the cognitive learning styles.
Cognitive learning style Functions of the cognitive learning style
Analytical skill Identifying simple figures hidden in a complex field; use the critical element of a problem in a different way Spatial skill
Identify geometric shapes and rotate objects in the
imagination; to recognize and construct objects in mental space
Discrimination skill Visualize the important elements of a task; to focus attention on required detail and avoid distractions Categorization skill
Use reasonable vs. vague criteria for classifying information; to form accurate, complete and organized categories of information
Sequential processing Process information sequentially or verbally; to readily
skill derive meaning from information presented in a step-by-step, linear fashion
Simultaneous processing Grasp visuospatial relationships; to sense an overall pattern skill from the relationships among the component parts Memory skill Retain distinct vs. vague images in repeated tasks; to detect
and remember subtle changes in information
Affective Styles
Affective styles of learning are a by-product of personality, cultural environment, parental and peer pressure and school influence. The following table shows the affective learning styles.
Affective learning style Functions of the affective learning style Persistence orientation Willingness to work at a task until completion Textrisk orientation Willingness to express opinions, speak out
Grouping preference Preference for whole class, large group, small group or dyadic grouping
Verbal-Spatial preference Preference for textor nontextactivities Manipulative preference Preference for "hands-on" activities
Physiological Styles
Physiological styles include perceptual modes and environmental factors that affect response to information as a visual, auditory, or emotive response. The following table shows the Physiological learning styles.
Physiological learning style
Functions of the physiological learning style
Perceptual response Initial reaction to information is visual, auditory or emotive Study time preference Preference for study time in early morning, late morning,
afternoon or evening
Posture preference Preference for formal or informal study arrangements
Mobility preference Preference for moving about and taking breaks vs. working until finished
Sound preference Preference for quiet study vs. background sound Lighting preference Preference for brighter or dimmer study areas
Temperature preference Preference for studying in a cooler or warmer environment
3. Implementation of MSAL
MSAL consists of eight modules: Course Editing Module, Tutoring System, System Management Module, User-Profile Database, Subject Material Database, Item Bank, Learning State Database and Learning Style Database (as shown in the following figure).
Instructors Students System Administrator
Learning State Database Learning Style
Database
Subject Material Database Item Bank
Database
User-Profile Database
Tutoring System
Computer Networks
System Management
Module
Course Editing Module
Course Editing Module enables the instructors to access the subject material database and the item bank. Tutoring System Module offers adaptive tutoring and test for the students based on each student’s learning style, ability and learning efficiency. The functions of Tutoring System Module include on-line tutoring, on-line discussions, profile modification, and self-assessment. System Management Module enables the system administrator to access the subject material database, the user-profile database and the item bank. The system administrator can also perform several system maintenance operations via this module, such as user account manipulation and discussion board management.
MSAL is implemented with PHP and MySQL on the Linux environment. The subject materials in MSAL consist of three difficulty levels and two presentation styles; that is, six versions of subject materials have been implemented to achieve the adaptive tutoring feature.
The following figure shows the administrator interface.
During the learning process, MSAL will record a student’s study portfolio including Unit Learning Time, Idle Time, Response time, Unit Test Score, Unit Learning Efficiency, Absorbed, and Difficulty level. For each subject unit, a test is performed to decide the learning status of current unit and the feasible level of next unit for the student based on the student’s score and learning efficiency. The following figure shows the post-test interface.
All of the learning status and test results are recorded by MSAL. The following figure Create new user account
Display user account Remove user account Change system parameters Manage discussion board Access databases
depicts the Learning-Portfolio interface.
Unit title
Difficulty Level Start of learning time
End of learning time
4. Multi-Source Adaptive Learning Approach
During the learning process, MSAL records student characteristics, which can be used to determine individual learning style and subject materials for each student. In the following subsections, the parameters for describing student characteristics and the algorithms for determining the difficulty levels of subject materials for individual students are presented.
4.1 Adaptive Learning characteristic parameters
The characteristic parameters recorded by MSAL for determining learning styles of students and difficulty levels of subject materials are listed as follows:
SLT(Uj): Suggested Learning Time for Unit Uj given by the instructor.
SQP(Si): Sequential Processing Skill of Student Si. Process information sequentially or verbally; to readily derive meaning from information presented in a step-by-step, linear fashion text environment.
DS(Si): Discrimination Skill of Student Si. Visualize the important elements of a task, to focus attention on required details and avoid distractions.
AS(Si): Analytic Skill of Student Si. Identifying simple figures hidden in a complex field, use the critical element of a problem in a different way.
SS(Si): Spatial Skill of Student Si. Identify geometric shapes and rotate objects in the imagination; to recognize and construct objects in mental space.
ULT(Si,Uj): Unit Learning Time for Student Si to Unit Uj without taking idle time and test time into consideration.
AIT(Si,Uj): Acceptable Idle Time for Student Si in learning Unit Uj.
RST (Si,Uj): Response time when Student Si learns Unit Uj. MSAL will randomly pop-up a window and ask the student to respond.
UPT(Si,Uj): Unit Post-Test score for Student Si in learning Unit Uj.
EFU(Si,Uj): Unit Learning Efficiency for Student Si in learning Unit Uj.
EFU(Si,Uj)=SLT(Uj) / ULT(Si,Uj)
ABS(Si,Uj): Absorbed .The concentration for Student Si in learning Unit Uj.
CDU(Si,Uj): Course Difficulty level for Student Si in learning Unit Uj. Three versions of the subject materials with different difficulty levels are provided; that is, Primary, Secondary and Advanced levels.
LST(Si) : Learning Style of Student Si. Two kinds of learning styles, “Verbal” and
“Visual”, are taken into consideration.
When students enter MSAL for the first time, they are asked to take a learning style test based on the Keefe’s approach [10]. MSAL then determines CDU and LST based on the SQP, DS, AS and SS values and computes EFU and ABS based on ULT, AIT, RST, and UPT values.
4.2 Analysis of Learning styles
MSAL determines learning style, idle time and concentration degree of a student based on the Keefe learning style test. Four parameters are adopted in the test, including Sequential Processing Skill, Discrimination Skill, Analytic Skill and Spatial Skill.
Learning Style（LST）
If a student obtains a high score in Sequential Processing Skill (SQP), it implies that he learns well on Sequential Frame materials; otherwise, he tends to be suitable for Hypermedia presentation. The corresponding fuzzy rule is:
IF SQP(Si) = High then
LST(Si) = Sequential Frame Else
LST(Si) = Hypermedia End if
For example, assume students A and B get 0.8 and 0.4 for SQP, respectively, student A
will be arranged to receive Sequential Frame subject materials, and student B will receive Hypermedia subject materials. The membership functions of subject material style is given in the following figure.
SOP
0 0.25 0.5 0.75 1.0
LOW AVERAGE HIGH
1.0
Degree of membership
0.0 0.5
Acceptable Idle Time（AIT）
The initial Ideal Time is five minutes for each student. For students who get higher scores in Discrimination Skill, MSAL assumes that they are willing to take more time to read the course contents, and hence will accept a longer Idle Time during the learning process.
Consequently, for the students who obtained lower scores in Discrimination Skill, MSAL assumes that they are not as willing to spend time on reading subject contents, and will accept a shorter Idle Time during the learning process. Therefore, we have the acceptable Idle Time for Student Si to Unit Uj as
AIT(Si,Uj)= 5+5*DS(Si)
Assume that the learning style profile test score in DS for Students S1 and S2 are 0.8 and 0.4, respectively. We have AIT (S1,Uj) = 5 + 5 × 0.8 = 9 (minutes) and AIT (S2,Uj) = 5 + 5 × 0.4 = 7 (minutes).
Absorbed（ABS）
The initial Absorbed score for each student is set at 1. MSAL will start to count the time when the student does not perform any action. If the counted time exceeds the limit of Idle
Time, MSAL will pop-up a window and request a response. If the response time is over five seconds, which indicates that the student did not concentrate on browsing the subject contents;
therefore, MSAL will decrease the Acceptable Idle Time as well as the Absorbed degree. On the contrary, if the no-action time is very short (less than half of AIT), the Acceptable Idle Time and the Absorbed degree will be increased. The corresponding rules as given as follows:
IF the no-action time > AIT (Si,Uj) THEN Pop up the response window
IF RST (Si) > 5 and AIT (Si,Uj) > 10 seconds and ABS (Si,Uj) > 0.1 THEN AIT (Si,Uj) = AIT (Si,Uj) – 10 seconds
ABS (Si,Uj) = ABS (Si,Uj) –0.1 End IF
ELSE
IF the no-action time < AIT (Si,Uj)/2 and ABS (Si,Uj) < 1.0 THEN AIT (Si,Uj) = AIT (Si,Uj) + 10 seconds
ABS (Si,Uj) = ABS (Si,Uj) + 0.1 End IF
END IF
4.3 Determination of Difficulty Levels of Subject Materials
MSAL determines difficulty levels of subject materials based on student profile. When a student takes Mathematics course, MSAL will decide the difficulty level based on AS and SS values. When a student takes Science course, MSAL will determine difficulty level based on the student’s AS value. That is, different criteria will be employed in determining the difficulty levels for different courses.
Determine the Initial Difficulty Levels
MSAL will determine the initial difficulty level of subject materials based on the student profile. When a student takes Mathematics course, MSAL will decide the difficulty
level based on AS and SS ratings. Higher AS and SS ratings represent an enhanced ability when learning mathematics courses. If the student takes Science courses, MSAL will determine difficulty level based on AS rating. Higher AS rating represents an enhanced ability when learning Science courses. For Technique courses, MSAL will determine difficulty level based on SS rating. Higher SS rating implies an enhanced learning ability for Technique courses. That is, different criteria will be adopted to determine the difficulty levels of different courses. The fuzzy rules are shown as follows:
Case Mathematics courses
IF AS(Si) and SS(Si) = High then CDU(Si,Uj) = High
IF AS(Si) and SS(Si) = Low then CDU(Si,Uj) = Low
IF AS(Si) and SS(Si) is Average CDU(Si,Uj) = Average
Else
CDU(Si,Uj) = Average End if
Case Science courses IF AS(Si) = High then CDU(Si,Uj) = High IF AS(Si) = Low then CDU(Si,Uj) = Low IF AS(Si) is Average CDU(Si,Uj) = Average End if
Case Technique courses IF SS(Si) = High then CDU(Si,Uj) = High IF SS(Si) = Low then CDU(Si,Uj) = Low IF SS(Si) is Average
CDU(Si,Uj) = Average End if
Case Else
CDU(Si,Uj) = Average
Assume the learning style profile test indicates that the AS ratings of S1, S3 and S2 are 0.9, 0.8 and 0.2; and the SS ratings of S1, S2 and S3 are 0.8, 0.3 and 0.3, respectively. Based on the fuzzy rules, the difficulty levels of S1 are “High” for all courses, the difficulty levels of S2 are “Average” for mathematics courses, “High” for science courses, and “Low” for Technique courses, and the difficulty levels of S3 are “Low” for all courses.
Adapt the Difficulty Levels
If the post-test score exceeds 60, the student will be allowed to proceed to the next unit.
If both the learning efficiency and the post-test score of a student are very high, the difficulty level of next unit will be increased. On the contrary, if the learning efficiency and post-test score are too low, which implies the subject materials may be too difficult for the student, and hence the difficulty level will be decreased. If the learning efficiency and post-test score are within average range, the difficulty level remains the same. The membership functions of passing the user of next difficulty level are given in Figure 6, and the corresponding fuzzy rules are given as follows:
IF UPT(Si,Uj) AND EFU(Si,Uj) = High then IF CDU(Si,Uj) = High then CDU(Si,Uj) Remain unchanged
IF CDU(Si,Uj) = Average then CDU(Si,Uj) = High IF CDU(Si,Uj) = Low then CDU(Si,Uj) = Average End IF
IF UPT(Si,Uj) AND EFU(Si,Uj) = Low then IF CDU(Si,Uj) = High then
CDU(Si,Uj) = Average IF CDU(Si,Uj) = Average then
CDU(Si,Uj) = Low IF CDU(Si,Uj) = Low then
CDU(Si,Uj) Remain unchanged End IF
Else IF
CDU(Si,Uj+1) Remain Unchanged End IF
For example, if the learning efficiency and the post-test score of Student S1 are 1.5 and 90, respectively, the difficulty level will be increased; if the learning efficiency and the post-test score of Student S2 are 0.5 and 65, the difficulty level will be decreased; if the learning efficiency and the post-test score of Student S3 are 0.5 and 85, the difficulty level will remain the same.
0 0.5 1.0 1.5 2 -1 0 1
Low EFU
High EFU
Low UPT
High UPT
Decrease CDU
Keep CDU
Increase CDU Low
EFU
High EFU
Low UPT
High UPT
Decrease CDU
Keep CDU
Increase CDU
60 70 80 90 100
Figure 6: Membership functions to determine the difficulty level for the next phase
5. Experiments and Evaluation
To find out if the web environment with individual learning style and adaptive course contents would be helpful to the learning process, three questions need to be answered:
R1: Will a learning environment with multi-source adaptive subject materials achieve better learning efficacy for science courses than a learning environment without adaptive subject materials?
R2: Will a learning environment with single-source adaptive subject materials achieve better learning efficacy for science courses than a learning environment without adaptive subject materials?
R3: Will a learning environment with multi-source adaptive subject materials achieve better learning efficacy for science courses than a learning environment with single-source adaptive subject materials?
R4: Will a learning environment with multi-source adaptive subject materials achieve better learning efficiency for science courses than a learning environment with single-source adaptive subject materials?
5.1 Experiment Design
To evaluate the performance of our approach, an experiment has been conducted.
Ninety-one students of Sanyi junior high school in Taiwan, including thirty-seven females and fifty-four males with an average age of 15, participated in the web-based mathematics courses. All of the students were taught by the same teacher, and have been trained to use MSAL system.
Four units of “mathematics” course of junior high school were adopted to develop the experimental subject materials; that is, “Sequence”, “Equal Difference Sequence”, “Calculate Sequence” and “Sequence Mid Item”. Three difficulty levels (Easy, Mid and Hard) and two presentation styles (Hypermedia and Sequential Frame) were considered in developing the course contents; that is, six versions of subject materials were constructed, including “Easy and Hypermedia”, “Easy and Sequential Frames”, “Mid and Hypermedia”, “Mid and Sequential Frames”, “Hard and Hypermedia”, “Hard and Sequential Frames”.
The “Easy”, “Mid” and “Hard” versions of subject materials were constructed by considering the detail degree of content descriptions and the concepts to be learned. For
“Easy” version, the basic concepts to be learned with very detailed descriptions (including the
prerequisite concepts) were given; for “Mid” version, only detailed descriptions of the basic concepts and the most relevant prerequisite concepts (defined by the teacher) were given; for
“Hard” version, only the descriptions of the basic concepts and some advanced concepts (defined by the teacher) were given.
In the registration time, the students received the Keefe learning style profile test to exam their SQP values, and were assigned to three learning groups, i.e. Experimental Group1, Control Group 1 and Control Group 2. The following table shows the SQP grades of the members in each group.
SQP grade Experiment Group1 Control Group 1 Control Group 2 Number of High SQP
students 16 17 19
Number of Low SQP
students 13 15 11
Total number of students 29 32 30
Consequently, the students in the three research groups were asked to use different web-based learning strategies provided by MSAL. Experimental Group 1 adopted all of the six versions of subject materials, in which the learning environment was adaptive based on each student’s learning ability and learning style; Control Group 1 adopted three versions of subject materials; that is “Easy and Hypermedia”, “Mid and Hypermedia” and “Hard and Hypermedia”; Control Group 2 adopted only one version of subject material; that is, “Mid and Hypermedia”. For Experimental Group 1 and Control Group 1, the learning environment was adaptive based on each student’s learning ability, and difficulty level of a new subject unit was determined based on the test results of the previous unit. For Control Group 2, the test results of a unit were only used to measure the unit progression, and did not change the difficulty level of the next unit.
Note that although part of the students in Control groups 1 and 2 received high SQP values, we assigned Hypermedia presentation materials to all of the students in those two groups to simulate the ordinary web-based learning environments. That is, in ordinary
learning environments, without proper arrangement, part of the student might receive subject materials with improper presentation style. The experiment environment designed is shown in the following table.
Group name Experiment Group1 Control Group 1 Control Group 2 Course Multi-Source Adaptive
course
Single-Source Adaptive course
Non-Adaptive course Learning Style
Test
Yes Yes Yes
Pre test Yes Yes Yes
Subject material
Junior high school mathematics chapter 6.1
Junior high school mathematics chapter 6.1
Junior high school mathematics chapter 6.1 Versions of
Subject Materials
Six versions
(Easy/Mid/Hard and Hypermedia /Sequential Frames)
Three versions (Easy/Mid/Hard and Hypermedia)
One version
(Mid and Hypermedia)
Course name Equal difference Sequence 1
Equal difference Sequence 2
Equal difference Sequence 3
Course unit 1 Sequence Sequence Sequence Course unit 2 Equal difference
Sequence
Equal difference Sequence
Equal difference Sequence
Course unit 3 Calculate Sequence Calculate Sequence Calculate Sequence Course unit 4 Sequence mid item Sequence mid item Sequence mid item
Post-test Yes Yes Yes
5.2 Analysis of Leaning Efficacy and Efficiency
We aim to analyze the independent and interactive effects of three independent variables (i.e. Multi-Source Adaptive course, Single-Source Adaptive course, and Non-Adaptive course) on one dependent variable (Test score). For the 3 ×1 factorial experimental research designs, one-way ANOVA statistical procedure was employed to analyze all of the experimental results. Three pilot studies were carried out to verify the experiment design and prior to the investigation.
The following table shows the score means and standard deviations of the experimental results for each group. It can be observed that Experimental Group1 had the best average score, and Control Group2 had the lowest average score.
Group Average scoreNumber of studentsStandard deviation Mean
Experimental Group 1 73.45 29 24.62 85.00
Control Group 1 70.81 32 15.60 73.00
Control Group 2 61.20 30 21.51 61.00
Total number of students 68.48 91 21.19 72.00
The following table shows the analysis of score variance (ANOVA) for Experimental Group 1 and Control Group 2. Statistically the results show significant difference (F = 4.151, P = 0.046 < 0.5) of learning efficacy between the learning environment with Multi-Source Adaptive course and the Non-Adaptive learning environment, which implies that the recognition of individual learning styles and difficulty levels is necessary.
Sum of Squares D.F. Mean Squares F Ratio F Prob.
Between Groups 2212.163 1 2212.163 4.151 0.046 Within Groups 30379.972 57 532.982
Total 32592.136 58
The following table shows the analysis of variance for Control Groups 1 and 2.
Statistically the results show significant difference (F = 4.096, P = 0.047 < 0.5) of learning efficacy between the learning environment with Single-Source Adaptive course and the Non-Adaptive learning environment, which assumes acknowledgement of the difficulty levels of subject materials is necessary.
Sum of Squares D.F. Mean Squares F Ratio F Prob.
Between Groups 1430.712 1 1430.712 4.096 0.047 Within Groups 20955.675 60 349.261
Total 22386.387 61
The following table shows the analysis of variance for Experimental Group1 and Control Group1. The analysis results (F = 0.254, P = 0.616 > 0.5) show no significant difference of learning efficacy between the learning environments with Multi-Source Adaptive courses and Single-Source Adaptive course.
Sum of Squares D.F. Mean Squares F Ratio F Prob.
Between Groups 105.690 1 105.690 0.254 0.616 Within Groups 24510.047 59 415.425
Total 24615.738 60
However, in our experiment environment, there is nearly 47% (fifteen) of the students in Control Group 1 were with the “Low SQP” learning style; that is, those students have received the appropriate Hypermedia subject materials as the ones in Experimental Group 1.
Therefore, it might be interesting to focus the learning efficacy analysis on the seventeen
“High SQP” students in Control Group 1 with the sixteen “High SQP” students in Experimental Group 1 so that the effects of the single-source and the multi-source adaptive learning environments can be precisely compared.
The following table shows the t-test result of learning efficacy for the “High SQP”
students in Experimental Group 1 and Control Group 1. As there are only sixteen “High SQP” students in Experimental Group 1, we use the average score of those sixteen students to simulate the score of the seventeenth student. Based on the analysis result, we have p = 0.143
> 0.05, which implies that the difference of learning efficacy between the multi-source adaptive learning environment and the single-source adaptive learning environment is not significant.
N Mean SD t p
Experimental Group 1 17 75.24 14.45
Control Group 1 17 58.76 10.46 3.807 0.143
To more deeply compare the effects of the multi-source and the single-source adaptive learning environments, another comparison on subject material browsing time has been conducted. The following table shows the t-test results of learning efficiency for the “High SQP” students in Experimental Group 1 and Control Group 1. The analysis result (p = 0.018
< 0.05) implies that the difference of learning efficiency between the multi-source adaptive learning environment and the single-source adaptive learning environment is significant.
N Mean SD t p Experimental Group 1 17 420.88 97.69
Control Group 1 17 649.53 146.66 -5.350 0.018 Two observations can be derived from the above analysis results:
(1) Adaptive learning environments (including the multi-source and the single-source environments) can improve the learning efficacy of students in comparison with the non-adaptive learning environment.
(2) The multi-source adaptive learning environment can improve the learning efficiency of students in comparison with the single-source adaptive learning environment.
To sum up, we conclude that the multi-source adaptive learning environment is helpful to students in improving both learning efficacy and efficiency.
6. Conclusions
In this project, we propose an adaptive learning platform, MSAL, which takes multiple sources of personalization information into consideration, including individual differences and learning styles. MSAL is an intelligent tutoring environment that can assist instructors to develop web-based science courses and provide students with suitable subject materials to improve their learning performance. To evaluate the performance of our approach, an experiment was conducted to compare the performance of the students learning in three different environments. The one-way ANOVA and t-test were employed to analyze the experimental results. The statistical analysis results show that our approach can significantly improve student learning efficacy and efficiency.
Currently, we are trying to employ information techniques to analyze the various learning behavior of students and assemble more personalization parameters, in order to more precisely determine suitable subject materials for individual student.
References
[1] W.B. Barbe, and R. H. Swassing, “Teaching Through Modality Strengths”, New York, NY:
Zane-Bloser, Inc, 1979.
[2] T.P. Cavaiani, “Cognitive style and diagnostic skills of student programmers”, Journal of Research on Computing in Education, 22, pp.411-420, 1989.
[3] P. Brusilovsky, “Methods and Techniques of Adaptive Hypermedia”, in A. Kobsa, & J.
Vassileva (Eds.), Adaptive Hypermedia and Hypermedia, London: Kluwer Academic Publishers, pp. 1–43, 1998.
[4] R. Dunn, K. Dunn and G.E. Price, “Productivity Environmental Preference Survey”, Lawrence: KS, Price Systems Inc., 1993.
[5] A.F. Gregorc, & H. B. Ward, “Implications for learning and teaching: A new definition for individual”, NASSP Bulletin, 61, 20-26, 1977.
[6] G..J. Hwang, “A tutoring strategy supporting system for distance learning on computer networks”, IEEE Transactions on Education, Vol. 41, No. 4, pp. 343-351, 1998.
[7] G..J. Hwang, “On the development of a cooperative tutoring environment on computer networks”, IEEE Transactions on System, man and Cybernetic Part C, Vol. 32, No. 3, 2002, pp. 272-278, 2002.
[8] G..J. Hwang, “A conceptual map model for developing intelligent tutoring systems”, Computers & Education, Vol. 40. No. 3, pp. 217-235, 2003.
[9] J.W. Keefe, “Learning style: an overview”, NASSP's Student learning styles: Diagnosing and prescribing programs, pp. 1-17, 1979.
[10] J.W. Keefe, “Theory and practice”, National Association of Secondary School Principals, 1987.
[11] J.W. Keefe, “Profiling and utilizing learning style”, National Association of Secondary School Principals, 1988.
[12] J.W. Keefe, “Learning style: Cognitive and thinking skills”, National Association of
Secondary School Principals, 1991.
[13] F.S. Keller, “Goodbye, Teacher”, Journal of Applied Behavior Analysis, 1,799, 1968.
[14] F.S. Keller, “The Fred S. Keller School: CABAS at work”, The Current Repertoire, 10, 3,1994.
[15] H.J. Klausmeier, R.A. Rossmiller, , & M. Saily, (Eds.), “Individually guided education:
concepts and practices”, New York: Academic Press, 1977.
[16] D.A. Kolb, “Learning Style Inventory: Self-scoring inventory and interpretation booklet”, Boston: McBer & Company. 1985.
[17] T. Larkin-Hein, & D.D. Budny, “Research on learning style: applications in the physics and engineering classrooms Education”, IEEE Transactions, Volume: 44 Issue: 3, pp:
276 –281, Aug. 2001.
[18] J.J. Lo, H.M. Wang, S.W. Yeh, “Effects of confidence scores and remedial instruction on prepositions learning in adaptive hypermedia”, Computers & Education, 42, pp. 45–63, 2004.
[19] M. Martinez, & C.V. Bunderson, "Building interactive world wide web learning environments to match and support individual learning differences", Journal of Interactive Learning Research, Vol. 11, No. 2, pp. 131-162, 2000.
[20] J.J.G. Merrienboer, “Instructional strategies for teaching computer programming:
Interactions with the cognitive style reflection-impulsivity”, Journal of Research on Computing in Education 23, 1,pp. 45- 52, 1990.
[21] S. Montgomery, “Addressing diverse learning styles through the use of multimedia”, University of Michigan, 1996.
[22] A. Mumford, & P. Honey, “Using your learning styles”, Honey, 1996.
[23] R. Paolucci, “Hypermedia and learning: the relationship of cognitive style and knowledge structure”. In Proceedings of ED-MEDIA/ED-TELECOM 1998, Freiburg, Germany, 20–25, June.
[24] J.C. Reiff, “Learning styles”. Washington, DC: National Education Association, 1992.
[25] T. Russell, “Technology wars: winners and losers”, Educom Review, 32(2), pp. 44-46, 1997.
[26] R. Snow, & M. Farr, “Cognitive-conative-affective processes in aptitude, learning, and instruction: an introduction”, Conative and affective process analysis, Vol. 3, pp. l-10, 1987.
[27] G.K. Talmadge and J. W. Shearer, “Relationship among learning styles, instructional methods and the nature of leaning experiences”, Journal of Educational Psychology, 57, 222-230, 1969.
[28] H.A. Witkin, C.A. Moore, P.K. Oltman, D.R. Goodenough, F. Friedman, D.R. Owen and E. Raskin, “Role of the field-dependent and field- independent cognitive styles in academic evolution: a longitudinal study”, Journal of Educational Psychology, Vol. 69, No. 3, pp.197-211, 1977.
