Designing an adaptive web-based learning system based on students
’ cognitive
styles identi
fied online
Jia-Jiunn Lo
a,*, Ya-Chen Chan
a, Shiou-Wen Yeh
b aDepartment of Information Management, Chung-Hua University, 300, Taiwan, ROCbInstitute of Teaching English to Speakers of Other Languages, National Chiao Tung University, Taiwan, ROC
a r t i c l e i n f o
Article history:Received 14 June 2011 Received in revised form 11 August 2011 Accepted 12 August 2011
Keywords:
Distance education and telelearning Human–computer interface Intelligent tutoring systems Multimedia/hypermedia systems
a b s t r a c t
This study developed an adaptive web-based learning system focusing on students’ cognitive styles. The system is composed of a student model and an adaptation model. It collected students’ browsing behaviors to update the student model for unobtrusively identifying student cognitive styles through a multi-layer feed-forward neural network (MLFF). The MLFF was adopted because of its ability on imprecise or incompletely understood data, ability to generalize and learn from specific examples, ability to be quickly updated with extra parameters, and speed in execution making them ideal for real time applications. The system then adaptively recommended learning content presented with a variety of content and interactive components through the adaptation model based on the student cognitive style identified in the student model. The adaptive web interfaces were designed by investigating the rela-tionships between students’ cognitive styles and browsing patterns of content and interactive compo-nents. Training of the MLFF and an experiment were conducted to examine the accuracy of identifying students’ cognitive styles during browsing with the proposed MLFF and the impact of the proposed adaptive web-based system on students’ engagement in learning. The training results of the MLFF showed that the proposed system could identify students’ cognitive styles with high accuracy and the temporal effects should be considered while identifying students’ cognitive styles during browsing. Two factors, the acknowledgment of students’ cognitive styles while browsing and the existence of adaptive web interfaces, were used to assign three classes of college freshmen into three groups. The experimental results revealed that the proposed system could have significant impacts on temporal effects on students’ engagement in learning, not only for students with cognitive styles known before browsing, but also for students with cognitive styles identified during browsing. The results provide evidence of the effec-tiveness of the adaptive web-based learning system with students’ cognitive styles dynamically identi-fied during browsing, thus validating the research purposes of this study.
Ó 2011 Elsevier Ltd. All rights reserved.
1. Introduction
Web-based learning systems utilize hyperlink and multimedia technology to access various online resources. They provide students a high level of user control and rich materials corresponding to their learning needs (Lo, Wang, & Yeh, 2004). However, web-based learning has its limitations. AsBrown (1998)pointed out, many knowledge nodes of web-based learning systems have been constructed arbitrarily. Thus students may fail to effectively grasp important information due to having total freedom in browsing the web course. Furthermore, student’s individual differences play a central role in education including web-based learning (Graf & Kinshuk, 2010). The structure of the web-based courseware and the learning strategies are two important issues in web-based learning, and designers need to pay close attention to how web-based courseware is constructed in the curriculum as well as how students navigate through it (Brown, 1998).
People have different cognitive styles that influence how they organize and process information, influencing their learning performance (Graf, 2007; Stash, 2007). Coupling information presentation with students’ cognitive styles can increase the effectiveness and efficiency of learning and a web-based system must also include information about students’ cognitive styles to adapt optimally information to the
* Corresponding author. Tel.: þ886 3 5186045; fax: þ886 3 5186546.
E-mail addresses:jlo@chu.edu.tw(J.-J. Lo),m9510008@chu.edu.tw(Y.-C. Chan),shiouwen@mail.nctu.edu.tw(S.-W. Yeh).
Contents lists available atSciVerse ScienceDirect
Computers & Education
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / c o m p e d u
0360-1315/$– see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2011.08.018
student. But while developing adaptive web-based systems, less attention has been paid to the different ways that students perceive and process information in learning (Carver, Howard, & Lane, 1999; Dara-Abrams, 2002; Kinshuk & Graf, 2007; Stash, 2007). One of the obstacles might be the difficulty of identifying student cognitive styles online. Most previous research has collected information about psychological traits such as cognitive styles by having users complete evaluation questionnaires or psychometric tests (Stash, 2007). However, obtrusively asking users might not motivate them to answer questions and it might not be applicable in some contexts, particularly for infrequent users (Ghazarian & Noorhosseini, 2010). As a result, developing ways to effectively extract psychological information to identify students’ cognitive styles without disturbing students’ learning processes is one of the key concerns for incorporating students’ cognitive styles into adaptive web-based learning systems. Another obstacle to incorporating student cognitive styles into web-based learning systems is the presentation of adaptive learning materials in web-based environments. Designing adaptive web interfaces intuitively might not authen-tically reflect students’ information organization and processing preferences. But there have been few studies to investigate the relation-ships between students’ psychological traits and browsing behaviors (Stash, 2007). Therefore, designing adaptive web interfaces appropriate to students’ psychological traits would be another key concern.
Based on these concerns, the purpose of this study is two-fold. First, this study presents an adaptive web-based learning system focusing on students’ cognitive styles. In this system, a multi-layer feed-forward neural network (MLFF) was designed to identify students’ cognitive styles by observing their browsing behavior without asking them to answer any questions orfill out any forms. The MLFF was adopted because of its ability on imprecise or incompletely understood data, ability to generalize and learn from specific examples, ability to be quickly updated with extra parameters, and speed in execution making them ideal for real time applications. Then the system presented adaptive interfaces to students based on the identified cognitive style accordingly. These adaptive interfaces were designed by investigating the relationships between students’ cognitive styles and the ways students interacted with a web-based learning system. Second, training of the MLFF and an experiment were conducted to evaluate the effectiveness of this system. Two research questions were examined: the accuracy level of identifying student cognitive styles during browsing a web-based learning system with the proposed MLFF, and the impact of the proposed adaptive web-based system on students’ engagement in learning.
2. Literature review
2.1. Adaptive web-based learning systems
The use of web-based learning systems has been claimed to promote a higher level of comprehension development because it requires the association and linking of different ideas and information rather than simply recalling facts and data (Paolucci, 1998). But, creating a learning system that meets students’ requirements can be challenging since students learn with not only different needs but also different learning characteristics. The variation in students’ learning characteristics therefore is an important concern for crafting a web site presence. An effective web-based system creates an attractive presence that meets learning objectives such as attracting students to the system, making the system interesting enough for students to stay and explore, convincing students to follow the system’s links to obtain infor-mation, and creating a knowledge structure consistent with the system’s desired image. It is important to give students alternative ways to learn. However, many learning systems are static and passively present information to students. One critical aspect in developing a learning system is how to implement personalized services by understanding the student’s personal characteristics. How to present the learning material with respect to personal characteristics (personalization) therefore is one of the key issues for web-based learning systems.
It has been suggested that some students could experience disorientation and cognitive overload due to the huge quantity of information presented and its lack of organization (Brown, 1998). To effectively apply hypermedia to enhance learning,Paolucci (1998)suggested two types of issues, authoring-related and learning-related, need to be considered.Fischer (2001)mentioned that one fundamental problem of system design is how to write software for many students at authoring time while making it work as if designed for each individual student who is using it only at learning time. An adaptive web-based learning system provides personalized information by automatically improving the organization and presentation of the web page based on its learning from students’ browsing patterns to achieve personalized services (Perkowitz & Etzioni, 2002). If the system is constantly adapting or is being adapted to students, learning time becomes a different kind of authoring time (Fischer, 2001) so that it can adapt the web content accessed by a particular student according to a user model based on individual characteristics and educational methodology for web-based learning systems (Dara-Abrams, 2002). According toBrusilovsky (1998; 2001), an adaptive web-based system can be useful in any application area where the system is expected to be used by people with different goals or characteristics and where the hyperspace is reasonably big. From the user’s point of view, an adaptive web-based system can help a user obtain information in a form thatfits the user’s characteristics and fulfills the user’s real needs automatically. It can also help a user avoid the problems of information overload, disorientation, cognitive overload, discontinuousflow, content unreadi-ness, and lack of comprehension (Chen & Ford, 1997; Murray, Shen, Piemonte, Condit, & Thibedeau, 2000). On the other hand, from the information provider’s point of view, the adaptive web-based system is helpful to deliver information to users more efficiently and effectively.
Most adaptive web-based systems focus on adapting to user features such as goals/tasks, knowledge, background, hyperspace experi-ence, preferexperi-ence, and interests (Brusilovsky, 2001; Stash, 2007). In addition to these features, students’ psychological traits are important factors affecting their learning. Therefore, including students’ psychological traits in web interface presentation is important for learning. The psychological trait discussed in this study is students’ cognitive styles. People organize and process information through a variety of different cognitive styles. The cognitive style models propose that the behavior of an individual is likely to be affected by an interaction between cognitive styles and the way the information is structured, the mode of presentation, and the type of content. They can indicate an individual’s preferences for different types of information or different ways of navigating through or interacting with the information space (Lipsky, 1984; Stash, 2007). Given a specific information form or environment, some people will acquire the information more effectively than others due to their distinct cognitive styles.
Researchers have developed a variety of cognitive style models. The cognitive style model applied in this study was the Myers-Briggs Type Indicator which is based on Jung’s theory of cognitive styles. It was selected according to the criteria suggested byLipsky (1984): based on a two-dimensional model of information processing, sufficient validity and reliability data, and appropriateness for use with
adults. This instrument has been widely applied for adults with sufficient validity and reliability data (Silver, Strong, & Perini, 2000). It is based on two fundamental cognitive functions, perception (how peoplefind out or absorb information) and judgment (how people process or make judgment about the absorbed information). Based on perception and judgment, the four cognitive styles of Mastery, Understanding, Self-expressive, and Interpersonal, have been discussed (Fig. 1). A style is a basic orientation toward the world based upon functional preferences (sensing vs. intuition, thinking vs. feeling) characterized by particular interests, habits of mind, and personal behaviors.
An increasing amount of research focusing on the impact of students’ characteristics has been conducted inweb-based learning (Kinshuk & Graf, 2007). Usually, psychological information is collected by asking students to complete evaluation questionnaires before the learning processes. However, obtrusively asking students might make them feel uncomfortable and it might not be applicable in some contexts. How to effectively extract psychological information to build user model without disturbing students’ learning processes therefore would be one of the key concerns for incorporating student cognitive styles into adaptive web-based learning systems and it is one of the goals of this study.
2.2. Collecting user data and recognizing user psychological traits
Adaptation decisions in adaptive web-based systems are based on the user’s characteristics represented in the user model. To provide adaptive interfaces in web-based learning systems, it is crucial to identify students’ behavior and learning preferences (Bousbia, Rebaï, Labat, & Balla, 2010). Collecting data about students to determine their characteristics, thereby is an important factor for developing adaptive learning systems. The methods for obtaining data about users include explicit and implicit approaches (Brusilovsky, 2001; Hanani, Shapira, & Shoval, 2001). Explicit approaches ask users to input their own data by requiring them to answer questions to describe their interests or other relevant parameters (Hanani et al., 2001). However, users are usually not motivated to answer questions and they are often reluctant to provide personal data (Joerding, 1999; Kobsa, 2002). Furthermore, asking users to answer questions may interrupt the browsing processes (Koychev & Schwab, 2000; Lieberman, 1995; Lo & Lin, 2006). Many users are unwilling to be interrupted during browsing and to feedback any information to a web-based system (Joerding, 1999; Kobsa, 2002; Koychev & Schwab, 2000). Therefore, directly asking for user data to build/update the user model may not be appropriate in some applications. Instead of asking users to input their own data, implicit approaches record the user’s reaction to each incoming data item to automatically infer the user model (Hanani et al., 2001). As there is more concern for the user privacy and users are less willing to provide private data, processing with implicit user data is becoming important. To enhance adaptivity, many systems try to identify users to retain their long-term data. For this purpose, many systems embed mechanisms, such as cookies and tracking software agents, in the user’s browser and/or record user IP address so that the system can identify the user as he/she connects to the system later. However, this may conflict with privacy concerns of users (Kobsa, 2002). Thus, to protect user privacy, building a short-term user model without using user data obtained from multiple sessions is important for adaptive web-based systems (Joerding, 1999) and is one of the goals of this study.
Usually, explicit approaches are used to collect psychological information by asking students to complete evaluation questionnaires before the learning processes (Stash, 2007). However, students might not be willing to complete an evaluation and it might not be applicable in some contexts. It has been suggested that browsing behavior offers the potential to extract some psychological information about the student, which may in turn be used to tailor the web interfaces toward the student (Graf, 2007; Lo & Shu, 2005; Mullier, 1999; Yeh & Lo, 2005). Therefore, recognizing students’ cognitive styles by analyzing implicitly collected students’ browsing behaviors is a possible alter-native to identify students’ cognitive styles. The task of identifying individual characteristics including students’ cognitive styles is similar to character recognition in that they both involve the classification of features from a potentially infinite number of possible inputs (Castellano, Fanelli, & Roselli, 2001; Lo & Shu, 2005; Mullier, 1999). Some researchers have suggested that neural networks could perform well in classification because of (1) their pattern recognition ability on imprecise or incompletely understood data, (2) their ability to generalize and learn from specific examples, (3) their ability to be quickly updated with extra parameters, and (4) their speed in execution making them ideal for real time applications (Chiu, Norcio, & Petrucci, 1991; Lo & Shu, 2005; Masters, 1993; Mullier, 1999).
Zatarain, Barron-Estrada, Reyes-Garcfa, and Reyes-Galavia (2011)proposed a set of three different neural network approaches to identify students’ learning styles for adaptive learning schemes in web-based and mobile environments. They used a neural-fuzzy network to select the best learning style, a neural-fuzzy network to classify learning styles with a genetic algorithm for training, and a self-organizing feature map (SOM) to identify students’ learning styles.Lo and Shu (2005)developed an MLFF to identify students’ learning styles by observing their browsing behaviors, including the selection of link types, the use of embedded support devices (ESDs), and visited/unvisited nodes. Their model performed well in identifying the students’ learning styles of Visual, Auditory, and Kinesthetic. In addition to identifying students’ learning styles, the MLFF model has also been successfully adopted to assess students’ metacognitive knowledge level by observing their online browsing behaviors (Yeh & Lo, 2005). In that study, similar toLo and Shu (2005), browsing behavior included the use of ESDs and navigation between visited/unvisited web pages. That study also verified the suitability of the proposed model and showed no significant differences of assessment accuracy with respect to the web page structures.
Intuitive Feeling Thinking (Perception) (Judgment) Interpersonal Sensing Mastery Understanding Self-expressive
The achievements of previous studies in web-based learning systems are the basis for this study to adopt the MLFF for identifying students’ cognitive styles without asking them to complete evaluation questionnaires. When students’ cognitive styles have been identified, an adaptive web-based learning system should be able to present the adaptive interfaces to students accordingly.
3. System architecture
The proposed adaptive web-based learning system included two submodels, the student model and the adaptation model. The system collected students’ browsing behaviors to update the student model for identifying students’ cognitive styles and adaptively recommended learning content presented with a variety of content and interactive components through the adaptation model based on the cognitive style identified in the student model. A content and interactive component is an object which would be usable, sensible, and of practical value to the information processing process for students to access content and interact with others such as table of contents, opening case, discussion forum, etc (Lo & Chan, 2008, pp. 51–57).
3.1. Student model: identifying students’ cognitive styles
This study included a multi-layer feed-forward neural network (MLFF) into the student model to identify students’ cognitive styles based on their browsing behaviors. The MLFF consists of three layers: input layer, hidden layer, and output layer (Fig. 2).
Researchers have suggested that it may be possible to extract some psychological information about users from their browsing behaviors and the utilization of neural networks is a possible means to recognize individual characteristics (Graf, 2007; Mullier, 1999; Yeh & Lo, 2005). A robust neural network structure design should correspond to the web-based courseware structure and should not need to change as the web page structure changes (Lo & Shu, 2005). In this MLFF, the input layer is used to reflect users’ browsing behaviors represented by students’ responses about various types of content and interactive components and content link types. Four input factors were included and each corresponds to one type of input node. Thefirst type of input nodes represents the browsing behavior of students by recording their selection frequencies of content and interactive components as illustrated as Equation (1). In Equation(1), n is the number of content and interactive components used in the web-based learning system. Each of the second type of input nodes represents a student’s behavior for the selection ratio of one content and interactive component. The selection ratios of content and interactive components are illustrated as Equation(2). The third type of input nodes reflects the temporal effect, which is defined as the average staying time on content and interactive components as illustrated in Equation(3). Thefirst three factors could be used to reflect students’ preferences for content and interactive components with respect to different cognitive styles. The fourth type of input nodes represents the browsing behavior of students of selecting course content link types. It could be used to reflect students’ preferences of web page content structure and is represented by the ratios of the selected content link types as illustrated in Equation(4). In Equation(4), m is the number of content link types used in the web-based learning system. In addition to these four types of input nodes, there is also afifth type of input node, the bias node, which is used to provide afixed level of activation on the output either above or below the activation provided by the input nodes. It can be used to modify the threshold level of activation required to excite the output node (Lo & Shu, 2005; Yeh & Lo, 2005).
Selection frequency of component i ¼ Frequency of students’ clicks on component i; i ¼ 1; n (1)
Selection ratio of component i¼ Frequency of students’ clicks on component i=
Frequency of students’ clicks on all components;i ¼ 1;n ð2Þ
Average staying time of component i ¼ Total Staying time of component i=Frequency of students’ clicks on component i; i ¼ n (3) Selection ratio of content link type j ¼ Frequency of students’ clicks on content link type j=
Frequency of students’ clicks on all content link types; j ¼ 1; m ð4Þ
Selection frequency of content andinteractive components
Bias
Average staying time on content and interactive components Selection ratio of content and interactive
components Mastery
Understanding Self-expressive Interpersonal
Input layer Hidden layer Output layer
(Cognitive styles) Selection ratio of
content link types
The output layer is used to represent various types of features to be recognized in an MLFF and the number of output nodes is directly driven by the problem. In this study, four distinct cognitive styles, Mastery, Understanding, Self-expressive, and Interpersonal, are to be identified. As a consequence, four output nodes are included in this MLFF. Each output node corresponds to one cognitive style and a student’s cognitive style is determined as the one corresponding to the output node having the maximum value obtained from the MLFF. The processing power of an MLFF is provided by the hidden layer, and the neural network’s ability to learn is affected by the number of hidden nodes (Lo & Shu, 2005; Masters, 1993). As suggested byYeh (2001), for categorization applications, the number of hidden nodes is set to be the average of input and output nodes.
3.2. Adaption model: presenting web interfaces according to students’ cognitive styles
In addition to identify students’ cognitive styles, another obstacle to incorporating student psychological traits into adaptive web-based systems is the adaptive presentation of learning materials in web-based environments. In this study, adaptive web interfaces for students with different cognitive styles were designed by investigating the relationships between their cognitive styles and the way they interact with a web-based learning system (Lo & Chan, 2011).
In this study, an experimental web-based learning system withfifteen content and interactive components (Table 1) was built (Lo & Chan, 2008, pp. 51–57). Unlike the other components that are separated from the text and represented as “buttons”, Keyword and Graphic are two types of in-text hyperlinks embedded within the learning material.Fig. 3illustrates the interface of the experimental web-based learning system with the content and interactive components at the top and the content links at the left of the window. For better descriptions of the system, English explanations of the content and interactive components on the screenshots are provided in white labels with component description as listed inTable 1.
In addition to thefifteen content and interactive components, three content link types were defined in the courseware structure: C, E, and X. The C-type link was the mainstream of the learning path defined by the courseware designers, among different instructional topics. E-type links (Explanatory) were links directing the learning path to the concept nodes for providing explanations of a concept node in the mainstream with C-type links. Rather than providing explanations, X-type links (eXtensive) were used to provide extensive knowledge of mainstream concept nodes with C-type links (Liu & Lin, 1999; Lo & Shu, 2005).
One hundred and seventy six college students who were enrolled in the“Introduction to Information Management” course in a northern Taiwan university were selected to participate in a study investigating the relationships between students’ cognitive styles and browsing behaviors. Before the experiment, participants were asked to complete a cognitive style evaluation questionnaire fromSilver et al. (2000)to identify their cognitive styles. Then they were asked to browse the experimental web-based learning system for 30 min in the computer lab. The topic of the learning content was“Protecting People & Information: Threats and Safeguards”. At the experimental time, the learning content had not been introduced in the class and participants had not read the content before the experiment. Participants’ browsing behaviors were collected in the logfile by the system for analysis.
After deleting fourteen participants’ abnormal browsing data, which included students’ links to outside web resources from the beginning of the experiment, browsing data from one hundred and sixty two participants was analyzed.Table 2lists the distribution of students with different cognitive styles for investigating the relationships between students’ cognitive styles and browsing behaviors. Most of the participants had Interpersonal style (51.23%), with Self-expressive style a distant second (25.31%). Participants with Understanding and Mastery styles were approximately equal (12.35% and 11.11%).
Students’ selection behaviors for content and interactive components could be used to reflect their preference for these components with respect to different cognitive styles (Lo & Chan, 2008, pp. 51–57). Therefore, the web interfaces for students with different cognitive styles
Table 1
Content and interactive components in the experimental system.
No. Component Description
1 Pretest Tests to assess students’ understanding about the prerequisite
knowledge of the learning content
2 Table of contents List of the learning content
3 Introduction Overview of the learning content
4 Learning objectives Outlines of what student can learn from the system
5 Opening case Practical applications provided before studying
the learning content
6 Closing case Practical applications provided after studying the
learning content
7 Chapter review Summary of the learning content in accordance with
the learning objectives
8 Glossary List of important and related terms of the content
with their definitions
9 Reference material Extended readings related to the learning content
10 Popular current events Related practical attention-getting examples
happened recently
11 Self evaluation Tests to assess the learning effectiveness on
students’ new knowledge
12 Discussion forum Component allowing students to seek peer interactions
13 E-mail to teacher A channel for students to communicate with the teacher
14 Keyword In-text hyperlinked texts embedded within the learning
content to explain key terms
15 Graphic In-text hyperlinked graphics embedded within the learning
would be designed based on the investigation of students’ selection behaviors for content and interactive components. Two types of browsing data were investigated: selection ratios of content and interactive components and average staying time on content and inter-active components. They were calculated with Equations(5) and (6).Figs. 4 and 5illustrate the distributions of students’ selection behaviors for content and interactive components according to different cognitive styles (Lo & Chan, 2008, pp. 51–57). Please refer toTable 1for component description.
Selection ratio of component i ¼ Frequency of students’ clicks on component i=
Frequency of students’ clicks on all component; i ¼ 1; n ð5Þ
Average staying time of component i ¼ Total Staying time of component i=Frequency of students’ clicks on component i; i ¼ 1; n (6)
The results show that selection ratios and average staying time had similar trends. For most components, if the selection ratio is high, there is also long staying time. Conversely, if the selection ratio is low, the staying time is short. This trend shows that if students like a component more than students of other styles do, they not only select it more frequently but also stay at it longer. Furthermore, the results also reveal that there were significant different preferences in selecting components among students of different cognitive styles (Lo & Chan, 2008, pp. 51–57) which could be, in turn, applied to design adaptive web interfaces for different cognitive styles.Table 3
summarizes students’ preferences for content and interactive components (Lo & Chan, 2011). Based on the preferences, adaptive web
Fig. 3. Screenshot of the experimental web site (No adaptive presentation).
Table 2
Distribution of students for investigating the relationships between students’ cognitive styles and browsing behaviors.
Style Number Ratio
Interpersonal 83 51.23% Mastery 18 11.11% Understanding 20 12.35% Self-expressive 41 25.31% Total 162 100% 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (Component)
Interpersonal Mastery Understanding Self-Expressive
interfaces for different cognitive style students were developed (Appendix). In the web interfaces for each cognitive style, students’ preferred content and interactive components were shown in the web page and the buttons of preferred components were shaded at the top of the window.
4. Evaluations
Training of the proposed MLFF and an experiment were conducted to examine the following research questions:
RQ1: What is the accuracy of identifying students’ cognitive styles while browsing a web-based learning system with the proposed MLFF?
RQ2: What is the impact of the proposed adaptive web-based system on students’ engagement in learning?
4.1. Evaluation of identifying student cognitive styles with MLFF
Training of the proposed MLFF was conducted to examine RQ1: what is the accuracy of identifying students’ cognitive styles while browsing a web-based learning system with the proposed MLFF?
4.1.1. Training settings of the MLFF
The data collected for investigating the relationships between students’ cognitive styles and browsing behaviors as discussed in Section
3.2[Adaption Model: Presenting Web Interfaces According to Students’ Cognitive Styles] was used to train the proposed MLFF. Since the experimental learning system includedfifteen content and interactive components and three content link types, in the proposed MLFF there are 49 input nodes (15 for selection frequencies of components, 15 for selection ratios of components, 15 for average staying time on components, 3 for selection ratios of content link types, and 1 for bias node) and 4 output nodes (the four cognitive styles to be identified). Training of an MLFF is part of supervised learning category, which needs training data consisting of pairs of inputs and desired outputs. The input data, students’ browsing behaviors, were obtained by analyzing the system log file. The output data, students’ cognitive styles were obtained by asking students to complete the cognitive style evaluation instrument included inSilver et al. (2000). One hundred and sixty two valid records were used for training. From those records, 114 records, were randomly selected as the training data and the remaining 48 records served as the testing data.Table 4lists the distribution of evaluation data for different cognitive styles.
4.1.2. Results of training the MLFF
The proposed MLFF was trained with different randomly selected training data and testing data sets. The weight sets of the training result with the highest accuracy ratios would be used for the following study. The accuracy ratio was defined as the number of accurate
0 20 40 60 80 100 120 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (Component) (Second)
Interpersonal Mastery Understanding Self-Expressive
Fig. 5. Distribution of average staying time on components (Lo & Chan, 2008).
Table 3
Preferences for components (Lo & Chan, 2011).
Style Preferred components
Interpersonal Pretest, Opening case, Closing case, Glossary, Discussion forum
Mastery Learning objectives, Opening Case, Closing case, Chapter review, Graphic
Understanding Pretest, Table of content, Opening case, Self evaluation, Keyword, Graphic
Self-expressive Pretest, Introduction, Opening case, Glossary, Discussion forum
Table 4
Distribution of evaluation data for different cognitive styles.
Style Number Ratio Number of training Number of testing
Interpersonal 83 51.23% 58 25
Mastery 18 11.11% 13 5
Understanding 20 12.35% 14 6
Self-expressive 41 25.31% 29 12
testing records divided by the number of total testing records. A testing record was said to be accurate if the cognitive style with the maximum value obtained from the MLFF was the same as the cognitive style with the maximum value obtained from the cognitive style evaluation questionnaire. The accuracy ratios were calculated every 5 min within the browsing session (Table 5).Table 5indicates that the temporal effects should be considered while identifying student cognitive styles during browsing. The highest accuracy ratio did not occurred when using the whole browsing session data (30 min). Within 25 min, the average accuracy ratios improved as longer browsing session data was used, after which, the accuracy ratios decreased. Moreover, the highest accuracy ratios occurred at different browsing intervals among the styles. The highest accuracy ratios for Interpersonal and Mastery styles occurred when using students’ browsing data during 0–20 min and for Understanding and Self-expressive styles the highest accuracy ratios occurred when using students’ browsing data during 0–25 min.
4.2. Evaluation of the adaptive web-based learning system
An experiment was conducted to examine RQ2: What is the impact of the proposed adaptive web-based system on students’ engage-ment in learning?
4.2.1. Experimental settings
Three classes of college freshmen from the College of Computer Science and Informatics at the same university as students participated in the training of the MLFF participated in this experiment to evaluate the adaptive web-based learning system. They have not studied the learning content introduced in the experimental system before the experiment. Two factors, the acknowledgment of students’ cognitive styles while browsing and the existence of adaptive web interfaces, were used to assign these three classes into three groups (Table 6). There werefifty eight students in the control group, forty nine students in the experimental group A, and sixty three students in the experimental group B. For the control group, students’ cognitive styles were unknown before and during the browsing and no adaptive web interfaces were provided. The conventional web interface (Fig. 3) was presented. For the experimental group A, students’ cognitive styles were determined before browsing by completing the cognitive style evaluation questionnaire and the web interfaces adapted to their styles (Figs. A.1–A.8) were presented from the beginning. For the experimental group B, students’ cognitive styles were unknown before browsing and the conventional web interface (Fig. 3) was presented in the beginning. Then as the student’s browsing proceeded and their cognitive styles
Table 5
The accuracy ratios.
Style Browsing data
0–5’ 0–10’ 0–15’ 0–20’ 0–25’ 0–30’ Interpersonal 64% 72% 76% 88%a 84% 84% Mastery 40% 60% 60% 80%a 80%a 80%a Understanding 33% 50% 67% 67% 83%a 83%a Self-expressive 42% 58% 67% 83% 92%a 75% Average 45% 60% 68% 80% 85%a 81%
aThe highest accuracy ratio.
Table 6
Treatment of evaluation of the adaptive web-based learning system.
Group User cognitive style Adaptive interfaces Number
Control group Before: Unknown No 58
During: Unknown
Experimental group A Before: Known Yes 49
During: Known
Experimental group B Before: Unknown Yes 63
During: Identified with MLFF
0 25 50 75 100 125 150 175 200 225 250 275 300 0~5 5~10 10~15 15~20 20~25 Minutes Interpersonal Master Understanding
Self-Average staying time of web pages
were identified with the proposed MLFF, the adaptive web interfaces (Figs. A.1–A.8) were presented according to the identified cognitive style. If one different cognitive style was identified in the next step, the interface would transit to fit the newly identified style. Students in all the three groups were asked to browse the experimental web-based learning system for 25 min. The average staying time of web pages, calculated from the logfile, was used to evaluate the system.
4.2.2. Experimental results
The average staying time on web pages was used to evaluate the system by representing the students’ degree of engagement with the learning content (Lo & Chan, 2011). In this study, every 5 min, the average staying time of web pages was calculated as Equation(7).
Average staying time of web pages ¼ Total staying time of web pages=Total number of visited web pages (7)
Figs. 6–8show the average staying time on web pages for different cognitive style students during the experiment. InFig. 6, for the control group using the conventional system, from 0 to 20 min, the average staying time on web pages was within a range approximately of 25 s–100 s and there were no significant differences in the average staying time among the four styles. However, during the last 5 min (20– 25 min), while Understanding and Self-expressive students still had approximately the same staying time, Interpersonal and Mastery students stayed much longer than former 20 min. This was especially true for Interpersonal students, who almost stopped navigating web pages during the last 5 min (the average staying time was about 225 s, close to 5 min). This implies that Interpersonal and Mastery students paid close attention on the learning material for former 20 min and lost patience afterward which might be resulted from the interface not matching with their cognitive styles. This is the limitation of conventional web-based systems and the problem to be solved in this study. The durations of students’ engagement in browsing the web pages coincided with the durations of browsing data for training the MLFF for different styles. The temporal effect of students’ engagement might be the reason behind the training results of the MLFF: The highest accuracy ratios of identifying student cognitive styles for Interpersonal and Mastery styles occurred when using students’ browsing data during 0–20 min and for Understanding and Self-expressive styles the highest accuracy ratios occurred when using students’ browsing data during 0–25 min.
On the other hand,Fig. 7for the experimental group A using the adaptive web-based learning system with cognitive styles known before browsing, shows that during the whole experiment (0–25 min), the average staying time on web pages remained stable within a range of approximately 50 s–125 s and there were no significant differences in the average staying time among the four styles. Though not as stable as the experimental group A for Interpersonal students,Fig. 8for the experimental group B using the adaptive web-based learning system with cognitive styles identified during browsing with the proposed MLFF, shows that during the whole experiment (0–25 min), like the experimental group A, the average staying time on web pages remained stable and there were no significant differences in the average staying time among the four styles.
0 25 50 75 100 125 150 175 200 225 250 275 300 0~5 5~10 10~15 15~20 20~25 Minutes Interpersonal Master Understanding
Self-Average staying time of web pages
Fig. 7. Average staying time on web pages for the experimental group A.
0 25 50 75 100 125 150 175 200 225 250 275 300 0~5 5~10 10~15 15~20 20~25
Interpersonal
Master
Understanding
Self-Minutes
5. Conclusions
Though it is recognized that incorporating students’ cognitive styles into web-based system design can be beneficial to learning, cognitive styles are usually ignored when designing adaptive web-based learning systems. This might be resulted from the difficulties of identifying student cognitive styles online and presenting adaptive interfaces in web-based environments. Most research has collected information about cognitive styles by having students complete evaluation questionnaires. However, this can make students feel uncom-fortable and is not applicable in practical contexts. Therefore, this study developed an adaptive web-based learning system focusing on students’ cognitive styles with a mechanism to unobtrusively identify students’ cognitive styles.
The cognitive style instrument applied in this study was the Myers-Briggs Type Indicator, which is based on Jung’s theory of cognitive styles. It was selected according to the criteria suggested byLipsky (1984): based on a two-dimensional model of information processing, sufficient validity and reliability data, and appropriate for use with adults. This instrument has been widely applied for adults with sufficient validity and reliability data (Silver et al., 2000). It is based on two fundamental cognitive functions, perception (how peoplefind out or absorb information) and judgment (how people process or make judgment about the absorbed information). From these two functions, four cognitive styles, Mastery, Understanding, Self-expressive, and Interpersonal, are discussed.
The proposed system is composed of two submodels, the student model and the adaptation model. The student model identified students’ cognitive styles based on their browsing behaviors through a multi-layer feed-forward neural network (MLFF) without asking students to complete an evaluation questionnaire so that students would not be bothered while learning. The adaptation model then presented adaptive web interfaces with a variety of content and interactive components based on the cognitive style identified in the user model. The adaptive web interfaces were designed by investigating the relationships between students’ cognitive styles and browsing patterns of content and interactive components (Lo & Chan, 2011).
Training of the MLFF and an experiment were conducted to examine the accuracy level of identifying students’ cognitive styles during browsing a web-based learning system with the proposed MLFF and the impact of the proposed adaptive web-based system on students’ engagement in learning. The training results of the proposed MLFF showed an acceptable average highest accuracy ratio of 85% for the four cognitive styles, Mastery, Understanding, Self-expressive, and Interpersonal. Moreover, the results revealed that temporal effects should be considered while identifying students’ cognitive styles during browsing. First, it is not always better to include as much data as possible to train the MLFF since the highest accuracy ratio did not occur when using the whole browsing session data. Within 25 min, the accuracy ratios improved as longer browsing session data was used and the accuracy ratios then decreased. Second, the highest accuracy ratios occurred at different browsing intervals among styles. The highest accuracy ratios for identifying Interpersonal and Mastery styles occurred earlier than identifying Understanding and Self-expressive styles (0–20 min vs. 0–25 min). A possible reason for this is that the students might begin losing patience after browsing the web site for a period of time. The temporal effects on learning were consistent with the results of students’ engagement in learning obtained from the experiment to evaluate students’ engagement. The results verified the potential benefits of MLFF in identifying students’ cognitive styles during browsing in web-based learning applications. This reinforces the potential of extracting psychological information about users from their browsing behavior (Graf, 2007; Lo & Shu, 2005; Mullier, 1999; Yeh & Lo, 2005).
Two factors, the acknowledgment of students’ cognitive styles while browsing and the existence of adaptive web interfaces, were used to assign three classes of college freshmen into three groups. The experimental results revealed that the adaptive web-based learning system could significantly affect temporal effects on students’ engagement in learning. For students using the conventional system, though there were no significant differences about the average staying time for the four styles from the beginning, however Interpersonal and Mastery students stayed much longer during the last 5 min (20–25 min). This might be due to the “sensing” of the cognitive function “perception”. Interpersonal and Mastery styles both rely on sensing as a mode of perception. Perception indicates how people find out or absorb information hence affect students’ patience on browsing web pages, which is the process of becoming aware of information (Silver et al., 2000). That sensing people prefer to be made aware of things directly from thefive senses might be the reason that Interpersonal and Mastery students lost their patience more quickly as they browsed the same web pages. Thisfinding is consistent with the results of training the MLFF, as shown inTable 5: The highest accuracy ratios for Interpersonal and Mastery styles occurred during 0–20 min and for Understanding and Self-expressive styles the highest accuracy ratios occurred during 0–25 min. On the other hand, for students using the adaptive web-based learning system either with cognitive styles known before browsing or with cognitive styles identified during browsing with the proposed MLFF, the average staying time on web pages remained stable and there were no significant differences in the average staying time among the four styles from the beginning to the end of the experiment. This demonstrated that the adaptive web-based learning system based on students’ cognitive styles could effectively enhance students’ engagement in learning for Interpersonal and Mastery styles especially.
Based on the researchfindings, we may conclude that the MLFF has the potential to identify students’ psychology traits such as cognitive styles without asking them to complete any evaluation questionnaires, and that the adaptive web-based learning system incorporating with students’ cognitive styles can meet the different information organization and process preferences of students. The proposed adaptive web-based learning system can create more attractive interfaces than do conventional systems to enhance students’ engagement in learning. This is possible not only for students with cognitive styles known before browsing but also for students with cognitive styles identified during browsing. The results provide evidence for the effectiveness of the adaptive web-based learning system with dynamically identifying students’ cognitive styles during browsing, thus validating the research purposes of this study.
Two limitations of this study should be considered. Thefirst limitation regards the adoption of MLFF for identifying students’ cognitive styles. Though MLFF could perform well in classification, there are other approaches such as rule-based systems and statistical systems. Extended studies to compare different approaches could be considered in the future. The second limitation concerns the evaluation of the adaptive web-based system. In this study, the impact of the proposed adaptive web-based system on students’ engagement in learning was evaluated. However, learning performance was not included as an evaluation criterion. This was because the three classes participated in the experiment were taught by different instructors so that we cannot prevent the influence of instructors. Since learning performance eval-uation is crucial for designing educational systems, further experiments involving classes taught by the same instructor can be conducted to evaluate students’ learning performance to enhance the credibility of the proposed model.
Acknowledgments
We gratefully acknowledge the research support of National Science Council of Taiwan, R.O.C. (NSC 94-2213-E-216-006; NSC 93-2213-E-216-009) and Chung-Hua University (CHU-NSC 94-2213-E-216-006; CHU-NSC 93-2213-E-93-2213-E-216-009). We would also like to thank the anonymous reviewers for insightful comments on this paper.
Appendix
Fig. A.1. Screenshot of the homepage for interpersonal style.
Fig. A.3. Screenshot of the homepage for mastery style.
Fig. A.4. Screenshot of the learning material for mastery style.
Fig. A.6. Screenshot of the learning material for understanding style.
Fig. A.7. Screenshot of the homepage for self-expressive style.
References
Bousbia, N., Rebaï, I., Labat, J.-M., & Balla, A. (2010). Learners’ navigation behavior identification based on trace analysis. User Modeling and User-Adapted Interface, 20(5), 455–494.
Brown, I. (1998). The effect of WWW document structure on students’ information retrieval. Journal of Interactive Media in Education, 98(12), 1–18.
Brusilovsky, P. (1998). Methods and techniques of adaptive hypermedia. In P. Brusilovsky, A. Kobsa, & J. Vassileva (Eds.), Adaptive hypertext and hypermedia (pp. 1–43). Kluwer Academic Publishers.
Brusilovsky, P. (2001). Adaptive hypermedia. User Modeling and User-Adapted Interface, 11(1/2), 87–110.
Carver, C. A., Howard, R. A., & Lane, W. D. (1999). Enhancing student learning through hypermedia courseware and incorporation of student learning styles. IEEE Transactions on Education, 42(1), 33–38.
Castellano, G., Fanelli, A. M., & Roselli, T. (2001). Mining categories of learners by a competitive neural network. In Proceedings of International Joint Conference on Neural Networks (IJCNN’01) (pp. 945–949).
Chen, S. Y.-H., & Ford, N. J. (1997). Towards adaptive information systems: individual differences and hypermedia. Information Research, 3(2), online.http://informationr.net/ir/ 3-2/paper37.html.
Chiu, C., Norcio, A. F., & Petrucci, K. E. (1991). Using neural networks and expert systems to model users in an OO environment. In. Proceedings of the 1991 IEEE Conference on Systems, Man, and Cybernetics, 3 (pp. 1943–1948).
Dara-Abrams, B. (2002). Applying multi-intelligent adaptive hypermedia to online learning. Ph.D. dissertation, Union Institute & University. Fischer, G. (2001). User modeling in human-computer interaction. User Modeling and User-Adapted Interaction, 11(1/2), 65–86.
Ghazarian, A., & Noorhosseini, S. M. (2010). Automatic detection of users’ skill levels using high-frequency user interface events. User Modeling and User-Adapted Interaction, 20(2), 109–146.
Graf, S. (2007). Adaptivity in learning management systems focusing on learning styles. Ph.D. dissertation, Vienna, Austria: University of Technology.
Graf, S., & Kinshuk, T.-C. L. (2010). Analysis of learners’ navigational behaviour and their learning styles in an online course. Journal of Computer Assisted Learning, 26(2), 116–131.
Hanani, U., Shapira, B., & Shoval, P. (2001). Informationfiltering: overview of issues, research and systems. User Modeling and User-Adapted Interface, 11(3), 203–259. Joerding, T. (1999). A temporary user modeling approach for adaptive shopping on the web. In Proceedings of the 2nd Workshop on Adaptive Systems and User Modeling on the
WWW (ASUM99), Toronto, CA.
Kinshuk, T.-C. L., & Graf, S. (2007). Considering cognitive traits and learning styles to open web-based learning to larger student community. ICTA 2007. Kobsa, A. (2002). Personalized hypermedia and international privacy. Communications of the ACM, 45(5), 64–67.
Koychev, I., & Schwab, I. (2000). Adaptation to drifting user’s interests. In Proceedings of ECML2000 Workshop: Machine learning in new information age, Barcelona, Spain. Lieberman, H. (1995). Letizia: an agent that assists web browsing. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI 95)
(pp. 924–929), Montreal.
Lipsky, S.A. (1984). Effect of Cognitive Style on the Success of Two Textbook Notetaking Techniques. PhD dissertation, USA: University of Pittsburgh.
Liu, M. C., & Lin, H. L. (1999). A concept map based web page structure analysis and design for adaptive learning. In Proceedings of 1999 Taiwan Area Network Conference (TANET’99), (in Chinese).
Lo, J.-J., & Chan, Y.-C. (2008). Relationships between user cognitive styles and browsing behaviors of an online learning web site. Hangzhou, China: Cyberworlds 2008. Lo, J.-J., & Chan, Y.-C. (2011). Design of adaptive web interfaces with respect to student cognitive styles. In The 2nd International Conference on Education and Educational
Technology (EET 2011), Chengdu, China.
Lo, J.-J., & Lin, Y.-S. (2006). An adaptive product recommendation system for anonymous internet visitors. Journal of Information, Technology and Society, 1, 27–48. Lo, J.-J., & Shu, P.-C. (2005). Identification of learning styles online by observing learners’ browsing behavior through a neural network. British Journal of Educational
Tech-nology, 36(1), 43–55.
Lo, J.-J., Wang, H.-M., & Yeh, S.-W. (2004). Effects of confidence scores and remedial instruction on prepositions learning in adaptive hypermedia. Computers & Education, 42(1), 45–63.
Masters, T. (1993). Practical neural network recipes in Cþþ. Academic Press.
Mullier, D. (1999). The application of neural network and fuzzy logic techniques to educational hypermedia, Ph.D. thesis.
Murray, T., Shen, T., Piemonte, J., Condit, C., & Thibedeau, J. (2000). Adaptivity for conceptual and narrativeflow in hyperbooks: the MetaLinks system. In Proceedings of Adaptive Hypermedia (AH 2000), Trento Italy.
Paolucci, R. (1998). Hypermedia and learning: the relationship of cognitive style and knowledge structure. In Proceedings of ED-MEDIA 98. Perkowitz, M., & Etzioni, O. (2002). Towards adaptive web sites: conceptual framework and case study. Artificial Intelligence, 118(1-2), 245–275.
Silver, H. F., Strong, R. W., & Perini, M. J. (2000). So each may learn: Integrating learning styles and multiple intelligences. Alexandria, VA., USA: Association for Supervision and Curriculum Development.
Stash, N. (2007). Incorporating Cognitive/Learning Styles in a General-Purpose Adaptive Hypermedia System. Ph.D. dissertation, Eindhoven: Technische Universiteit, Eindhoven, Netherlands.
Yeh, I.-C. (2001). Applications and practice of neural network models (7th ed.). Taipei: Scholars Books, Taiwan. (in Chinese).
Yeh, S.-W., & Lo, J.-J. (2005). Assessing metacognitive knowledge in web-based CALL: a neural network approach. Computers & Education, 44(2), 97–113.
Zatarain, R., Barron-Estrada, L., Reyes-Garcfa, A., & Reyes-Galavia, O. F. (2011). Applying intelligent systems for modeling students’ learning styles used for mobile and web-based systems. In O. Castillo, J. Kacprzyk, & W. Pedrycz (Eds.), Soft computing for intelligent control and mobile robotics, studies in computational intelligence, 318 (pp. 3–22).
