A Web-based Tutoring System with Styles-matching Strategy for Learning Spatial Geometry

全文

(1)Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. A Web-based Tutoring System with Styles-matching Strategy for Learning Spatial Geometry Hao-Chuan Wang1, Tsai-Yen Li1, and Chun-Yen Chang2 National Chengchi University1, National Taiwan Normal University2 haochuan@iis.sinica.edu.tw, li@nccu.edu.tw, changcy@cc.ntnu.edu.tw The concept of Spatial Geometric Transformation (or SGT for short) is an important foundation for computer graphics and mechanical engineering students. SGT is about how to represent and compute 3D transformations (e.g. scaling, rotation and translation) of objects in the mathematical coordinate systems. It is observed that SGT learning would require learners to construct and manipulate mental images of the spatial configurations in order to learn spatial geometry topics [22]. Suitable instruction tools are usually required for effective learning in classroom. Therefore, (1) how to develop suitable tools for instructing/learning SGT and (2) how to tackle the fact that different learners are with different levels of spatial skills and learning styles, would be the two major concerns to be coped with in this research. By extending the previous work described in [22], this paper introduces a Web-based tutoring system, called CooTutor (Coordinate Tutor), with additional integration of the instructional strategy: stylesmatching. By considering the characteristics of SGT, CooTutor was initially designed as a visualization tool that demonstrates SGT concepts with interactive 3D graphics. Moreover, along with the ongoing studies of intelligent tutoring systems (ITS) and adaptive hypermedia (AH) [6], CooTutor aims to personalize its presentation adaptively to fit individual’s difference, including knowledge status and individual traits (i.e., spatial ability and learning styles). This research specifically focuses on using the stylesmatching strategy to accommodate learners’ traits. Researchers of the AH field have noted the potential of using learners’ latent traits, e.g. spatial ability and learning styles, as indictors for meaningful adaptation [19]. But the discussion of considering learning traits in AH is just about to begin. Two main questions exist on this research topic are identified. They are (1) what instructional strategy should be used to handle learners’ learning traits, and (2) how to fulfill the strategy in practical AH systems. This research presents the mechanism of adaptive material selection fulfilling the strategy of stylesmatching in CooTutor. As a pilot exploration of this issue, an empirical evaluation was conducted to probe the influence of styles-matching on learners’ SGT achievement and spatial ability enhancement. Results, analysis, and implications of this evaluation are presented in this paper. The rest part of this paper is structured as follows. In Section 2, the background of learning styles and. Abstract- How to incorporate instructional strategies in computer-assisted learning systems in a systematic manner has been a major research topic in recent years. In this paper, a style-matching strategy that attempts to match learning materials’ styles to learners’ latent traits is proposed and realized in a Web-based tutoring system, called CooTutor. The mechanism of adaptive material selection takes learners’ different spatial ability and learning styles into account, and performs traits-based personalization of learning experience. This system is specifically designed to conquer the difficulty of tutoring the topic on fundamental spatial geometry in the curriculums of engineering education. By conducting empirical evaluation with a small group of students, we observe that this adaptive mechanism can prevent potentially harmful scenario of styles-mismatching for learners with extreme learning styles. Suggestions from the empirical evaluations and the analysis of effect sizes are also presented in this paper as implications for further studies. Keywords: Adaptive Educational Hypermedia Systems, Web-based learning, Computer Graphics Education, Learning Styles, Spatial Ability.. 1.. Introduction. In the past decades, researchers have built various types of computer-assisted instruction (CAI) tools to help teachers/learners to instruct/construct domain knowledge in a more effective way. It is evident that effective CAI tools would consider and incorporate appropriate instructional strategies for instructing particular domains, such as visualizing abstract concepts (e.g., the visualization of computer algorithms). Although the history of using computers to facilitate learning is long, how to systematically and principledly incorporate adequate instructional strategies into computer-based educational systems remains attractive and challenging to researchers. In particular, with the development of the World Wide Web (WWW), Web-based instruction/learning has gradually become a new type of instruction/learning for teachers and learners. However, the fact is that most Web-based learning today is undertaken upon dedicated learning or course management systems (LMS or CMS). And less instructional strategy or educational concern has been addressed at the system level in today’s Web-based learning paradigm.. 226.

(2) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan. possible instructional strategies are described. In Section 3, the overview of the CooTutor system is described. In Section 4, the mechanism of adaptive material selection will be introduced. Section 5 presents the empirical evaluation, including the experimental design, results, analysis, and implications. Finally, Section 6 draws some conclusions and future works of this research.. 2.. Materials. Application Application Container Container. 3D Objects. Server s Browser. Tutor Console. Learners’ Traits and Instructional Strategies. Lunching Learning Materials. Client si. Main Document Area (XHTML, MathML). (Java Applet) HTML frames. Accessing 3D Obj ects. 3D Blackboard (Java 3D Applet). JavaScript. : HTTP connection : other connection. Figure 1. Overview of the CooTutor system architecture ined dimension forming a continuum of visual/verbal learning style. Some other types of learning style like FD (field dependent)/FI (field independent) learners, sequential/global learners etc., have been frequently imported into education as the basis of instructional design. By considering learning styles for instructional design, designers frequently take the approach of styles-matching to deign the instruction. The strategy is to adapt the content or structure of the instruction with proper pedagogical/teaching styles to match learners’ learning styles [11][16]. CooTutor adopts the strategy of styles-matching as well. The FelderSilverman learning styles model used in this research has formulated these pedagogical styles to cope with different type of learning styles [11]. However, note that some researchers argued that the effect of stylesmatching is doubtable [10]; while some studies suggested that the teachers should attempt to address all styles equally in the instruction and help learners to adapt themselves to learn in their less preferred modes [12]. Although in conventional learning situations, such as classroom-based lecturing, it seems that less experimental result can reveal the effect of statistical interaction between matching and mismatching learning styles [10]. However, in computer-based learning environments some studies have detected statistical interaction regarding the use of styles-matching [16][17][21]. The instructional strategy seems potentially beneficial for designing ITS and AH systems. This research also conducted an empirical experiment to take a look at this point, and the results will be presented in Section V.. Spatial visualization ability. Spatial ability is a psychometric construct that is recognized influential to activities related to spatial reasoning such as engineering activities and scientific thoughts [1][4]. In this study, we are interested in asking how to make use of spatial ability as a basis to adapt the presentation? Two assumptions are given as instructional strategies. First, learners with different spatial ability should receive contents with different types of media representations as assistance. Second, the higher a learner’s spatial ability is, the less degree of visualization she/he will need. For example, if 2D-based (i.e., texts and diagrams) and 3D-based illustrations (i.e., interactive 3D media) are both available for describing a concept, we could scaffold learners with low spatial ability by adopting 3D visualization. For learners with enough spatial reasoning skills, letting the learner practice to form and manipulate the mental image with abstract 2D-based representation is reasonable.. 2.2.. Student Model. Applet-Servlet Comm unication. In CooTutor, two major types of learners’ traits are considered for adaptation. One is the psychometric construct, spatial visualization ability derived from the instrument of Purdue Visualization of Rotation Test (PVRT) proposed by Bodner et al. [4]; while the other is the Felder-Silverman learning styles model [11] derived via the instrument called Index of Learning Styles (ILS) proposed by Soloman et al. [18] The two major traits and their corresponding strategies are described respectively as follows.. 2.1.. Tutor Model. Domain model. Learning styles. Learning styles usually refer to different approaches learners would take to learn. A rather clear explanation on learning styles is as “… (learning styles are) strategies, or regular mental behaviors, habitually applied to learning, particularly deliberate educational learning, and built on her/his underlying potentials. [10]” Many endeavors are to identify types of learning style that can classify learners into distinguishable extremes of that type. For example, visual learners who intend to learn with pictorial representations and verbal learners who prefer to percept textual descriptions. Then visual and verbal styles of learning are proposed as two end points of an imag-. 3.. Overview of the CooTutor System. The CooTutor (Coordinate Tutor) system is an adaptive Web-based tutoring system with interactive 3D media for SGT learning. Details of the design and the underlying considerations of the system are de-. 227.

(3) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. Domain model (prerequisite graph). 3D Blackboard. Concept sequencing. Main Docs. Learning goal. Overlay model. concept sequence. Learning materials. Material selecting. server client. recommendations. Fine tuning. Learners’ traits. Figure 3. The flow of adaptivity in CooTutor. Tutor Console. ing to the student model and learning materials’ features. Figure 3 shows the flow of adaptivity in CooTutor. Three main levels exist: concept sequencing, adaptive material selection and client-side tuning. Server-side decision making is divided into two levels. At the first level, the mechanism of concept sequencing is responsible for generating a sequence of concepts in accordance with learners’ knowledge status. If learners lack some required prior knowledge to learn a particular concept, they will be directed to learn those concepts first as proper complements. The concept sequencing algorithm used in CooTutor is described in [22]. Once the sequence of concept has been determined, the next phase, i.e., adaptive material selection, proceeds. The task is to select learning materials from the content repository to illustrate the concepts to learners. The styles-matching strategy is applied in this phase to account for learners’ learning styles as described in next section.. Figure 2. User interface with 3D blackboard presented at the client-side. scribed in [22]. In this section, the system is described summarily from an overall point of view. The system architecture of CooTutor is illustrated in Figure 1. The system could be decomposed into two sides according to the Web-based nature, serverside and client-side. At the server-side, several models, including domain, student and tutor models are explicitly designed and incorporated for performing adaptivity. At the client-side, three main parts of the user interface can be identified. They are the part of tutor console, main document area, and the 3D blackboard. The appearance of the user interface is shown in Figure 2. Among these modules at the client-side, main document is the area used for presenting regular Web pages. Tutor console is the module that mainly takes the responsibility of managing the communication between the server and the client. This module is also employed to interact with users to collect learners’ information for adaptation.. 3.1.. 4.. Presenting SGT concepts with 3D blackboard. The mechanism of adaptive material selection is responsible to take materials’ pedagogical styles and learners’ traits as inputs and perform the stylesmatching strategy. In CooTutor, adaptive material selection is abstracted as a task of information retrieval (IR). The information of learners’ traits is used as the query, and each learning material’s style is employed as the feature. The styles-matching strategy can then be realized by computing the similarity between the query and the associated feature vector of learning materials [3]. This type of IR is to retrieve documents at the level of pedagogical styles.. The 3D blackboard module is specifically designed for presenting 3D content, and is extremely useful for tutoring SGT concepts. In the 3D blackboard, learners are allowed to navigate the 3D environment by dragging the mouse to realize the spatial relation between objects and the coordinate system in a clearer manner. This type of 3D visualization could offer learners without sufficient spatial reasoning skills appropriate scaffolding. The integration and interaction between the 3D blackboard and HTML documents provides an environment for learnercentered construction. Learners can read descriptions appeared in HTML pages, and then do experiments via interacting with embedded buttons and fields in pages to see how different spatial configurations would influence the geometric transformation in the 3D blackborad.. 3.2.. Adaptive Material Selection in CooTutor. 4.1.. Features of learning materials. In the SGT domain, learning materials are usually with different degree of abstraction (i.e., abstract mathematical descriptions vs. concrete and practical examples), different modalities of media representation (i.e., 2D-based Web pages vs. interactive 3D visualization), different types of learning activities (i.e., lecture vs. experiment), and different levels of details. Taking these features described above altogether, learning materials themselves are actually. Adaptivity in CooTutor. CooTutor adopts the course sequencing approach to attain adaptivity [5][22]. In brief, the task is to select a set of learning materials for learners accord-. 228.

(4) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. Spatial Ability. and abstract materials, such as mathematical descriptions. Third, active learners would prefer doing experiments, while reflective learners would prefer learning materials in the form of typical lecture. Fourth, level-of-details is set to medium (numeric value 0.5) initially. The form of queries is illustrated below, which is represented as a vector consisting of seven elements:. Learning Styles. Query Generator. Default Assumptions. Query of Pedagogical Styles. Q = <is_2D, is_3D, is_concrete, is_abstract, is_lecture, is_experiment, level_of_details>. Figure 4. Query formulation with default assumptions. Each element has a numerical value varies from 0 to 1. The first three pairs of elements are designed to be complementary to assure that the representation is robust if any element is indexed as zero in computing similarity:. with various pedagogical styles. Note that the feature of main representation, abstractness and activity type are bidirectional. For example, a research paper with plenty mathematical descriptions is thought to be more abstract. Then the feature of abstractness would be assigned a high value, such as 0.8. Similarly, for concrete and practical learning materials, a lower value would be assigned on the same feature. That is, only one field is employed to record such a bidirectional feature.. 4.2.. is_2D + is_3D = 1, is_concrete + is_abstract = 1, and is_lecture + is_experiment = 1. The job of the query generator shown in Figure 4 is to transform learners’ traits into the query Q. Note that the representation of a query is of the same format as what will be described next—the feature vector of learning materials.. Query formulation from learners’ traits. 4.3.. As mentioned previously in Section 2, the instruments of PVRT and ILS are used to assess learners’ spatial ability and learning styles, respectively. The ILS questionnaire consists of 40 question items in four dimensions of learning styles: visual/verbal learning style, sequential/global learning style, sensing/intuitive learning style, and active/reflective learning style. However, since this research intends to treat spatial ability as ability instead of a style for material adaptation and sequential/global learning style is not appropriate at the level of material selection, items for the first two styles are not employed. In other words, CooTutor assesses learners’ spatial ability, sensing/intuitive learning style, and active/reflective learning style for material selection. The information of learners’ traits is used to trigger the query of pedagogical styles. The process of query formulation is shown schematically in Figure 4. By using learners’ quantitative scores of spatial ability and learning style as external inputs, with default assumptions, the query of pedagogical styles is derived as the result. The default assumptions used in this case are enumerated as follows. First, as mentioned in Section 2, the higher a learner’s spatial ability is, the less degree of visualization she/he will need. Second, sensing learners would prefer concrete or practical learning materials, while intuitive learners prefer theoretical. Computing similarity for material selection. It is quite popular to use the cosine measure to determine the similarity between two vectors in traditional IR [19]. However, according to the characteristics of our task, we intend to use extended Jaccard coefficient to measure the similarity between content objects [14][19]. The advantage of using extended Jaccard coefficient is evident that it can tackle both binary values and numerical values. To compute the measure, each object should be represented as a feature vector. In our case, features of learning materials are transformed to a feature vector M having the same form as Q. Each element of the vector has a numerical value varies from 0 to 1. The first two elements is_2D and is_3D stem from the feature of Main_representation. For example, the feature-value pair, Main_representation = 0.8, could be transformed into is_2D = 0.2 and is_3D = 0.8. Given the query vector Q and learning material i’s feature vector Mi, the similarity measure using extended Jaccard coefficient is computed as:. SQ , M = S Jaccard (Q, M i ) = i. 229. QT M i QT Q + M iT M i − QT M i. (1).

(5) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan. Table 1. Four versions of CooTutor compared in the experiment. The higher the measure derived by Eq. (1), the more similar it is between the learning material i and the query Q. By repeatedly measuring the similarity measure of the query and all candidate learning materials, a threshold, Sthreshold, could be set to divide learning materials into two categories, “recommended” and “not recommended” Note that such a threshold is set empirically, and should be determined by considering characteristics of the set of learning materials, the learning domain, and the pedagogical strategies intended to be applied. Then, the category CM of each learning material Mi can be determined by the following formula.. recommended , CM =. 4.4.. Group. LS PreAuthor No Filter MisLS. not recommended ,. otherwise. (2). Tuning the decision with learners’ feedback. Since the mechanism of material selection described above only takes the measured scores of learners’ traits and default assumptions as the basis of decision making, the generated query is just a rough stereotype, and may be improper to reflect learners’ actual preference of styles. Therefore, CooTutor will further interact with learners by asking them if they satisfy with the presentation style after a session is finished. A simple question like “Do you want to see more pictures?” will be presented to learners. Learners’ feedbacks will be used to tune the query vector for the next round of material selection.. 5. 5.1.. 10 12. Strategy of selection. Stylesmatching Manually pre-selected. Adaptive Ranking?. # of materials. Yes. Varies. No. 16. 4. No filtering. No. 33. 5. Stylesmismatching. Mismatch. Varies. fixed duration. All of the four groups were assigned the same learning goal—the concept of “Gimbal Lock” in SGT with a concept sequence consisting of 14 concepts. However, these four groups would receive different degree of adaptation in terms of concept sequencing and adaptive navigation support. Table 1 summarizes how these four versions of CooTutor differ. Note that the number of participants in each group differs. Especially the last two groups shown in the table only have 4 and 5 participants respectively. This is because the main interest of this research is actually upon the first two groups (i.e., the LS and PreAuthor groups), but the effect of the last group is also suspected. In order to address both issues, this research choose to assign a large portion of participants into the first two groups, but keep a small portion of them in the last two for references. Among these four versions, LS is the version that employs the mechanism of adaptive material selection. And the score of threshold of recommendation, Sthreshold was set as 0.6. Therefore, learning materials would be selected and ranked adaptively based on participants’ traits. Note that since learners’ spatial ability and learning styles would vary, so the size of hyperspace (number of materials) would vary by employing the mechanism of adaptive material selection. PreAuthor is the version that does not use the adaptive mechanism, but ask a human teacher of SGT to select a fixed set of learning materials a priori. This version could be thought as the group without traits-based adaptivity. Totally 16 learning materials were pre-selected for learning these 14 concepts. The third group, NoFilter, is the group that offers the participants all available materials stored in the content repository which now holds 33 learning materials. That is, no filtering or selection would be done to reduce the size of hyperspace. By exchange the values of is_lecture and is_experiment, the last version, MisLS, is designed to probe what if learning materials are with totally improper pedagogical styles. Five participants with extreme scores on this learning style were specifically dispatched to this group. Therefore, some decisions have been undertaken to form groups strategically.. if S Q , M ≥ Sthreshold i. # of participants. Evaluation Design of the experiment. This experiment aims to probe the effectiveness of the styles-matching strategy in CooTutor. Four different versions of CooTutor with different methods on material selection were used in this experiment. This experiment adopted a pre-test/post-test comparison-group experimental design which is also used by [7]. These four versions of system differ from each other on the strategy of material selection. The experiment was held in June 2004 at National Chengchi University (NCCU), Taiwan. Totally 31 graduate-level participants majored in Computer Science or Information Systems (master program) from NCCU have participated in the experiment. They were grouped as four double-blinded groups, and each group was assigned to use one version of CooTutor. The whole duration of the experiment lasted for three weeks. Participants are asked to sign in the system, take the pre-tests, view all learning materials, and finally be tested by post-tests. Since Web-based learning is naturally self-paced, participants were not enforced to operate the system at a specific time and. 230.

(6) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. 5.2.. like this experiment, achieving statistical significance is inherently more difficult than for a large one. Some researchers even recognized that “an SST (statistical significance testing) is largely a test of whether or not the sample is large.”(Daniel 1998, p26) [9]. Therefore, for this experiment, the coefficient of effect size may offer more informative implication of the data. Taking these concerns altogether, the effect sizes of gain scores between post- and pre- tests are the main statistics this study focuses on. For the data of LS and PreAuthor groups, the method for analyzing covariance (ANCOVA) is also applied.. Measuring Instruments. Two types of scores are measured in this experiment: learners’ spatial ability and achievement on the topic of spatial geometric transformation. The PVRT test was employed to measure spatial ability. A self-compiled achievement test consisting of 7 items was authored to measure learners’ achievement on SGT. In this experiment, the estimated reliabilities of the PVRT derived by Kuder-Richardson 20 formula (KR-20) are 0.69 and 0.75 for the pre-test for the post-test, respectively. The 7-item SGT achievement test was compiled and employed on both pre- and post-test. The estimated KR-20 coefficient is 0.59 for the pre-test and 0.28 for the post-test. For the scenario of low reliability, it is suspected that two main factors take effect. First, the number of items of this test is rather small. It is naturally difficult to achieve high reliability for a test with few items. Second, because the underlying computation of reliability coefficient relates to the distribution of measuring results [13], the low reliability reflects that the homogeneity between participants may be high. Small variation is there between individuals. This implies that amount of participants has got scores very close to the upper limit of measurement of this instrument. By taking the global observation on results of both pre- and post-test, this situation is very likely to suggest that learning with CooTutor has substantially changed the distribution of the participants toward the high score area. This observation implicates that, the instrument itself is probably not that un-reliable (KR-20 0.59 on the pretest), but the improvement of SGT understanding has contributed side-effects to computing the reliability coefficient of post-test.. 5.3.. 5.4.. Experimental Results. Table II and III present the results of PVRT test, SGT achievement test respectively. A paired 2-tailed t test was performed to compare the difference of means between post- and pre- tests. Meanwhile, the effect size is specifically computed by using Cohen’s d coefficient [8]. Since Cohen’s d coefficient (and other type of effect size measure) is a standardized score, some criterion is required to conceptualize the result. From the literature, researchers have suggested the value of Cohen’s d coefficient to be 0.2 for a small effect size; 0.5 for a medium size; and above 0.8 for a large effect size [2][8]. By using the viewpoint of effect size, the results shown in Table 2, 3 are described as follows. 5.4.1.. Spatial Ability Enhancement. An ANCOVA analysis has been conducted upon the data of LS and PreAuthor. The post-test score of PVRT is the dependent variable and the pre-test score of PVRT is used as the covariate. The difference, F(1,19)=0.06, p=0.809, is statistically insignificant. Back to Table 2 to study the report of effect sizes, it is worth noting that the PreAuthor group reveals the best performance among all groups on spatial ability enhancement. The result of paired t-test reveals statistically significant (p=0.013 < 0.05), and the effect size is quite large (d=1.069). Though the LS group did not reveal strong effectiveness on this task, but the MisLS group performs quite worse on the post-test. The effect size, d=-0.785, is very large on the inverse (i.e., negative) direction. No similar scenario happened in other groups.. Data Analysis. In this experiment, the objective is to detect if any difference of the effects existed between the four groups. Two major issues are considered in the analysis: 1) The unbalanced degree of freedoms: Note that the number of participants (i.e., degree of freedom of statistics) differ among the four groups. Ten participants are in the group of LS and PreAuthor and five for the other groups. Therefore, it is not tenable to compare the results of all groups by applying a single statistical significance test [2]. 2) The needs of comparing effect sizes between groups: Researchers have noted that using only statistical significance testing to interpret experimental data may not be sufficient [8][9][15] because the computation of statistical significance is related to the sample size involved. For a small size of samples,. 5.4.2.. SGT Achievement. The ANCOVA analysis on comparing LS and PreAuthor shows no statistical significance as well. The pre-test score of SGT achievement is also used as the covariate in this analysis. The result is statistically insignificant: F(1,19)=0.034, p=0.856. These two groups seemed to perform equally well on the task of enhancing SGT achievement.. 231.

(7) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. Table 2. Statistics of participants’ pre- and post- PVRT scores with effect size coefficients. Pre_PVRT. Post_PVRT. Mean. SD. Mean. SD. Post vs. Pre t-test, p. Effect size, d. LS (n=10). 15.60. 3.03. 16.60. 2.12. 0.437. 0.383. PreAuthor (n=12). 13.25. 3.22. 16.17. 2.13. 0.013**. 1.069‡. NoFilter (n=4). 17.00. 2.16. 18.25. 2.22. 0.194. 0.570†. MisLS (n=5). 16.60. 1.95. 13.20. 5.81. 0.252. -0.785∆. Overall. 15.03. 3.14. 16.10. 3.18. 0.173. 0.337. *p<0.1 **p<0.05. ‡. †. ∆. Effect size: large, medium, negatively large. Table 3. Statistics of participants’ pre- and post- SGT achievement scores with effect size coefficients. Pre_Achievement. Post_Achievement. Mean. SD. Mean. SD. Post vs. Pre t-test, p. LS (n=10). 3.90. 1.85. 4.90. 1.52. 0.052*. 0.589†. PreAuthor (n=12). 4.67. 1.72. 5.33. 1.23. 0.166. 0.445†. NoFilter (n=4). 4.00. 1.63. 5.25. 1.26. 0.015**. 0.857‡. MisLS (n=5). 4.40. 1.82. 4.60. 0.89. 0.749. 0.140♦. Overall. 4.29. 1.72. 5.07. 1.26. 0.003***. 0.514†. *p<0.1 **p<0.05 ***p<0.01. Effect size: ‡large, †medium, ♦small. From Table 3, the LS group performed slightly better than the PreAuthor group. From the view of effect size, the d coefficient is 0.589 (medium effect size) for the LS group, and 0.445 (close to medium) for the PreAuthor group. The best group is the NoFilter group, and the result of paired t-test comparing the means of post- and pre- tests indicates statistical significance (p=0.015<0.05). Its effect size, d=0.857 is a large one. Finally, for the MisLS group, the gain effect is very small (d=0.140) and distant from the effects of improvement shown by other groups and the overall performance.. 5.5.. Effect size, d. 6.. Conclusion. In this paper, the instructional strategy of stylesmatching is incorporated into the adaptive Webbased learning system, CooTutor, for tutoring spatial geometric concepts. The system employs information retrieval techniques to retrieve learning materials regarding pedagogical styles. Learners with different degrees of spatial reasoning skills and learning styles can then be tutored adaptively. An empirical evaluation was conducted to evaluate this design. By this result, it is suggested that the mechanism of adaptive material selection fulfilling styles-matching strategy is applicable, especially to prevent the harmful scenario of mismatching. Recent studies of learning styles have suggested other strategies for learning styles [12], e.g., to offer needed scaffolding to learners and let learners try to learn from not preferred styles. In other words, besides adaptive material selection, it would be beneficial to develop corresponding mechanisms in Webbased learning regarding extreme learning styles. With this trend of development, it is suggested that Web-based instruction/learning should not only consider instructional strategies. At the next step, how to incorporate more meta-cognitive concerns into Webbased instruction/learning to help learners adapt themselves to unfamiliar styles/environments of teaching would be a new challenge.. Discussion. In sum, adaptive material selection regarding learners’ traits does not outperform other designs, especially the version intended to be compared with, a set of learning materials selected by a human teacher. However, it is worth noting that the situation of styles mismatching might yield negative impacts on learning, specifically for those learners with extreme learning styles. However, since the sample size is small in this experiment, refinement and more replications of the experiment are necessary. The result of effect sizes reported by this research could be employed for proper meta-analyses and comparison along with other empirical studies in the future.. 232.

(8) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan. [11] R. M. Felder and L. K. Silverman, “Learning and Teaching Styles in Engineering Education,” Engineering Education, 78(7), pp. 674-681, 1988. [12] R. M. Felder, G. N. Felder, and E. J. Dietz, “The Effects of Personality Type on Engineering Student Performance and Attitudes,” Journal of Engineering Education, 9(1), pp. 3-17, 2002. [13] D. N. M. de Gruijter and L. J. Th. van der Kamp, Statistical Test Theory for Education and Psychology, Graduate School of Education, Universiteit Leiden, Netherlands, 2003. [14] J. Han and M. Kamber, Data mining: Concepts and Techniques, Academic Press, 2001. [15] J. E. McLean and J. M. Ernest, “The role of statistical significance testing in educational research,” Research in the Schools, Vol. 5, pp. 15-22, 1998. [16] P. Rosati, R. K. Dean and S. M. Rodman, “A Study of the Relationship Between Students’ Learning Styles and Instructors’ Lecture Styles,” IEEE Trans. on Education, Vol. 31, No. 3, 1998. [17] V. J. Shute, “A Comparison of Learning Environments: All That Glitters…,” Computers as Cognitive Tools, pp. 47-73, NJ: Erlbaum., 1993. [18] B. A. Soloman and R. M. Felder, Index of Learning Styles, http://www.engr.ncsu.edu/learningstyles/ilsweb.html [19] N. Stach, A. Cristea and P. De Bra, “Authoring of Learning Styles in Adaptive Hypermedia,” in Proc. of WWW Conference, NY, USA, 2004. [20] A. Strehl and J. Ghosh, “Relationship-based Clustering and Visualization for High-Dimensional Data Mining.,” INFORMS Journal on Computing, pp. 208230, Spring 2003. [21] E. Triantafillou, A. Pomportsis, S. Demetriadis and E. Georgiadou, “The Value of Adaptivity based on Cognitive Style: An Empirical Study,” British J. of Educational Technology, 35(1), pp. 95-106, 2004. [22] H-C. Wang, T-Y. Li and C-Y. Chang, “Adaptive Presentation for Effective Web-based Learning of 3D Content,” in Proc. of 4th IEEE Intl. Conf. on Advanced Learning Technologies, Finland, Aug. 2004.. Acknowledgement This work was partially supported by a grant from National Science Council under contact NSC 932213-E-004-001.. References [1] M. Alias, T. R. Black, D. E. Gray, “Effect of Instruction on Spatial Visualisation Ability in Civil Engineering Students,” International Education Journal, Vol. 3, No 1, 2002. [2] A. Aron and E. N. Aron, Statistics for Psychology, 2nd ed., Prentice-Hall, 2000. [3] R. Baeze-Yates and B Ribeiro-Neto, Modern Information Retrieval, Addison Wesley, 1999. [4] G. M. Bodner and R. B. Guay, “The Purdue Visualization of Rotations Test,” The Chemical Educator, Vol. 2, No. 4, 1997. [5] P. Brusilovsky and J. Vassileva, “Course Sequencing techniques for large-scale web-based education,” Int. J. Cont. Engineering Education and Lifelong Learning, 2003. [6] P. Brusilovsky, “Adaptive Hypermedia,” User Modeling and User-Adapted Interaction, 11, pp. 87-110, 2001. [7] C-Y Chang, “The Impact of Different Forms of Multimedia CAI on Students’ Science Achievement,” Innovations in Education and Teaching International (IETI), Vol. 39, Is-sue 4, pp. 280-288, 2002. [8] J. Cohen, Statistical Power Analysis for the Behavioral Sciences 2nd ed., NJ: Lawrence Erlbaum, 1988. [9] L. G. Daniel, “Statistical significance testing: a historical overview of misuse and misinterpretation with implication for the editorial policies of educational journals,” Research in the Schools, Vol. 5, pp. 23-32, 1998. [10] S. Draper, “Learning Styles notes,” Psychology Department, University of Glasgow, 2003. http://www.psy.gla.ac.uk/~steve/lstyles.html. 233.

(9)