The Function aggregates the element of the set Xkof WCLR concerning concept Ck, and define the ACRL of concept Ck as
where wijk indicates the WCLR defined in Definition 5.
Example 6: Aggregation Concept Learning Response index (ACLR)
According to the ICRT, the test items T1, T2, T3 are decomposed into {C1,C2,C3}, {C1,C3}, {C3}, respectively. The learning response index of concepts in each test item is obtained by the product of the test items’ LRI and the weight in ICRT. Each concept
Cjhas a set WLRI from several test items Tk concerning concept Cj.
Table 4 The WLRI and the ACLR of concepts
Item C1 C2 C3
T1 0.64 0.80 0.35 T2 0.69 0 0.87 T3 0 0 0.69 ACLR 0.902 0.800 0.996
Suppose student Si has such a table shown in Table 4 after testing. T1 includes the
concepts of C1, C2 and C3 with the WLRI of 0.64, 0.80 and 0.35; T2 include the concepts of C1 and C3 with the WLRI of 0.69 and 0.87, and T3 include the concepts of C3 with the WLRI of 0.69. By looking through the columns, concept C1, C2 and C3 will have the sets of WLRI {0.64,0.69,}, {0.80,} and {0.35,0.87,0.69}through T1, T2 and T3, respectively. Then the ACLR of student Si concerning concepts C1, C2 and C3 are calculated as below.
i
(1- 0.97 0.64)(1- 0.97 0.69) -1
({0.64,0.69}) 0.902
-0.97
g × ×
= =
(1- 0.97 0.80) -1
({0.80}) 0.800
-0.97
gi ×
= =
(1- 0.97 0.35)(1- 0.97 0.87)(1- 0.97 0.69) -1
({0.35,0.87,0.69}) 0.996
-0.97
gi × × ×
= =
4) Fuzzy ACLR Generator
In order to mine association rule further, it is necessary to transform the numeric data into symbolic data. We achieve the transformation by applying fuzzy Theory. Here we have two membership functions shown in Fig. 8 to transform the students’ numeric ACLR into symbolic notation. The symbolic notations obtain by the fuzzification is “L” and “H”, which denote “Poor Learning” and “Well Learning”, respectively. Ci.L and Ci.H denote the value obtain from the LOW Fuzzy Function and the HIGH Fuzzy Function of the concept Ci.’s ACLR, respectively. The value of each concept’s ACLR will transform into the notation “L”
and “H” if Ci.L>Ci.H and Ci.L ≤ Ci.H, respectively. For example, if ACLR=0.65, by the given membership functions, we have Ci.L=0.2 and Ci.H=0.66. Since Ci.L ≤ Ci.H the Fuzzy ACLR is denoted as “H”. Completely, fuzzification of ACLR is described in Example 4.6.
ACLR
Fig. 8 The given membership functions of students’ numeric ACLR.
Example 7: Fuzzy ACLR Generator
Suppose six students are tested with five concepts. Students’ ACLR of each concept is shown in Table 5.
Table 5 The Students’ ACLR of each concept
C1 C2 C3 C4 C5
By the given membership functions, ACLR of each student has the degree value of Ci.L and Ci.H. Table 6 shows the fuzzy degree values of ACLR obtain by the LOW/HIGH fuzzy membership functions.
Table 6 The degree value of ACLR translated by the LOW /HIGH fuzzy membership functions
C1 C2 C3 C4 C5
We then transform the numeric ACLR into the symbolic notation of “H” if Ci.L ≤ Ci.H and “L” if Ci.L>Ci.H. The notation “H” and “L” represent the meaning of “well learning”
and “poor learning”, respectively. Table 7 shows the fuzzification result of ACLR.
Table 7 The Students’ fuzzy SCCI of each concept.
C1 C2 C3 C4 C5
S1 H L H H L
S2 H L L H H
S3 H L L L H
S4 H L H H H
S5 H L L L L
S6 H L L H H
Concept FACLR
Student
4.2
Data Mining Module
After the fuzzification of ACLR, the symbolic data “H” and ”L” are then process by the Data Mining Module. First, the Apriori Algorithm of data mining is adopted to discover the association rules. Finally, CERM is constructed based on the scenario explanation of the mined association rule.
Algorithm 1: Apriori Algorithm Symbol Definition:
α : The minimum support threshold in the A -large itemset.
CA: Candidate itemset of size A . LA: Frequent itemset of size A .
λ : The minimum confidence threshold.
Input:
The FACLR of students.
The threshold of minimum support α . The threshold of minimum confidence λ.
Output : The association rules of FACLR of students.
LA = {frequent items};
for ( A = 1; LA=φ; A ++) do begin
CA+1= candidates generated from LA; for each transaction t in database,
do increment the count of all candidates in CA+1, that are contained inCA+1 LA+1 = candidates in CA+1 with min_support
end
return CAand LA
1) Association Rule Mining
We mine the association rules from the Fuzzy ACLR by using Apriori Algorithm of data mining. Four types of association rules L-L, L-H, H-L and H-H are used as the model to discover the assimilation and mis-concept effect relations among concepts.
Example 8: Apriori Association Rule Mining Algorithm
From the data shown in Table 7, Fig. 9 shows the process of mining the association rules
by Apriori algorithm with minimum support 0.6 and minimum confidence 0.6.
The 1 -Candidate itemset
Fig. 9 Process of Apriori algorithm
Table 8 shows the Association Rule mining with minimum support 0.6 and minimum confidence 0.6 generated from large 2 itemset into L-L, L-H, H-H, and H-L types. The Confidence is used to indicate the important degree of ith mined association rule. For example, the Confidence of rule C2L→C3L can be obtained as follows.
Table 8 The Mining Results (Confidence ≥0.6)
The Large 2 Itemset
Rule Types Mined Rules Confidence
1H 4H
2) Concept Map Constructor
We define the direction by the effect relationship, and the weight of edge indicates the influent probability defined by the order pairs, support and confidence. According to the above concept effect direction and weight, Concept Effect Relation Map of the students’ is constructed based upon the scenario explanation of the association rule shown in Table 9.
Table 9 The scenario explanation of association rule
Association
C1 is the prior concept of C2with support value higher then C2.
1L 2L
C →C
Assimilation
(Positive related) C2 →C1 ,
C2is the prior concept of C1 with support value lower then C1 .
1L 2H
C →C
2 1
C →C ,
C2is the alternative concept of C1 with higher confidence value.
1H 2L
C →C
Misconception
(Negative related) C1 →C2,
C1 is the alternative concept of C2with higher confidence value.
5. Experiment
In this thesis, we applied the IRT-Based Data Preprocessing Concept Effect Relation Map Construction System in Mathematics to evaluate its effectiveness. The experiment is based upon Table 10, which is the basic data of the experiments in Mathematics, this chapter
describes the experiment in detail.
Table 10 Statistics of the experiment.
Course Mathematics
School Senior High School
Grade K-11
Number of students 42
Average score 62.36
Number of test item 32
Standard deviation of scores 15.43 Average difficulty of the test items 0.535 Difficulty range of the test items 0.125~0.938 Average discrimination of the test items 0.107 Discrimination range of the test items -0.5~0.75
The experiment was based on Mathematics tests administered at a senior high school.
There are 42 students participated in the experiment, and their average test score was 62.36, while the average discrimination level of the test items is 0.107. Table 11 list the notation of concepts included in the test sheet. In this experiment, the weight of concepts included in ICRT would be set to 1 to simplify our discussion. Moreover, the fuzzification threshold is set to 0.6, and the mining support and confidence are both 0.6.
C1
0.61 (0.64, 0.96) (0.98, 0.98) (1.00, 0.64) (1.00, 0.98)
Table 11 Notation of concepts included in the test sheet.
Concept Notation Concept
C1 Spatial Relation of Points, Lines and Planes
C2 Logical Concept
C3 Symmetrical Point
C4 Distance of Two Point
C5 Trigonometric function
C6 Cosine Theorem
C7 Angle of Two Intersections Planes C8 Coordinates Reference of Spatial Object
C9 Perpendicular Point
C10 Equations of Coordinate Planes C11 Three Perpendicular Lines Theorem
As shown in Fig. 10, Fig. 11 and Fig. 12, the left side of the Figure shows the data preprocessing with IRT-Based, while the right side of Figure shows the data preprocessing with correct answering ratio which is un based on IRT.
Fig. 10 H-H type CERM constructed with and without IRT-Based Data preprocessing.
Fig. 10 shows the H-H type CERM. Attention to the un IRT-Based CERM in Fig.11, the CERM has two circulated effect relationship, one is the C3¼C6¼C11¼C3 and the other
C5
one is C3¼C6¼C11¼C9 ¼C3. As the experiment shows, based on the same support and confidence, the CERM construction with IRT-Based data preprocessing is better and reasonable than the one without IRT-Based data preprocessing.
Fig. 11 L-L type CERM constructed with and without IRT-Based Data preprocessing.
Fig. 11 shows the L-L type CERM. The concept at the bottom of L-L type CERM means much more difficult than other. For example, in the left part of Fig. 11,if C3 is not well learning by student, the key problem in learning C3 is a lack of understanding of concepts C7, C8 and C11, so the student should learn concepts C7, C8 and C11 before learning C3. The left CERM suggests the learning strategy of C3: instead of learning C3 repeatedly, the learning of C11, C8 and C7 has to be firstly enhanced.
Fig. 12 shows the L-H & H-L type CERM, which indicate the mis-concept effect among concepts. The mis-concept effect may be caused by misunderstanding of concept, confuse among concept, etc. As shown in the left of Fig. 12, the concept set {C1,C2,C3,C7}is the
C5
C1
alternative concept of the concept set {C8,C9,C10,C11}. The CERM construction with IRT-Based Data preprocessing generates more Association rules than the one without IRT-Based Data preprocessing.
Fig. 12 L-H and H-L type CERM constructed with and without IRT-Based Data preprocessing.
6. Conclusion
The assessment analysis and the concept mapping representation of the analysis result have become an important issue of e-learning. The result of the assessment can be analyzed to discover effect relations among concepts, such as assimilation effect and mis-concept effect.
Diagnoses by analyzing the result of assessment can improve students’ learning status, and the teaching while tutoring.
Concept Effect Relation Map (CERM) constructed by data mining with naïve data preprocessing causes monotonous concept fuzzification result. At the same time, the circulating association rules exit, and the association rules may not reflect the concept relation physically. In this thesis, we propose an IRT-Based Data Preprocessing Concept Effect Relation Map Construction System with the consideration of the difficulty and the discrimination of test item,
IRT-Based Data Preprocessing Concept Effect Relation Map Construction System includes two modules: the Data Preprocessing Module and the Data Mining Module. The first module has four procedures: Test Item Analysis, Learning Response Index (LRI) Generator, Concept Decomposition/Aggregation and Fuzzy ACLR Generator. The second module is called Data Mining Module with two procedures. Association rule mining and concept map constructor.
The experiment of the proposed approach shows the improvement of constructing CERM and the reduction of the circulated association rules’ number generated. The main contributions of this thesis are:
(1) The IRT-Based Data preprocessing Approach we proposed refines the assessment of concept learning response.
(2) Based upon the Item Response Theory, we define a fuzzy membership function to quantify the learning status of concept.
(3) The experiment of the proposed Approach indicates the improvement of CERM construction in association rules mining.
There are some interesting issues in extending the application of CERM in the nearly future:
(1) The distribution weight of the items and concepts may affect the concept learning response, CERM can be use to indicate the quality of the test sheet.
(2) The development of comparing technique: comparing the CERM of different learning groups or the CERM of teachers.
Reference
[1] Cmap Tools,IHMC, http://cmap.ihmc.us/.
[2] Item Response Theory- Theory and application, http://www.edutest.com.tw/e-irt/irt.htm [3] Test Diagnostics, http://www.edtech.vt.edu/edtech/id/assess/diagnostics.html
[4] J. Appleby, P. Samuels, and T.T. Jones , “Diagnosis–A Knowledge-based Diagnostic Test of Basic Mathematical Skills” , Computers & Education , 28(2), February, 1997, pp.
113-131.
[5] J.D. Barbara, R. S.Christine, B. Toni, G. Kate, P. Linda, "Concept Maps: A Stratagy to Teach and Evaluate Critical Thinking", Journal of Nursing Education , 38(1),
January,1999, pp. 42-47.
[6] E. Bruillard & G.L. Baron, “Computer-based concept mapping: a review of a cognitive tools for students”, International Conference on Educational Uses of Communication and Information Technologies, Beijing, China, August 21-25, 2000, pp. 331-338.
[7] G. Frosini, B. Lazzerini, and F. Marcelloni, “Perform Automatic Exams”, Computers &
Education, 31(3), November, 1998, pp. 281-300.
[8] B.R. Gaines and M.L.G. Shaw, “Concept maps as hypermedia components”, International Journal of Human Computer Studies, 43(3), 1995, pp. 323--361.
[9] H. Gamboa, “Designing Intelligent Tutoring Systems : A Bayesian Approach”, Proceedings of Ana Fred 3rd International Conference on Enterprise Information
Systems (ICEIS'2001), 2001, pp. 452-458.
[10] J.L. Gordon, “Creating knowledge maps by exploiting dependent relationships”, Knowledge-Based Systems, 13(2000), pp.71-79.
[11] J. Han and M. Kamber, Data mining concepts and techniques, Morgan Kaufmann Publishers. P230-239.
[12] R.K. Hambleton and S.H. waminathan, (1985). Item Response Theory, Kluwer-Nijhoff Publishing, Boston, Massachusetts.
[13] G.J. Hwang, “A Conceptual Map Model For Developing Intelligent Tutoring System”, Computers & Education, 40(3), April, 2003, pp. 217-235.
[14] G.J. Hwang, C.L. Hsiao, and C.R. Tseng, “A Computer-Assisted Approach to Diagnosing Student Learning Problem in Science Course”, Journal of Information Science & Engineering, 19(2), 2003, pp. 229-248.
[15] C.L. Hsiao, G.J. Hwang, and C.R. Tseng, “An interactive concept relationship construction assisted system for learning diagnosis,” The 5th Global Chinese Conference on Computers in Education, 2001, pp. 925-932.
[16] C.S. Hsu, S.F. Tu, and G.J. Hwang, “A Concept Inheritance Method for Learning Diagnosis of a Network-based Testing and Assessment System”, Proceedings of The 7th International Conference on Computer-Assisted Instructions, 1998, pp. 602-609.
[17] D.H. Jonassen, (1996). Computer in classroom: Mindtools for critical thinking.
Englewood Cliffs, NJ: Prentice-Hall, Inc.
[18] J.D. Novak, Learning, Creating, and Using Knowledge: Concept Maps As Facilitative Tools in Schools and Corporations, Lawrence Erlbaum Assoc 1998.
[19] K.A. Papanikolaoua, M. Grigoriadoua, G.D. Magoulasb, and H. Kornilakisa, “Towards New Forms of Knowledge Communication: The Adaptive Dimension of a Web-based Learning Environment”, Computers & Education, 39(4), December, 2002, pp. 333-360.
[20] W.J. Popham, Classroom Assessment: What Teachers Need to Know, Pearson Allyn &
Bacon, 1999, pp.222-227.
[21] H.F Schmidi and D. Volke, "Shift of meaning and students’ alternative concepts", International Journal in Science Education, 25(11), November, 2003, pp. 1409-1424.
[22] P.C. Sue, J.F. Weng, J.M. Su, and S.S. Tseng. “A New Approach for Constructing the Concept Map,” IEEE International Conference on Advanced Learning Technologies, ICALT 2004.
[23] E. Triantafllou, A. Pomportsis, and S. Demetriadis, “The Design And The Formative Assessment of An Adaptive Educational System Based on Cognitive Styles”, Computers
& Education, 41(1), August, 2003, pp. 87-103.
[24] C.J. Tsai, S.S. Tseng, and C.Y. Lin, "A Two-Phase Fuzzy Mining and Learning Algorithm for Adaptive Learning Environment", Proceedings of International Conference on Computational Science (ICCS'2001), Lecture Notes in Computer Science
(LNCS 2074), Vol. 2, pp. 429-438, CA, USA, May, 2001.