國
立
交
通
大
學
理學院網路學習學程
碩
士
論
文
建 構 數 學 學 科 概 念 效 應 關 係 圖 之 方 法
Approach for Constructing the Concept Effect Relation Map of
Mathematics
研 究 生：廖經益
建構數學學科概念效應關係圖之方法
Approach for Constructing the Concept Effect Relation Map
of Mathematics
研 究 生：廖經益 Student：Ching-Yi Liao
指導教授：曾憲雄 Advisor：Dr. Shain-Shyong Tseng
國 立 交 通 大 學
理學院網路學習學程
碩 士 論 文
A Thesis
Submitted to Degree Program of E-Learning College of Science
National Chiao Tung University in partial Fulfillment of the Requirements
for the Degree of Master
in
Degree Program of E-Learning
June 2005
建構數學學科概念效應關係圖之方法
Approach for Constructing the Concept Effect Relation Map
of Mathematics
研究生：廖經益 指導教授：曾憲雄 博士
國 立 交 通 大 學
理學院網路學習學程
摘要
測驗理論是一種解釋測驗資料間實證關係的系統化理論學說。當代測驗理論主要是以試題反應理論(IRT：Item Response Theory)為架構，考慮試題參數及受試者的反應等特
性(包括難度、鑑別度、學生能力等)，因此在估計受試者個人能力時，能夠提供一個較
精確的估計值。
以資料探勘的技術來架構概念效應關係圖(CERM：Concept Effect Relation Map)，若
透過不成熟的資料前處裡將導致：(1)概念模糊化結果的單調性，(2)所探勘的關聯規則
無法反映實際的概念效應關係及(3)產生循環迴圈的關聯規則。本文應用 IRT 來處裡學生
概念學習的反應結果。因此，學生概念學習的反應結果加上了試題難度、鑑別度的考量
下，我們提出一個基於 IRT 資料前處裡的概念效應關係圖架構系統(IRT-Based Data
基於 IRT 資料前處裡的概念效應關係圖架構系統包含資料前處裡與資料探勘兩個 模組。資料前處裡模組內含四個程序：試題分析、產生學習反應指標、概念分解/整合及 概念學習反應指標整合結果的糢糊化；資料探勘模組內含關聯規則探勘與概念效應關係 圖架構兩個程序。 我們所提出的方法透過實驗的結果證明，概念效應關係圖的效應關係可以被改進且 可以減少循環迴圈關聯規則的產生。 關鍵字:試題反應理論，概念效應關係圖，教學策略，概念同化效應，迷思概念，學習診斷。
Approach for Constructing the Concept Effect Relation Map
of Mathematics
Student: Liao Ching-Yi Advisor: Dr. Shian-Shyong Tseng
Degree Program of E-Learning College of Science National Chiao Tung University
Abstract
Test theory is an explanation of empirical relationships among examination data. The
modern test theory is based on the Item Response Theory (IRT), which considers the
parameters of test item and the response of test-receiver (including difficulty, discrimination,
ability and so on),and the estimation of its test-receiver’s ability becomes more precise.
Concept Effect Relation Map (CERM) constructed by data mining with naïve data
preprocessing causes: (1) monotonous concept fuzzification result, (2) the association rules
may not reflect the real concept relation and (3) the circulating association rules exit. In this
thesis, we apply the IRT as the assessment of students’ concept learning response. With the
consideration of the difficulty and the discrimination of test item, we propose an IRT-Based
Data Preprocessing Concept Effect Relation Map Construction System.
IRT-Based Data Preprocessing Concept Effect Relation Map Construction System
includes two modules: the Data Preprocessing Module and the Data Mining Module. The
Concept Decomposition/Aggregation and Fuzzy ACLR Generator, and the latter has two
procedures: Association rule mining and concept map constructor.
The experiment results of the proposed Approach show that the CERM construction can
be improved and the number of circulated association rules generated can be reduced.
Key Words: Item Response Theory, Concept Effect Relation Map, teaching strategy, concept assimilation,
致謝
網路學習碩士雖然是我第二個碩士學位，但卻一點也不輕鬆。回顧過去攻讀的第一 個碩士學位，研讀的時間雖多，卻欠缺解決問題的脈絡與思路，而今在職念碩士專班， 時間的分割雖然不易掌握，也許過去的人生閱歷，反而讓我學習更多。研究的過程不斷 地印證或修正自己解決問題的策略與想法，學習因而更加踏實，凡此種種都要感謝論文 指導教授曾憲雄博士，在問題研究與論文寫作上所給予的督導與幫助。同時亦感謝論文 催生的推手--博士班翁瑞峰學長，從黃昏到晨曦，無怨無悔的陪我討論與修正論文，由 於他不斷的鼓勵與鞭策，使得此篇論文得以順利進行迅速完成。論文口試令人期待又怕受傷害，所幸在平時的 group meeting 與 outside meeting，有
勞 KDE Lab.各位學長、同學與學弟妹提問所給予的建議與磨鍊，讓我在口試當天得以 臨危不亂，從容不迫地掌握主題侃侃而談，更感謝口試委員：莊祚敏教授、葉耀明教授 與曾秋蓉教授能撥冗參與給予指教。整個口試的前置作業，誠摯地感謝雅惠助理與怡靜 同學的幫忙與支援，使得整個口試的活動得以順利圓滿達成。 在論文寫作期間，由於身兼內壢高中註冊組長，繁忙業務有勞註冊組組員—張錦 玲、陳世芬與楊昱芳為我分擔解憂。最後要感謝我的愛妻許瓊華，不僅是我的精神支柱 還得擔任我的私人司機，在我疲憊不堪時接送我往返中壢與新竹之間，成為我不可獲缺 的後勤支援。希望本論文能對相關領域議題的研究有所幫助與啟發，使此論文的存在發 揮其價值與貢獻，這是我最深切的期盼。感謝閱讀此篇論文的您與所有幫助過我的人。
Table of Contents
摘要 ...i
Abstract ...iii
致謝 ...v
Table of Contents ...vi
List of Definitions ...vii
List of Examples ...viii
List of Algorithms ...ix
List of Figures ... x
List of Tables ...xi
1.Introduction ... 1
2.Related Work ... 4
3.IRT-Based Data Preprocessing Approach ... 9
3.1 IRT-Based Concept Learning Response Function ... 10
3.2 System Architecture ... 16
4.IRT-Based Data Preprocessing CERM Construction System ... 20
4.1 Data Preprocessing Module... 20
4.2 Data Mining Module ... 34
5.Experiment ... 38
6.Conclusion ... 42
List of Definitions
Definition 1 Fuzzy Membership Function of Learning Response... 13
Definition 2 Student’s Learning Ability... 22
Definition 3 Difficulty and Discrimination of Test Item... 23
Definition 4 The Learning Response Index of the Test Item ... 24
Definition 5 The Weight Concept Learning Response Index ... 27
List of Examples
Example 1: Measures Function Of LRI ... 15
Example 2: Student’s Learning Ability ... 22
Example 3: Student’s Learning Response Index ... 24
Example 4: Test Item - Concept Relativity Table (ICRT)... 25
Example 5: Weight Concept Learning Response... 27
Example 6: Aggregation Concept Learning Response index (ACLR)... 29
Example 7: Fuzzy ACLR Generator... 32
List of Algorithms
List of Figures
Fig. 1 The difficulty effect of IRT-Based CLR Function... 10
Fig. 2 The discrimination effect of IRT-Based CLR Function... 11
Fig. 3 The curve of the Fuzzy Learning Response Membership Function... 13
Fig. 4 The difficulty effect in the Fuzzy Learning Response Membership Function ... 14
Fig. 5 The discrimination effect in the Fuzzy Learning Response Membership Function ... 14
Fig. 6 IRT-Based Data Preprocessing Concept Effect Relation Map Construction System ... 16
Fig. 7 ACLR index is aggregated from item’s WLRI... 28
Fig. 8 The given membership functions of students’ numeric ACLR... 31
Fig. 9 Process of Apriori algorithm ... 36
Fig. 10 H-H type CERM constructed with and without IRT-Based Data preprocessing.39 Fig. 11 L-L type CERM constructed with and without IRT-Based Data preprocessing.. 40
Fig. 12 L-H and H-L type CERM constructed with and without IRT-Based Data preprocessing. ... 41
List of Tables
Table 1 relative frequencies of skill ... 6
Table 2Testing Result Table (TRT) ... 21
Table 3 Test Item-Concept Relation Table (ICRT) ... 26
Table 4 The WLRI and the ACLR of concepts ... 29
Table 5 The Students’ ACLR of each concept ... 32
Table 6 The degree value of ACLR translated by the LOW /HIGH fuzzy membership functions ... 32
Table 7 The Students’ fuzzy SCCI of each concept... 33
Table 8 The Mining Results (Confidence ≥0.6)... 36
Table 9 The scenario explanation of association rule ... 37
Table 10 Statistics of the experiment. ... 38
1. Introduction
In the last few years, the technology of Internet and database has been improved rapidly.
There are a lot of teaching activities adopt the e-learning as the way of teaching. Therefore,
the substitution of traditional teaching by e-learning becomes a significant trend. Among the
change of teaching manner, the approach of learning assessment will then be affected
inevitably. Therefore, the analysis of assessment in e-learning becomes an important issue.
With the transformation from traditional pen paper examination into the on-line
examination[7], there are many researches in the assessment of e-learning. As we know, the
testing records are useful in analyzing the learning status of students’, e.g. analyzing student’s
concept effect relations [13]. The results of assessment could provide the suggestion of
teaching strategy and learning guidance[5].
Test theory is an explanation of empirical relationships among examination data. There
are two main developments: one is the classical test theory, which is based on the true score
model. That is, observation score is the sum of the real score and the erroneous score; the
other is the modern test theory, which is based on the Item Response Theory (IRT)[2][12].
Since IRT considers the parameters of test item and the response of test-receiver (including
difficulty, discrimination, ability and so on), the estimation of its test-receiver’s ability
give different ability estimation of the test-receiver.
Data mining approach is one of the assessment of learning diagnosis analysis, which
usually mines the raw data directly from the students’ testing result [16][22][24]. Thus the
results of analyzing the testing record directly will not be able to response the students’
learning status properly without considering the difficulty and discrimination of the test item.
Furthermore, it may result inefficient diagnosis. To improve such situation, we propose an
IRT-Based Data Preprocessing Approach to construct the Concept Effect Relation Map
based upon the Item Response Theory (IRT). With the approach mentioned above, the testing
record is firstly preprocessed before data mining with the consideration of item’s difficulty
and discrimination, thus the IRT-Based Data Preprocessing Approach can rectify the testing
records for analysis to represent the fact of students’ concept learning status. Moreover, we
further visualize the effect relationships among concepts into Concept Effect Relation Map
(CERM) in order to make the analysis more effective, and hence promote the consulting value
of suggestion to the students’ learning diagnosis and the teachers’ teaching strategy.
IRT-Based Data Preprocessing Concept Effect Relation Map Construction System
includes two modules: the Data Preprocessing Module and the Data Mining Module. The
former has four procedures: (1) the Test Item Analysis, which calculates the difficulty and the
discrimination of each item from students’ testing result; (2) generates the item’s Learning
Response Theory; (3) the concept decomposition and aggregation. Accordingly, the
Item–Concept Relationship Table (ICRT) separates the concept with relativity weight of the
test item. After the concept decomposition, the Sugeno Fuzzy Measure Function aggregates
the dissociated concept with an attribute value of Weight Learning Response Index (WLRI).
The aggregate valve of WLRI is an attribute value of concept called the Aggregated Concept
Learning Response (ACLR); (4) transforms the ACLR from numeric into symbolic H/L by
the Fuzzy membership function. The latter has two procedures: firstly, the Apriori Algorithm
of data mining [11] mines the association rules from the Fuzzification of ACLR. Secondly,
the scenario explanation of the mining association rule is proposed to construct CERM.
The result of the IRT-Based Data Preprocessing Approach makes the CERM much more
reasonable, which is helpful for the diagnosis of student’s learning problems, and teachers in
adjusting their teaching strategy. The main contributions of this thesis are:
(1) The proposed Approach refines the assessment of concept learning response.
(2) The IRT-Based Measure Function is defined to quantify the learning status of
concept. With the consideration of item difficulty and discrimination, the bias of the
learning response is reduced.
(3) The number of circulated association rules generated can be reduced by the proposed
2. Related Work
The theory and model of students’ cognition often conflicts with the theory and model of
science[21]. Since the student often develops individual scientific concept by experiences
through their consciousness. Even if they are able to answer correctly in the examination after
the teaching of scientific curriculum, only little of the mis-concept can be revised (Strike &
Posner, 1985). Concept mapping is a strategy to visualize the learners’ intermediate
concept[1][5][8]. While elaborating Novak’s Concept Mapping, Anderson(1995) pointed out
that concept mapping is quite a good dialectical method in mis-concept. That is, the student
may reorganize and describe the hierarchy of concepts by the approach of concept
mapping[6][18].
With the development of e-learning, the technology of assessment grows[19], too, on line
assessment can be also take place by Internet[7]. The effect relations among concepts can be
constructed by the analysis of assessment result[15][21][24], such as assimilation effect and
mis-concept effect[23]. Simultaneously, the concept mapping can be the graphical
representation of learner’s learning result, which indicates the connection (link) among the
knowledge or concepts. Diagnosis with the assessment result[3] can improve students’
mentioned above, the assessment analysis and the concept mapping representation of the
analysis result[9][10] have thus become an important issue of e-learning.
Test theory is an explanation of empirical relationships among examination data, which
develops into two big schools of thought: (1) one is the classical test theory, which is based on
the true score model (Gullikson, 1987; Lord & Novick, 1968). That is, observation score is
the sum of the real score and the erroneous score; (2) other one is the modern test theory
(Hambleton & Swaminathan, 1985; Hambleton, Swaminathan, & Rogers, 1991; Hulin,
Drasgow, & Parsons, 1983; Lord, 1980), which is based on the Item Response Theory
(IRT)[12]. The Item Response Theory considers the parameters of test item and the response
of test-receiver (including difficulty, discrimination, ability and so on). IRT may give
precisely different ability estimation to the test-receiver while regarding the same primitive
scores.
To model the learning effect relationships among concepts, Hsu [15] proposed a Concept
Effect Relationships (CER) as a conceptual map-based notation. In brief, if C is the i
prerequisite of concept C for efficiently learning, then a CER j C Æ i C exists. A single j
concept may have multiple prerequisite concepts, and can also be a prerequisite concept of
multiple concepts. Thus, based upon CER, the learning guidance of necessary concepts to
Appleby [4] proposed an approach to create the potential links among skills in
Mathematics domain. The direction of a link is determined by a combination of educational
judgment, the relative difficulty of skills, and the relative values of cross-frequencies.
Moreover, a harder skill should not be linked forwards to an easier skill. As shown in Table 1,
AB
f represents the amount of learners with wrong answers of skill A and right answers of
skill B. If fAB> fAB, a skill A could be linked to a harder skill B, but backward link is not
permitted.
Table 1 relative frequencies of skill Answer right Answer wrong
Answer right fAB fAB
Answer wrong fAB fAB
Later, based upon statistical prediction and approach of Hsu [16], a CER Builder was
proposed by Hwang [14]. Firstly, CER Builder finds the test item that most students failed to
answer correctly and then collects the other test items, which were failed to answer by the
same students. Thus, CER Builder can use the information to determine the relationships
among the test items. Though the CER Builder can find the tutoring path of low learning
achievement students, which may be not easy to find out from high learning achievement
students, and the pattern of mis-concept from the test. Moreover, mining the testing result
directly without the consideration of item’s difficulty and discrimination might cause
monotone or circulating result in association rules mining.
A B
Tsai [24] proposed a Two-Phase Fuzzy Mining and Learning Algorithm. In the first
phase, Look Ahead Fuzzy Mining Association Rule Algorithm (LFMAlg) was proposed to
find the embedded association rules from the historical learning records of students. In the
second phase, the AQR algorithm is applied to find the mis-concept map indicating the
missing concepts during students learning.The obtained mis-concept map as recommendation
can be fed back to teachers for remedy learning of students. However, because the creating
mis-concept map, which is not a complete concept map of a course, only represents the
missing learning concepts, its usefulness and flexibility are decreased. In addition, their
approaches generate many noisy rules and only use single rule type to analyze the prerequisite
relationship among learning concepts.
Sue [22] proposed a Two-Phase Concept Map Construction (TP-CMC) algorithm to
automatically construct a concept map of a course by historical testing records. In the data
preprocessing, Item Analysis with Norm-Referencing is applied to refine the mining result of
grade fuzzy association rules. The Concept Map Constructing (CMC) Algorithm is proposed
to be the post processing of the map construction. However, Item Analysis with
Norm-Referencing as the data preprocessing still can’t get better performance of the map
construction.
1. Without the consideration of test items’ difficulty and discrimination, naïve data preprocessing may cause the monotonous concept fuzzification result.
2. Without the consideration of test items’ difficulty and discrimination, naïve data preprocessing may cause the result of circulated association rule.
3. Mining with naïve data preprocessing may not reflect the physical effect relations of concepts.
3. IRT-Based Data Preprocessing Approach
For solving the problems of issue mentioned above, we apply Item Response Theory to
indicate the status of concepts’ learning. The Item Response Theory considers the parameters
of test item and the response of test-receiver (including difficulty, discrimination, ability and
so on). Therefore the estimation of its test-receiver’s ability becomes more precise. Moreover,
while regarding the same primitive scores, IRT may also give different ability estimation of
the test-receiver.
In order to find out the cognition sequence relations among the concepts, including
assimilation effect and mis-concept relation, with the consideration of difficulty and
discrimination of test item, we propose an IRT-Based Data Preprocessing Concept Effect
Relation Map Construction System to construct the Concept Effect Relation Map, with
influence weights and effect relations among learning concepts of a course.
With the consideration of difficulty and discrimination of the test item, the status of
concept learning is quantified by the Learning Response Index, which creates a fuzzy
membership function between poor learning (0) and well learning (1). Thus LRI rectifies the
bias of the testing result, which may cause the circulation or monotone effects among the
association rules. By the rectification based upon IRT, multiple association rule types were
mined and the hidden strategy of teaching is discovered. Hence applying LRI expands the
3.1 IRT-Based Concept Learning Response Function
The quality of CERM construction deeply depends on the method of data preprocessing
before data mining. For example, ratio of incorrect/correct answers is a naïve way of data
preprocessing to represent the learning status of concept. However, the ratio value mentioned
above may be affected by the difficulty and discrimination of the test item. In order to
eliminate the bias affect of concept learning response cause by the difficulty and
discrimination, we propose an IRT-Based Concept Learning Response (CLR) Function.
Easy Difficult Easy Difficult
1 T C CT2 C Score T3 CT1 CT2 C ScoreT3 S1
2 3 3
2 S1 0 0.53 0.44 0.97 S23
3
2
2 IRT-Based CLR Function S2 0.24 0.53 0 0.77 Fig. 1 The difficulty effect of IRT-Based CLR FunctionIn Fig. 1, Two students S1 and S2 are tested by three items CT1, CT2andC T3 concerning the same concepts. As we can see, S2 has answered the CT1 correctly but S1 doesn’t. Contrarily, S1 has answered the C correctly but S2 doesn’t. Identically, both S1 T3 and S2 have answered CT2 correctly. On the left side of Fig. 1, we may say the “weight” of learning response is the same in CT1, CT2 and C . In other words, since S1 and S2 have T3 the same score = 2, without the consideration of difficulty, it is hard to distinguish the
learning status of the two students in learning concept C. But the situation changes while we
the difficulty ofCT3 is higher than CT1. Totally, S1’s CLR=0.97 is higher than S2’s CLR=0.77, and we are able to say that S1 has learned the concept better than S2.
Low High Low High
1 T C CT2 C Score T3 CT1 CT2 C ScoreT3 S1
2 3 3
2 S1 0 0.53 0.67 1.20 S23
3
2
2 IRT-Based CLR Function S2 0.17 0.53 0 0.70 Fig. 2 The discrimination effect of IRT-Based CLR FunctionIn Fig. 2, Two students S1 and S2 are tested by three items CT1, CT2andC T3
concerning the same concepts. As we can see, S2 has answered the CT1 correctly but S1 doesn’t. Contrarily, S1 has answered the C correctly but S2 doesn’t. Identically, both S1 T3 and S2 have answered CT2 correctly. On the left side of Fig. 2, we may say the “difference” of the learning response is the same in CT1, CT2 and C . In other words, since S1 and S2 T3 have the same score = 2, without the consideration of discrimination, it is hard to distinguish
the learning status of the two students in learning concept C. Again the situation changes
while we apply the IRT-Based CLR Function. That is, S1 would have higher CLR than S2 has.
Since the “difference” of CLR in concept C is larger than the “difference” of CLR in T3 concept CT1. Finally, S1’s CLR=1.20 is higher than S2’s CLR=0.70, Thus we are able to say that S1 has learned the concept better tha S2.
As mentioned above, it is necessary to have a fuzzy membership function of learning
Fuzzy Learning Response Membership Function, difficulty and discrimination of test item
should be considered. Meanwhile, the Fuzzy Learning Response Membership Function must
have the characteristics below:
1. Positive relative to the difficulty of the test item.
2. Positive relative to the discrimination of the test item.
With the goals and characteristics mentioned above, we consider the Two-parameter
Logistic Model function
1.7 ( ) 1 ( ) 1 D x P P x e− − = + , e =2.719
of the Item Response Theory (IRT). Originally, IRT is used to estimate the aptness of the test
item. The ability x of the student is the variable of the Logistic function, which includes two
parameters, difficulty P and discrimination D. The aptness of the test item is indicated by
distribution of the answering probabilities of different abilities of students.
We adopted the Two-parameter Logistic model as our Fuzzy Membership Function of
learning response to indicate the student’s learning status responded from the testing items
with the consideration of difficulty and discrimination. The definition of the Fuzzy Learning
( )i LRI x i x 1 0 Lea rn ing Re sp on se In de x Learning Ability 3 -3
Definition 1 Fuzzy Membership Function of Learning Response
We define the Fuzzy Learning Response Membership Function with two parameters, the difficulty and discrimination of the item, and the variable of the function is the ability of the student. The Fuzzy Learning Response Membership Function is denoted as
( , ) 1.7 ( - ) 1 ( ) 1 j j j i j P D i D x P LRI x e− = + ,
where xi: the learning ability of the student S , i j
P : the difficulty of the test item T , j
j
D : the discrimination of the test item T . j
The graph of the Fuzzy Learning Response Membership Function is shown in Fig. 3.
Fig. 3 The curve of the Fuzzy Learning Response Membership Function
The difficulty effect in the Fuzzy Learning Response Membership Function is shown In
Fig. 4. With the same learning ability and discrimination given, the difficulty of the test item
decreases cause the function’s curve shift to the right, thus the student’s LRI decreases as the
1 0 1 x x2 Learni ng Res ponse Index Learning Ability 1 LRI ∆ 2 LRI ∆ 3 -3 1 0 1 LRI 2 LRI x Learning Res p onse Index Learning ability 3 -3
Fig. 4 The difficulty effect in the Fuzzy Learning Response Membership Function
Fig. 5 shows the discrimination effect in the Fuzzy Learning Response Membership
Function. The same difficulty of the test item and the difference of students’ ability were
given; the curvature of the function increases while the discrimination of the test item
increases. Thus the difference of the LRI increases (D2 >D1⇒ ∆LRI2 > ∆LRI1).
Fig. 5 The discrimination effect in the Fuzzy Learning Response Membership Function
The Fuzzy Learning Response Membership Function includes the parameter of difficult
and discrimination. It is useful in indicating the actual degree of wellness in concepts learning
Example 1: Measures Function Of LRI
If a student has correctly answered a test item, based on the result of the testing, we say
that the student learns well but no further information about how well it is. With the
same situation, suppose the difficulty and the discrimination of the item is 0.813 and
0.375, respectively. If the student with learning ability of 1.8, we would have the
following LRI to indicate the learning performance of the student.
1.7 0.375(1.8-0.813) 1 (1.8) =0.65 1 LRI e− × = +
3.2 System Architecture
The Concept Effect Relation Map of a course is quite useful as mentioned above.
However, mining with naïve data preprocessing may not reflect the physical effect relations of
concepts. Therefore, in this thesis, we propose an IRT-Based Data preprocessing approach to
construct the Concept Effect Relation Map, which is a map of directional graph with influence
weights among cognition learning concepts of a course. Fig.7 shows the IRT-Based Data
Preprocessing Concept Effect Relation Map Construction System with two modules: Data
Preprocessing Module and Data Mining Module.
Fig. 6 IRT-Based Data Preprocessing Concept Effect Relation Map Construction System
In the module of Data Preprocessing, four procedures are held: Test Item Analysis, LRI
Generation, Concept Decomposition/Aggregation and ACLR Fuzzification. In the Test Item
Analysis procedure, Instruction Theory is applied to generate the difficulty and discrimination
of test item and define the learning ability of the student. In the second procedure, Item
Response Theory is applied to LRI in order to indicate the students’ learning status responded
Test Item Analyzer
LRI Generator
Data Preprocessing Module
Concept Map Constructor
Concept Effect Relation Map Associatio n Rule Mining Testing Records Database Concept decomposition/ Aggregation Fuzzy ACLR Generator Item-Concept Relationship Table A C B E D
from the test items. Concept Decomposition/Aggregation is the third procedure, which the
Item-Concept Relationship Table is applied in concepts decomposition from items with
Weight Learning Response Index (WLRI) and each concept has the attribute value called
Aggregation Concept Learning Response index (ACLR) after the aggregation of the same
concept separate in different items. The final procedure is the fuzzification of ACLR, where
Fuzzy Theory is applied in transforming the numeric ACLR into symbolic “H” and ”L” to
indicate well learning and poor learning, respectively.
The second module, Data Mining Module, has two procedures. In the former, applying
Apriori Algorithm of data mining discovers four association rule types, L-L, L-H, H-H and
H-L. In the latter, CERM is constructed based upon the scenario explanation of the mining
association rule we proposed.
Base upon the historical testing records of students, we are able to preprocess the testing
records with IRT-Based. Later, the embedded association rules are discovered by Data mining
process. Finally, the procedure of Concept Effect Relation Map Construction Generates the
Concept Effect Relation Map by scenario explanation of association rules. The procedures of
1) Data Preprocessing Module
y Test Item Analyzer:
Difficulty and discrimination of the test item are analyzed, and the students’ score are
normalized by the normal reference as the relative learning ability of the students’.
y LRI Generator:
We adopt the two-parameter Logistic model function of Item Response Theory as our
Learning Response Index of item (LRI), where the difficulty and discrimination of the
test item is the parameters of our Learning Response measure Function, and the relative
learning ability of the student is the variable of the function. Each test item answer by a
student will have a value of LRI with valve between 0 and 1. The LRI of the test item
individually responses the student’s learning status of the involved concepts.
y Concept Decomposition/Aggregation:
Usually, a test item may include several concepts; we separate the involved concepts of
the test item by the test Item Concept Relationship Table (ICRT). Also, Concept may be
involved in several test items. Concepts included in each test item can be separated by
weight according to the entries of ICRT. The attributed value of decomposition concept
decomposition weight of concept and the LRI of item. Same concepts’ WCLR will be
aggregated by applying Sugeno Fuzzy Measure Function and are defined as the
Aggregation Concept Learning Response index (ACLR).
y Fuzzy ACLR Generator:
In order to mine further association rules, we translate the students’ ACLR into the
notation of “H”(Well Learning) and “L”(Poor Learning) by the Fuzzy membership
function.
2). Data Mining Module
y Association Rule Mining
The association rules are mined from the Fuzzy ACLR by using Apriori Algorithm. Four
types of association rules L-L, L-H, H-L and H-H are used as the model to discover the
assimilation and mis-concept effect relations among concepts.
y Concept Map Constructor
We define the direction and the weight of edge by the effect relationship and the value of
support and confidence, respectively. Concept Effect Relation Map of the students’ is
4. IRT-Based Data Preprocessing CERM Construction
System
IRT-Based Data Preprocessing CERM Construction System includes two modules, the Data Preprocessing Module and the Data Mining Module. There are four procedures
included in the Data Preprocessing Module: Test Item Analyzer, Learning Response Index
(LRI) Generator, concept decomposition/aggregation and Fuzzy ACLR Generator. The
second Module includes two procedures: Association Rule Mining and Concept Map
Constructor.
4.1 Data Preprocessing Module
The first module has four procedures. Sequentially, the Test Item Analysis is the first
procedure, which calculates the difficulty and the discrimination of each item from students’
testing result. The Learning Response Index (LRI) is generated in the second procedure. The
LRI of each item indicates the students’ learning response base upon Item Response Theory.
The third procedure handles the concept decomposition and aggregation, while Item–Concept
Relationship Table (ICRT) is applied in concept decomposition of each item with the weight
of response and the Sugeno Fuzzy Measure Function is applied in concept aggregation with
Weight Learning Response Index (WLRI) that is dissociated by ICRT. Several WLRI of the
which indicates the concept learning status of the student. The final procedure is to transform
the ACLR from numeric into symbolic H/L by the Fuzzy membership function.
1) Test Item Analyzer
The Test Item Analyzer is the one who calculates the difficulty and the discrimination of
each item from the result of students’ testing. First of all, we build up the Testing Result Table
(TRT) according to the students’ answer sheet. Let Am n× be the matrix of TRT, the element ji
a is the answered results of the test items T , j=1,2,…,m, from students j S , i=1,2,…,n. i
The elements a =1 and ji a =0 denote the ith student having right or wrong to the jth test ji
item, respectively. Table 2 shows the example of TRT with six students tested by seven items.
Table 2Testing Result Table (TRT)
Test item Student ID 1 T T2 T3 T4 T5 T6 1 S 1 0 0 1 1 0 2 S 0 1 1 0 1 1 3 S 0 1 1 0 1 1 4 S 1 0 1 1 0 1 5 S 1 0 1 1 0 1 6 S 1 1 0 1 1 1
Definition 2 Student’s Learning Ability
We standardize the score of student S as the student’s learning ability. i
-i i X x X x S = , i=1,2,…,n,
where x : the score of students i S . i
X : the average score of the students.
1 1 n i i X x n = =
∑
SX : the standard deviation of the students’ scores.
2 1 ( - ) n i i X x X S n = =
∑
Example 2: Student’s Learning Ability
If the student has the score of 92, with the class average of 62 and standard deviation of
15, then the standardized score of the student would be 2. We use the standardized score
Definition 3 Difficulty and Discrimination of Test Item
Based upon the theory of instruction, let B and P be the set of high achievement
students (the best 27%) and low achievement students (the last 27%), respectively.
y Difficulty of Test Item T : j
2 B P j j j R R P = + , j=1,2,…,m, y Discrimination of Test Item T : j Dj =RBj -RPj , j=1,2,…,m,
where RBj : the ratio of answering right of test item T in set B. j
P j
R : the ratio of answering right of test item T in set P. j
The value of difficulty is between 0 and 1, which 0 means hard and 1 means easy of the test
item. Also, the value of discrimination is between 0 and 1, which 0 means the low
discrimination and 1 means the high discrimination of the test item.
2) LRI Generator
The difficulty and discrimination of each test item are computed after the Test Item
Analyzer. Let Tm n× be the matrix of Learning Response Mapping Table, where the student
i
S , i=1,2,…,n is column variable and the test item T , i=1,2,…,m, is the row variable. The j
Definition 4 The Learning Response Index of the Test Item
Let Tm n× be the matrix of the Learning Response Mapping Table, and t be the ji
entries Tm n× , which indicates the student’s learning status obtain by the test item. We define the Learning Response Index of the Test Item as
( j, j)( )
ji ji P D i
t =a ×LRI x , i=1,2,…,n, j=1,2,…,m,
where a is the answered results of the test item ji T of the student j S , and i
(P Dj, j)( )i
LRI x is defined in Definition 1. The value of t which is between 0 and 1 ji
denotes the student S ’s learning status response through test item i T . j
Example 3: Student’s Learning Response Index
The difficulty and discrimination of the test item are 0.813 and 0.375 as given,
respectively. If the student’s ability is 1.8 and has right answer of the test item, then the
student’s LRI through the test item is
( , ) (0.813,0.375) 1.7 0.375(1.8-0.813) 1 ( )= (1.8)= =0.65 1 j j P D i LRI x LRI e− × +
The value 0.65 indicates the learning status of the student while having the right answer
3) Concept Decomposition and Aggregation
Intuitively, a test item may include several concepts, so we have to decompose the
concepts included in a test item with the attribute called WLRI. Later, we aggregate the WLRI
of the same concept involved in several test items. The aggregated value of attribute is treated
as Aggregation Concept Learning Response index (ACLR).
i. Concept Decomposition :Item Concept Relationship Table
First of all, we decompose the concepts performed in the test item by the test Item -
Concept Relationship Table (ICRT). If the test sheet has m test items T , j=1,2... m, with p j
concepts C tested, namely k=1,2... ,p. Let k Bm p× be the matrix of ICRT, and the element jk
b is the weight between 0~1, which indicates the relativity of concept C involved in test k
item T . j
Example 4: Test Item - Concept Relativity Table (ICRT)
Table 3 shows the ICRT of seven test item include five concepts, where the concept is
the row variable and the test item is the column variable. By referring to the ICRT, we
can see that the Test item T includes Concepts 3 C 、3 C4and C with different 5
relativity weight 1, 0.3, 0.4, respectively. Simultaneously, test items T2and T have 6
6
T with the weight of relativity, 0.75, 1, 1, 0.8, respectively.
Table 3 Test Item-Concept Relation Table (ICRT) Concept Test item C1 C2 C3 C4 C5 T1 0.9 0 0.75 0 0 T2 0 0.8 1 0 0 T3 0 0 1 0.3 0.4 T4 1 0 0 0 0 T5 0 1 0 0 0 T6 0 1 0.8 0 0 T7 1 1 0 0 1
ii. Concept Aggregation
The concepts learning response of a student through each test item are given according to
Definition 5 The Weight Concept Learning Response Index
Since b and jk t are the entries of matrixes ji Bm p× and Tm n× , respectively. Recall
that Bm p× and Tm n× is the matrixes of ICRT and Learning Response Mapping Table,
respectively. Let Wn m p× × be the matrix of Weight Concept Learning Response mapping
Table (WCLRT), and the values of entries w is between 0~1. We obtain ijk w by the ijk
following.
ijk jk ji
w =b × , i=1,2,…,n, j=1,2,…,m, k=1,2,…,p, t
where w indicates the concept ijk C ’s learning status of student k S , which the i
concept C is involved in test item k T . j
Example 5: Weight Concept Learning Response
Suppose the student S has the LRI=0.6 obtain from the test item i T which involves j
concepts C1 and C2 with the weight of relativity 0.75 and 1, respectively. Then the
learning status indexes of the student concerning concept C1 and C2 are
1 1
ij j ji
Aggregation Module C1 0.64 C2 0.80 C3 0.35 T 1 C1 0.87 C3 0.69 C3 T 3 T 2 C1 0.80C2 C3 0.99 6 0.69 0.902 ACLR
Since each concept C has a set of WCLR, which is obtain from several test items j T , k
k=1,2,…,p, having concept C . As Fig. 7 shows, the test items j T1, T2 and T are 3
decomposed into several concepts, like C1, C2 and C . Next, we can see that the sets of 3
WCLR concerning concepts C1, C2 and C are {0.64,0.69}, {0.80}, {0.35,0.87,0.69}. 3
The aggregation WCLR of the set concerning concept C is defined as the Aggregation j
Concept Learning Response index (ACLR), which indicates the status of the student in
learning concept C . j
Fig. 7 ACLR index is aggregated from item’s WLRI
So far, it is important to figure out the function to achieve the aggregation mention above.
In this thesis, we apply the Sugeno fuzzy measure function as our aggregation function. In
order to set the value between 0~1, the aggregation function must satisfy the boundary
condition with (1) gλ( )φ = , (2) ( ) 10 gλ X = , and the properties, (3) if A B⊆ , then
( ) ( )
Definition 6 The Function of Aggregation Concept Learning Response index (ACLR)
The Function aggregates the element of the set Xkof WCLR concerning concept
k
C , and define the ACRL of concept Ck as
1 (1 ) -1 ( ) m ijk j i k w g X λ λ = + × =
∏
, λ=-0.97,where wijk indicates the WCLR defined in Definition 5.
Example 6: Aggregation Concept Learning Response index (ACLR)
According to the ICRT, the test items T1, T2, T3 are decomposed into {C1,C2,C3},
{C1,C3}, {C3}, respectively. The learning response index of concepts in each test item is
obtained by the product of the test items’ LRI and the weight in ICRT. Each concept
j
C has a set WLRI from several test items Tk concerning concept Cj.
Table 4 The WLRI and the ACLR of concepts
Item C1 C2 C3
T1 0.64 0.80 0.35
T2 0.69 0 0.87
T3 0 0 0.69 ACLR 0.902 0.800 0.996
Suppose student Si has such a table shown in Table 4 after testing. T1 includes the concepts of C1, C2 and C3 with the WLRI of 0.64, 0.80 and 0.35; T2 include the
concepts of C1 and C3 with the WLRI of 0.69 and 0.87, and T3 include the concepts of
C3 with the WLRI of 0.69. By looking through the columns, concept C1, C2 and C3 will
have the sets of WLRI {0.64,0.69,}, {0.80,} and {0.35,0.87,0.69}through T1, T2 and T3,
respectively. Then the ACLR of student Si concerning concepts C1, C2 and C3 are
calculated as below. i (1- 0.97 0.64)(1- 0.97 0.69) -1 ({0.64,0.69}) 0.902 -0.97 g = × × = (1- 0.97 0.80) -1 ({0.80}) 0.800 -0.97 i g = × = (1- 0.97 0.35)(1- 0.97 0.87)(1- 0.97 0.69) -1 ({0.35,0.87,0.69}) 0.996 -0.97 i g = × × × =
4) Fuzzy ACLR Generator
In order to mine association rule further, it is necessary to transform the numeric data
into symbolic data. We achieve the transformation by applying fuzzy Theory. Here we have
two membership functions shown in Fig. 8 to transform the students’ numeric ACLR into
symbolic notation. The symbolic notations obtain by the fuzzification is “L” and “H”, which
denote “Poor Learning” and “Well Learning”, respectively. Ci.L and Ci.H denote the value
obtain from the LOW Fuzzy Function and the HIGH Fuzzy Function of the concept Ci.’s
ACLR, respectively. The value of each concept’s ACLR will transform into the notation “L”
and “H” if Ci.L>Ci.H and Ci.L ≤ Ci.H, respectively. For example, if ACLR=0.65, by the
given membership functions, we have Ci.L=0.2 and Ci.H=0.66. Since Ci.L ≤ Ci.H the Fuzzy
ACLR is denoted as “H”. Completely, fuzzification of ACLR is described in Example 4.6.
ACLR 0.5 0.5 0.4 0.4 0.6 0.7 0.8 0.9 0.3 0.2 0.1 0.1 0.2 0.3 0.6 0.7 0.8 1.0 0 Degree Low High 1.0 0.9
Example 7: Fuzzy ACLR Generator
Suppose six students are tested with five concepts. Students’ ACLR of each concept
is shown in Table 5.
Table 5 The Students’ ACLR of each concept
C1 C2 C3 C4 C5 S1 1.00 0.42 0.67 0.65 0.38 S2 1.00 0.00 0.54 0.68 0.68 S3 1.00 0.30 0.55 0.53 0.77 S4 1.00 0.00 0.78 0.73 0.75 S5 1.00 0.32 0.32 0.40 0.40 S6 0.98 0.00 0.43 0.74 0.82
By the given membership functions, ACLR of each student has the degree value of
Ci.L and Ci.H. Table 6 shows the fuzzy degree values of ACLR obtain by the
LOW/HIGH fuzzy membership functions.
Table 6 The degree value of ACLR translated by the LOW /HIGH fuzzy membership functions
C1 C2 C3 C4 C5 C1.L C1.H C2.L C2.H C3.L C3.H C4.L C4.H C5.L C5.H S1 0.00 1.00 0.66 0.00 0.14 0.84 0.66 0.20 0.70 0.00 S2 0.00 1.00 1.00 0.00 0.42 0.00 0.14 0.90 0.14 0.90 S3 0.00 1.00 0.90 0.00 0.40 0.00 0.44 0.00 0.00 1.00 S4 0.00 1.00 1.00 0.00 0.00 1.00 0.04 1.00 0.00 1.00 S5 0.00 1.00 0.86 0.00 0.84 0.00 0.70 0.00 0.70 0.00 S6 0.00 1.00 1.00 0.00 0.64 0.00 0.02 1.00 0.00 1.00 Concepts SCCI Students Concept Degree Student
We then transform the numeric ACLR into the symbolic notation of “H” if Ci.L ≤ Ci.H
and “L” if Ci.L>Ci.H. The notation “H” and “L” represent the meaning of “well learning”
and “poor learning”, respectively. Table 7 shows the fuzzification result of ACLR.
Table 7 The Students’ fuzzy SCCI of each concept.
C1 C2 C3 C4 C5 S1 H L H H L S2 H L L H H S3 H L L L H S4 H L H H H S5 H L L L L S6 H L L H H Concept FACLR Student
4.2
Data Mining Module
After the fuzzification of ACLR, the symbolic data “H” and ”L” are then process by
the Data Mining Module. First, the Apriori Algorithm of data mining is adopted to discover
the association rules. Finally, CERM is constructed based on the scenario explanation of the
mined association rule.
Algorithm 1: Apriori Algorithm Symbol Definition:
α : The minimum support threshold in the A -large itemset. CA: Candidate itemset of size A .
A
L : Frequent itemset of size A .
λ : The minimum confidence threshold.
Input:
The FACLR of students.
The threshold of minimum support α . The threshold of minimum confidence λ.
Output : The association rules of FACLR of students. A
L = {frequent items}; for ( A = 1; LA=φ; A ++) do begin
+1
CA = candidates generated from LA; for each transaction t in database,
do increment the count of all candidates in CA+1, that are contained inCA+1 1
LA+ = candidates in CA+1 with min_support end
1) Association Rule Mining
We mine the association rules from the Fuzzy ACLR by using Apriori Algorithm of data
mining. Four types of association rules L-L, L-H, H-L and H-H are used as the model to
discover the assimilation and mis-concept effect relations among concepts.
Example 8: Apriori Association Rule Mining Algorithm
From the data shown in Table 7, Fig. 9 shows the process of mining the association rules
by Apriori algorithm with minimum support 0.6 and minimum confidence 0.6.
The 1 -Candidate itemset
Itemset Sup. count
1H C 9 1 0 10 4 3H C 2L C 2H C 1L C 3L C 8 4 6 6 2 5L C 5H C 4L C 4H C
The 2 -Candidate itemset
Itemset Sup. count
8 6 6 9 6 5 2 6 5 5 1 2 {CH,CL} 1 3 {CH,CL} 1 4 {CH,CH} 1 5 {CH,CH} 2 3 {CL,CL} 2 4 {CL,CH} 2 5 {CL,CH} 3 4 {CL,CH} 3 5 {CL,CH} 4 5 {CH,CH}
The 1 -large itemset Confidence=0.6
Itemset Sup. count
1H C 9 10 2L C 3L C 8 6 6 5H C 4H C
The 2 -large itemset Confidence=0.6
Itemset Sup. count 9 1 2 {CH,CL} 6 6 1 3 {CH,CL} 1 4 {CH,CH} 8 6 1 5 {CH,CH} 2 3 {CL,CL} 6 2 5 {CL,CH}
The 3 -Candidate itemset
Itemset Sup. count
2 7 5 6 2 4 5 5 5 2 1 2 3 {CH,CL,CL} 1 2 4 {CH,CL,CH} 1 2 5 {CH,CL,CH} 1 3 4 {CH,CL,CH} 2 3 4 {CL,CL,CH} 1 3 5 {CH,CL,CH} 2 3 5 {CL,CL,CH} 1 4 5 {CH,CH,CH} 2 4 5 {CL,CH,CH} 3 4 5 {CL,CH,CH}
The 3 -large itemset Confidence=0.6
Itemset Sup. count
1 2 3
{CH,CL,CL} 6
1 2 5
Fig. 9 Process of Apriori algorithm
Table 8 shows the Association Rule mining with minimum support 0.6 and minimum
confidence 0.6 generated from large 2 itemset into L-L, L-H, H-H, and H-L types. The
Confidence is used to indicate the important degree of ith mined association rule. For
example, the Confidence of rule C2L→C3L can be obtained as follows.
2 3 2 3 2 support _ ({ L, L}) L L : 0.67 support _ ({ L}) count C C C C Confidence count C → = =
Table 8 The Mining Results (Confidence ≥0.6) The Large 2 Itemset
Rule Types Mined Rules Confidence
1H 4H C →C 0.60 HÆH 1H 5H C →C 0.80 1H 2L C →C 0.90 HÆL 1H 3L C →C 0.60 LÆH C2L→C5H 0.78 LÆL C2L→C3L 0.67
2) Concept Map Constructor
We define the direction by the effect relationship, and the weight of edge indicates the
influent probability defined by the order pairs, support and confidence. According to the
above concept effect direction and weight, Concept Effect Relation Map of the students’ is
constructed based upon the scenario explanation of the association rule shown in Table 9.
Table 9 The scenario explanation of association rule
Association Rule Concept Effect Relation 1H 2H C →C 1 2 C →C , 1
C is the prior concept of C2with support value higher then 2 C . 1L 2L C →C Assimilation (Positive related) C2 →C1 , 2
C is the prior concept of C1 with support value lower then 1 C . 1L 2H C →C 2 1 C →C , 2
C is the alternative concept of C1 with higher confidence value. 1H 2L C →C Misconception (Negative related) C1 →C2, 1
C is the alternative concept of C2with higher confidence value.
5. Experiment
In this thesis, we applied the IRT-Based Data Preprocessing Concept Effect Relation
Map Construction System in Mathematics to evaluate its effectiveness. The experiment is
based upon Table 10, which is the basic data of the experiments in Mathematics, this chapter
describes the experiment in detail.
Table 10 Statistics of the experiment.
Course Mathematics
School Senior High School
Grade K-11
Number of students 42
Average score 62.36
Number of test item 32
Standard deviation of scores 15.43 Average difficulty of the test items 0.535 Difficulty range of the test items 0.125~0.938 Average discrimination of the test items 0.107 Discrimination range of the test items -0.5~0.75
The experiment was based on Mathematics tests administered at a senior high school.
There are 42 students participated in the experiment, and their average test score was 62.36,
while the average discrimination level of the test items is 0.107. Table 11 list the notation of
concepts included in the test sheet. In this experiment, the weight of concepts included in
ICRT would be set to 1 to simplify our discussion. Moreover, the fuzzification threshold is set
C1 1.00 C6 0.56 C3 0.89 C2 0.81 C7 0.61 (0.64, 0.96) (0.98, 0.98) (1.00, 0.64) (1.00, 0.98) (1.00 , 0.98) (0.64 , 0.63) C10 0.95 C9 0.74 C6 0.65 C3 0.73 (0.95, 0.93) (0.95, 0.90) (0.93, 0.74) (0.76, 0.97) (0.95 , 0.75) (0.88 , 0.92) C11 0.88 (0.95, 0.78) (0.88, 0.78) (0.74, 1.00)
Table 11 Notation of concepts included in the test sheet.
Concept Notation Concept
C1 Spatial Relation of Points, Lines and Planes
C2 Logical Concept
C3 Symmetrical Point
C4 Distance of Two Point
C5 Trigonometric function
C6 Cosine Theorem
C7 Angle of Two Intersections Planes C8 Coordinates Reference of Spatial Object
C9 Perpendicular Point
C10 Equations of Coordinate Planes C11 Three Perpendicular Lines Theorem
As shown in Fig. 10, Fig. 11 and Fig. 12, the left side of the Figure shows the data
preprocessing with IRT-Based, while the right side of Figure shows the data preprocessing
with correct answering ratio which is un based on IRT.
Fig. 10 H-H type CERM constructed with and without IRT-Based Data preprocessing.
Fig. 10 shows the H-H type CERM. Attention to the un IRT-Based CERM in Fig.11, the
C5 0.58 C8 0.27 C3 0.89 C11 0.45 C7 0.61 (0.93, 0.90) (0.83 , 0.66) (0.93 , 0.92) (0.88 , 0.95) (0.93, 0.90) (0.93 , 0.92) (0.93 , 1.00)
one is C3¼C6¼C11¼C9 ¼C3. As the experiment shows, based on the same support and
confidence, the CERM construction with IRT-Based data preprocessing is better and
reasonable than the one without IRT-Based data preprocessing.
Fig. 11 L-L type CERM constructed with and without IRT-Based Data preprocessing.
Fig. 11 shows the L-L type CERM. The concept at the bottom of L-L type CERM means
much more difficult than other. For example, in the left part of Fig. 11,if C3 is not well
learning by student, the key problem in learning C3 is a lack of understanding of concepts C7,
C8 and C11, so the student should learn concepts C7, C8 and C11 before learning C3. The left
CERM suggests the learning strategy of C3: instead of learning C3 repeatedly, the learning of
C11, C8 and C7 has to be firstly enhanced.
Fig. 12 shows the L-H & H-L type CERM, which indicate the mis-concept effect among
concepts. The mis-concept effect may be caused by misunderstanding of concept, confuse
among concept, etc. As shown in the left of Fig. 12, the concept set {C1,C2,C3,C7}is the C5 0.45 C1 0.47 C4 0.52 C8 0.45 (1.00, 0.67) (1.00 , 0.74) (0.60 , 1.00)
C1 1.00 C3 0.89 C2 0.81 C7 0.61 C8 0.27 C10 0.41 C11 0.45 C9 0.33 C9 0.74 C10 0.95 C11 0.88 C3 0.61 C8 0.27 C4 0.52 C8 0.27 C5 0.45 C6 0.65
alternative concept of the concept set {C8,C9,C10,C11}. The CERM construction with
IRT-Based Data preprocessing generates more Association rules than the one without
IRT-Based Data preprocessing.
Fig. 12 L-H and H-L type CERM constructed with and without IRT-Based Data preprocessing.
6. Conclusion
The assessment analysis and the concept mapping representation of the analysis result
have become an important issue of e-learning. The result of the assessment can be analyzed to
discover effect relations among concepts, such as assimilation effect and mis-concept effect.
Diagnoses by analyzing the result of assessment can improve students’ learning status, and the
teaching while tutoring.
Concept Effect Relation Map (CERM) constructed by data mining with naïve data
preprocessing causes monotonous concept fuzzification result. At the same time, the
circulating association rules exit, and the association rules may not reflect the concept relation
physically. In this thesis, we propose an IRT-Based Data Preprocessing Concept Effect
Relation Map Construction System with the consideration of the difficulty and the
discrimination of test item,
IRT-Based Data Preprocessing Concept Effect Relation Map Construction System
includes two modules: the Data Preprocessing Module and the Data Mining Module. The first
module has four procedures: Test Item Analysis, Learning Response Index (LRI) Generator,
Concept Decomposition/Aggregation and Fuzzy ACLR Generator. The second module is
called Data Mining Module with two procedures. Association rule mining and concept map
The experiment of the proposed approach shows the improvement of constructing CERM
and the reduction of the circulated association rules’ number generated. The main
contributions of this thesis are:
(1) The IRT-Based Data preprocessing Approach we proposed refines the assessment of
concept learning response.
(2) Based upon the Item Response Theory, we define a fuzzy membership function to
quantify the learning status of concept.
(3) The experiment of the proposed Approach indicates the improvement of CERM
construction in association rules mining.
There are some interesting issues in extending the application of CERM in the nearly
future:
(1) The distribution weight of the items and concepts may affect the concept learning
response, CERM can be use to indicate the quality of the test sheet.
(2) The development of comparing technique: comparing the CERM of different
Reference
[1] Cmap Tools,IHMC, http://cmap.ihmc.us/.
[2] Item Response Theory- Theory and application, http://www.edutest.com.tw/e-irt/irt.htm
[3] Test Diagnostics, http://www.edtech.vt.edu/edtech/id/assess/diagnostics.html
[4] J. Appleby, P. Samuels, and T.T. Jones , “Diagnosis–A Knowledge-based Diagnostic
Test of Basic Mathematical Skills” , Computers & Education , 28(2), February, 1997, pp.
113-131.
[5] J.D. Barbara, R. S.Christine, B. Toni, G. Kate, P. Linda, "Concept Maps: A Stratagy to
Teach and Evaluate Critical Thinking", Journal of Nursing Education , 38(1),
January,1999, pp. 42-47.
[6] E. Bruillard & G.L. Baron, “Computer-based concept mapping: a review of a cognitive
tools for students”, International Conference on Educational Uses of Communication
and Information Technologies, Beijing, China, August 21-25, 2000, pp. 331-338.
[7] G. Frosini, B. Lazzerini, and F. Marcelloni, “Perform Automatic Exams”, Computers &
Education, 31(3), November, 1998, pp. 281-300.
[8] B.R. Gaines and M.L.G. Shaw, “Concept maps as hypermedia components”,
International Journal of Human Computer Studies, 43(3), 1995, pp. 323--361.
[9] H. Gamboa, “Designing Intelligent Tutoring Systems : A Bayesian Approach”,
Systems (ICEIS'2001), 2001, pp. 452-458.
[10] J.L. Gordon, “Creating knowledge maps by exploiting dependent relationships”,
Knowledge-Based Systems, 13(2000), pp.71-79.
[11] J. Han and M. Kamber, Data mining concepts and techniques, Morgan Kaufmann
Publishers. P230-239.
[12] R.K. Hambleton and S.H. waminathan, (1985). Item Response Theory, Kluwer-Nijhoff
Publishing, Boston, Massachusetts.
[13] G.J. Hwang, “A Conceptual Map Model For Developing Intelligent Tutoring System”,
Computers & Education, 40(3), April, 2003, pp. 217-235.
[14] G.J. Hwang, C.L. Hsiao, and C.R. Tseng, “A Computer-Assisted Approach to
Diagnosing Student Learning Problem in Science Course”, Journal of Information
Science & Engineering, 19(2), 2003, pp. 229-248.
[15] C.L. Hsiao, G.J. Hwang, and C.R. Tseng, “An interactive concept relationship
construction assisted system for learning diagnosis,” The 5th Global Chinese
Conference on Computers in Education, 2001, pp. 925-932.
[16] C.S. Hsu, S.F. Tu, and G.J. Hwang, “A Concept Inheritance Method for Learning
Diagnosis of a Network-based Testing and Assessment System”, Proceedings of The 7th
International Conference on Computer-Assisted Instructions, 1998, pp. 602-609.
Englewood Cliffs, NJ: Prentice-Hall, Inc.
[18] J.D. Novak, Learning, Creating, and Using Knowledge: Concept Maps As Facilitative
Tools in Schools and Corporations, Lawrence Erlbaum Assoc 1998.
[19] K.A. Papanikolaoua, M. Grigoriadoua, G.D. Magoulasb, and H. Kornilakisa, “Towards
New Forms of Knowledge Communication: The Adaptive Dimension of a Web-based
Learning Environment”, Computers & Education, 39(4), December, 2002, pp. 333-360.
[20] W.J. Popham, Classroom Assessment: What Teachers Need to Know, Pearson Allyn &
Bacon, 1999, pp.222-227.
[21] H.F Schmidi and D. Volke, "Shift of meaning and students’ alternative concepts",
International Journal in Science Education, 25(11), November, 2003, pp. 1409-1424.
[22] P.C. Sue, J.F. Weng, J.M. Su, and S.S. Tseng. “A New Approach for Constructing the
Concept Map,” IEEE International Conference on Advanced Learning Technologies,
ICALT 2004.
[23] E. Triantafllou, A. Pomportsis, and S. Demetriadis, “The Design And The Formative
Assessment of An Adaptive Educational System Based on Cognitive Styles”, Computers
& Education, 41(1), August, 2003, pp. 87-103.
[24] C.J. Tsai, S.S. Tseng, and C.Y. Lin, "A Two-Phase Fuzzy Mining and Learning
Algorithm for Adaptive Learning Environment", Proceedings of International
