Abstract—In this paper, we present new methods for evaluating students’ answerscripts using fuzzy numbers associated with degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students’ answerscripts are represented by triangular fuzzy numbers associated with degrees of confidence between zero and one. The arithmetic operations between theα–cuts of fuzzy numbers are used to evaluate the total mark of each student, where the value of α is between zero and one. The proposed methods can evaluate the answerscripts of students in a more flexible and more intelligent manner than the existing methods.
I. INTRODUCTION
n recent years, some methods have been presented for dealing with students’ evaluation [1]-[8], [10]-[15]. In [1], Biswas presented a fuzzy evaluation method (fem) and a generalized fuzzy evaluation method (gfem) for applying fuzzy sets in students’ answerscripts evaluation. In [2], Chang and Sun presented a method for dealing with fuzzy assessment of learning performance of junior high school students. In [3], Chen et al. presented two methods for applying fuzzy sets in students’ answerscripts evaluation. In [4], Cheng and Yang presented a method using fuzzy sets in education grading systems. In [5], Chiang and Lin presented a method for applying the fuzzy set theory for teaching assessment. In [6], Echauz and Vachtsevanos presented a fuzzy grading system. In [7], Frair presented a method for student peer evaluations based on the analytic hierarchy process (AHP) method. In [8], Kaburlasos et al. presented a software tool for computer-based testing and evaluation used in the Greek higher education system. In [10], Law built a structure model of a fuzzy education grading system and proposed an algorithm with it. He also presented a method to build membership functions of linguistic values with different weights. In [11], Ma et al. presented a fuzzy set approach for the assessment of student-centered learning. In [12], McMartin et al. used scenario assignments as assessment tools for undergraduate engineering education.
H.-Y. Wang is with the Department of Education, National Chengchi University, Taipei 116, Taiwan, R. O. C. (e-mail: [email protected]).
S.-M. Chen is with the Department of Computer Science and Information Engineering , National Taiwan University of Science and Technology, Taipei 106, Taiwan, R. O. C. (e-mail: [email protected]).
In [13], Nolan presented an expert fuzzy classification system for supporting the grading of student writing samples.
In [14], Pears et al. presented a method for student evaluation in an international collaborative project courses.
In [15], Wu presented a method based on the fuzzy set theory and item response theory to evaluate learning performance.
In this paper, we present new methods for evaluating students’ answerscripts using fuzzy numbers associated with degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students’
answerscripts are represented by triangular fuzzy numbers associated with degrees of confidence between zero and one.
The arithmetic operations between the α–cuts of fuzzy numbers are used to evaluate the total mark of each student, where α∈[0, 1]. The proposed methods can evaluate students’ answerscripts in a more flexible and more intelligent manner than the existing methods.
II. ARITHMETIC OPERATIONS BETWEEN FUZZY NUMBERS In this section, we briefly review some basic definitions of fuzzy numbers and the arithmetic operations between α–cuts operations of fuzzy numbers from [9], [17] and [18].
A fuzzy number is a fuzzy set in the universe of discourse X that is both convex and normal. A fuzzy number A of the universe of discourse X can be characterized by a triangular membership function parameterized by a triplet (a, b, c) as shown in Fig. 1.
X ( )X
µA
0 . 1
a b c
A
0
Fig. 1. A fuzzy number A.
The α–cut Aαof the fuzzy number A in the universe of discourse X is defined by
Aα= {xi | µA(xi) ≥α and xi ∈X}
= [a1(α), a2(α)], (1) whereα∈[0, 1], 0 ≤ a1(α) ≤ a2(α) ≤ Max(X), and Max(X) denotes the maximum value in the universe of discourse X.
Let [a1(α), a2(α)] and [b1(α), b2(α)] be theα–cuts of the
New Methods for Evaluating Students’ Answerscripts Using Fuzzy Numbers Associated with Degrees of Confidence
Hui-Yu Wang and Shyi-Ming Chen, Senior Member, IEEE
I
Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada July 16-21, 2006
fuzzy numbers A and B, respectively, whereα∈[0, 1], 0 ≤ a1(α) ≤ a2(α) ≤ Max(X), 0 ≤ b1(α) ≤ b2(α) ≤ Max(X), and Max(X) denotes the maximum value in the universe of discourse X.
The arithmetic operations between the α–cuts [a1(α), a2(α)] and [b1(α), b2(α)] of the fuzzy numbers A and B, respectively, are defined as follows:
The Addition between theα–Cuts [a1(α), a2(α)] and [b1(α), b2(α)]:
[a1(α), a2(α)] + [b1(α), b2(α)] = [a1(α)+ b1(α), a2(α)+ b2(α)]. (2) The Multiplication between theα–Cuts [a1(α), a2(α)] and [b1(α), b2(α)]:
[a1(α), a2(α)] × [b1(α), b2(α)] = [a1(α)× b1(α), a2(α) × b2(α)]. (3)
III. ANEW METHOD FOR EVALUATING STUDENTS’ ANSWERSCRIPTS USING FUZZY NUMBERS
ASSOCIATED WITH DEGREES OF CONFIDENCE In this section, we present a new method for students’
answerscripts evaluation using fuzzy numbers associated with degrees of confidence between zero and one, where nine satisfaction levels are used to evaluate students’
answerscripts regarding a question of a test/examination, i.e., Extremely Good (EG), Very Good (VG), Good (G), More or Less Good (MG), Fair (F), More or Less Bad (MB), Bad (B), Very Bad (VB) and Extremely Bad (EB). These nine satisfaction levels are represented by triangular fuzzy numbers parameterized by the triples shown as follows:
Extremely Good = (100, 100, 100), Very Good = (90, 100, 100), Good = (70, 90, 100),
More or Less Good = (50, 70, 90), Fair = (30, 50, 70),
More or Less Bad = (10, 30, 50), Bad = (0, 10, 30),
Very Bad = (0, 0, 10), Extremely Bad = (0, 0, 0).
Table I shows a fuzzy grade sheet with satisfaction levels associated with degrees of confidence of the evaluator between zero and one, where F1, F2, … , and Fn are satisfaction levels represented by triangular fuzzy numbers corresponding to the questions Q.1, Q.2, … , and Q.n, respectively, and the nine satisfaction levels are used, i.e., Extremely Good (EG), Very Good (VG), Good (G), More or Less Good (MG), Fair (F), More or Less Bad (MB), Bad (B), Very Bad (VB), and Extremely Bad (EB); α, β, …, and δ are the degrees of confidence of the satisfaction levels F1, F2, …, and Fn, respectively,where α∈[0, 1], β∈[0, 1], …, and δ∈[0, 1]. It is obvious that the satisfaction level awarded to each Question Q.i shown in Table I is associated with a degree of confidence between zero and one, where 1 ≤ i ≤ n. The larger the value, the higher the confidence of the
evaluator to give the satisfaction level regarding the answer to the question.
TABLE 1
A FUZZY GRADE SHEETWITH SATISFACTION LEVELS ASSOCIATED WITH DEGREES OF CONFIDENCE
Question
No. Satisfaction Levels Degrees of Confidence of Satisfaction Levels
Q.1 F1 α
Q.2 F2 β
… … …
Q.n Fn δ
Total Mark =
The Degree of Confidence of the Total Mark =
Consider the situation that the total mark of a student’s answerscript is 100 marks. Assume that there are n questions to be answered, i.e.,
TOTAL MARKS = 100,
Q.1 carries sl marks, Q.2 carries s2 marks, Q.n carries sM n marks,
where
∑
= n i
si 1
= 100, 0 ≤ si ≤100, and 1 ≤ i ≤ n. Assume that an evaluator evaluates a student’s answerscript as shown in Table I, where the satisfaction levels F1, F2, …, and Fn are described by nine satisfaction levels represented by triangular fuzzy numbers, α denotes the degree of confidence of the satisfaction level F1 awarded to the question Q.1, β denotes the degree of confidence of the satisfaction level F2 awarded to the question Q.2, …, and δ denotes the degree of confidence of the satisfaction level Fn
awarded to the question Q.n, where α∈[0, 1], β∈[0, 1], …, and δ∈[0, 1]. Assume that an optimism index λ [4]
determined by the evaluator is used to indicate the degree of optimism of the evaluator for evaluating students’
answerscripts, where λ∈[0, 1]. If 0 ≤λ< 0.5, then the evaluator is a pessimistic evaluator. If λ= 0.5, then the evaluator is a normal evaluator. If 0.5 <λ≤ 1.0, then the evaluator is an optimistic evaluator. The proposed method for the student’s answerscript evaluation is now presented as follows:
Step 1: Calculate theα–cut (F1)αof the fuzzy number F1, theβ–cut (F2)βof the fuzzy number F2, …, and theδ–cut (Fn)δof the fuzzy number Fn, respectively, where (F1)α = [a1, a2], (F2)β = [b1, b2], …, (Fn)δ= [z1, z2],α∈[0, 1], β∈[0, 1], …, and δ∈[0, 1].
Step 2: Calculate the interval-valued total mark [m1, m2] of the student’s answerscript, where
[m1, m2] = [
sn
s s
s + + + 2 ...
1
1 ×(F1)α+
sn
s s
s + + + 2 ...
1
2 ×
(F2)β + … +
n n
s s
s s
+ + + 2 ...
1
×(Fn)δ]
= [
sn
s s
s + + + 2 ...
1
1 × [a1, a2] +
sn
s s
s + + + 2 ...
1
2 ×
[b1, b2] + … +
n n
s s
s s
+ + + 2 ...
1
×[z1, z2]. (4)
Step 3: The total mark of the student is evaluated as follows:
(1 –λ)× m1 +λ×m2, (5)
where λ denotes the value of the optimism index λ determined by the evaluator and λ∈[0, 1]. The degree of confidence of the total mark awarded to the student is equal to Min(α,β,γ, …, δ), where Min(α,β,γ,…,δ)∈[0, 1]. Put this total mark and the degree of confidence in the appropriate box at the bottom of the fuzzy grade sheet.
In the following, we use an example to illustrate the students’ answerscript evaluation process.
Example 3.1: Consider the situation that total marks of a student’s answerscript to an examination is 100 marks.
Assume that in total there are four questions to be answered:
TOTAL MARKS = 100, Q.1 carries 20 marks, Q.2 carries 25 marks, Q.3 carries 25 marks, Q.4 carries 30 marks.
Assume that an evaluator awards the student’s answerscript by a fuzzy grade sheet as shown in Table II and assume that the optimism index λ of the evaluator is 0.60 (i.e.,λ = 0.60).
TABLE II
A FUZZY GRADE SHEET OF EXAMPLE 3.1 Question
No. Satisfaction Levels Degrees of Confidence of Satisfaction Levels Q.1 More or Less Good 0.75
Q.2 Good 1.0
Q.3 More or Less Bad 0.75
Q.4 Very Good 0.95
Total Mark =
The Degree of Confidence of the Total Mark =
[Step 1] Because the fuzzy number representations of the satisfaction levels More or Less Good, Good, More or Less Bad and Very Good are as follows:
More or Less Good = (50, 70, 90), Good = (70, 90, 100),
More or Less Bad = (10, 30, 50), Very Good = (90, 100, 100),
we can see that the 0.75-cut (More or Less Good)0.75 of the fuzzy number More or Less Good is [65, 75], the 1.0-cut (Good)1.0 of the fuzzy number More or Less Good is [90, 90], the 0.75-cut (More or Less Bad)0.75 of the fuzzy number More or Less Bad is [25, 35], and the 0.95-cut (Very Good)0.95 of the fuzzy number Very Good is [95, 100]. That is,
(More or Less Good)0.75 = [65, 75], (Good)1.0 = [90, 90],
(More or Less Bad)0.75 = [25, 35], (Very Good)0.95 = [95, 100].
[Step 2] Based on formula (4), we can calculate the interval-valued total mark [m1, m2] of the student’s answerscript, where
[m1, m2] =
30 25 25 20
20 + +
+ ×(More or Less Good)0.75 +
30 25 25 20
25 + +
+ ×(Good)1.0 +
30 25 25 20
25 + +
+ ×
(More or Less Bad)0.75 +
30 25 25 20
30 + +
+ ×
(Very Good)0.95
= [20 25 25 30 20
+ +
+ × [65, 75] +
30 25 25 20
25 + +
+ ×
[90, 90] +
30 25 25 20
25 + +
+ × [25, 35] +
30 25 25 20
30 + +
+ × [95, 100]
= 0.2×[65, 75] + 0.25×[90, 90] + 0.25×[25, 35] + 0.3
×[95, 100]
= [13, 15] + [22.5, 22.5] + [6.25, 8.75] + [28.5, 30]
= [70.25, 76.25].
[Step 3] Because the value of the optimism index λ determined by the evaluator is 0.60 (i.e., λ= 0.60), based on formula (5), the total mark of the student can be evaluated as follows:
(1 – 0.60)×70.25+ 0.60×76.25
= 0.40 ×70.25 + 0.60×76.25
= 28.1 + 45.75
= 73.85
≅74 (assuming that no half mark is given in the total mark).
The degree of confidence of the total mark of the student is equal to Min(0.75, 1.0, 0.75, 0.95) = 0.75.
IV. AGENERALIZED METHOD FOR EVALUATING
STUDENTS’ANSWERSCRIPTS USING FUZZY NUMBERS ASSOCIATED WITH DEGREES OF CONFIDENCE In this section, we present a generalized method for evaluating students’ answerscripts using fuzzy numbers associated with degrees of confidence of the evaluator. Let X be a set of satisfaction levels, X = {extremely good (EG), very very good (VVG), very good (VG), good (G), more or less good (MG), fair (F), more or less bad (MB), bad (B), very bad (VB), extremely bad (EB)}, where these satisfaction levels are represented by triangular fuzzy numbers. Assume that there are n questions to be answered:
TOTAL MARKS = 100, Q.1 carries sl marks, Q.2 carries s2 marks
M
Q.n carries sn marks, where
∑
= n i
si 1
= 100, 0 ≤ si ≤100, and 1 ≤ i ≤ n. Assume that an evaluator evaluates the questions of the student’s answerscripts using the following four criteria [1]:
C1: Accuracy of Information, C2: Adequate Coverage, C3: Conciseness, C4: Clear Expression.
Assume that the weights of the criteria C1, C2, C3 and C4 are wl, w2, w3 and w4, respectively, where wi ∈ [0, 1] and 1 ≤ i ≤ 4. Furthermore, assume that the evaluator can evaluate each question of students’ answerscripts based on the above four criteria using the method presented in Section III. Let Fij
denotes a satisfaction level represented by a triangular fuzzy number indicating the satisfaction level of the question Q.i of the student’s answerscript with respect to the criterion Cj, where 1 ≤ i ≤ n and 1 ≤ j ≤ 4; let αj denote the degree of confidence of the satisfaction level F1j awarded to the question Q.1 of the student’s answerscript with respect to the criterion Cj, where αj ∈[0, 1] and 1 ≤ j ≤ 4; letβj denote the degree of confidence of the satisfaction level F2j awarded to the question Q.2 of the student’s answerscript with respect to the criterion Cj, where βj ∈[0, 1] and 1 ≤ j ≤ 4; …, let δj
denotes the degree of confidence of the satisfaction level Fnj
awarded to the question Q.n of the student’s answerscript with respect to the criterion Cj, where δj ∈[0, 1] and 1 ≤ j ≤ 4. Assume that an optimism index λ [4] determined by the evaluator is used to indicate the degree of optimism of the
evaluator for evaluating student’s answerscripts, where λ
∈[0, 1]. If 0 ≤λ< 0.5, then the evaluator is a pessimistic evaluator. If λ= 0.5, then the evaluator is a normal evaluator.
If 0.5 <λ≤ 1.0, then the evaluator is an optimistic evaluator.
The proposed generalized method for the student’s answerscript evaluation is now presented as follows:
Step 1: Calculate theα1–cut, theα2–cut, theα3–cut and the α4–cut of the fuzzy numbers F11, F12, F13 and F14, respectively, where theα1–cut, theα2–cut, theα3–cut and theα4–cut of the fuzzy numbers F11, F12, F13 and F14 are [f11(L), f11(U)], [f12(L), f12(U)], [f13(L), f13(U)] and [f14(L), f14(U)], respectively, α1 ∈[0, 1], α2 ∈[0, 1], α3∈[0, 1] andα4 ∈[0, 1];
calculate theβ1–cut, theβ2–cut, theβ3–cut and theβ4–cut of the fuzzy numbers F21, F22, F23 and F24, respectively, where theβ1–cut, theβ2–cut, theβ3–cut and theβ4–cut of the fuzzy numbers F21, F22, F23 and F24 are [f21(L), f21(U)], [f22(L), f22(U)], [f23(L), f23(U)] and [f24(L), f24(U)], respectively,β1 ∈[0, 1], β2 ∈[0, 1], β3∈[0, 1] andβ4 ∈[0, 1];
calculate theδ1–cut, theδ2–cut, theδM 3–cut and theδ4–cut of the fuzzy numbers Fn1, Fn2, Fn3 and Fn4, respectively, where theδ1–cut, theδ2–cut, theδ3–cut and theδ4–cut of the fuzzy numbers Fn1, Fn2, Fn3 and Fn4 are [fn1(L), fn1(U)], [fn2(L), fn2(U)], [fn3(L), fn3(U)] and [fn4(L), fn4(U)], respectively,δ1 ∈[0, 1], δ2 ∈[0, 1], δ3∈[0, 1] andδ4 ∈[0, 1].
Step 2: Calculate the interval-valued mark [m11, m12] awarded to the question Q.1 of the student’s answerscript, where
[m11, m12] =
4 3 2 1
1
w w w w
w + +
+ × [f11(L), f11(U)]+
4 3 2 1
2 w w w w
w + +
+ × [f12(L), f12(U)] +
4 3 2 1
3
w w w w
w + +
+ × [f13(L), f13(U)] +
4 3 2 1
4
w w w w
w + +
+ × [f14(L), f14(U)], (6)
where 0 ≤ m11≤ m12≤ 100;
calculate the interval-valued mark [m21, m22] awarded to the question Q.2 of the student’s answerscript, where
[m21, m22] =
4 3 2 1
1
w w w w
w + +
+ × [f21(L), f21(U)]+
4 3 2 1
2
w w w w
w + +
+ × [f22(L), f22(U)] +
4 3 2 1
3
w w w w
w + +
+ × [f23(L), f23(U)] +
4 3 2 1
4
w w w w
w + +
+ × [f24(L), f24(U)], (7) where 0 ≤ m21≤ m22≤ 100;
M
calculate the interval-valued mark [mn1, mn2] awarded to the question Q.n of the student’s answerscript, where
where 0 ≤ mn1≤ mn2≤ 100.
Step 3: Calculate the interval-valued total mark [I(L), I(U)] awarded to the student, where
[I(L), I(U)] =
sn
s s
s + + + 2 ...
1
1 ×[m11, m12] +
sn
s s
s + + + 2 ...
1
2 ×[m21, m22] + … +
n n
s s
s s
+ + + 2 ...
1
×[mn1, mn2], (9)
where 0 ≤ I(L) ≤ I(U) ≤ 100.
Step 4: The total mark of the student is evaluated as follows:
(1 –λ)×I(L) + λ× I(U), (10)
where λ denotes the value of the optimism index determined by the evaluator and λ∈[0, 1]. The degree of confidence of the total mark awarded to the student is equal to Min(α1,α2,α3,α4,β1,β2,β3,β4,…,δ1,δ2,δ3,δ4), where Min(α1,α2,α3,α4, β1,β2,β3,β4,…,δ1,δ2,δ3, δ4)∈[0, 1]. Put this total mark and the degree of confidence in the appropriate box at the bottom of the fuzzy grade sheet.
V. CONCLUSIONS
We have presented new methods for evaluating students’
answerscripts using fuzzy numbers associated with degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students’
answerscripts are represented by triangular fuzzy numbers associated with degrees of confidence between zero and one.
The arithmetic operations between the α–cuts of fuzzy numbers are used to evaluate the total mark of each student, where α∈[0, 1]. The degree of confidence of the total mark of each student also can be calculated by the proposed methods. The proposed methods can evaluate students’
answerscripts in a more flexible and more intelligent manner than the existing methods.
ACKNOWLEDGMENTS
The authors would like to thank Professor Jason Chihyu Chan, Department of Education, National Chengchi University, Taipei, Taiwan, Republic of China, for providing very helpful comments and suggestions.
REFERENCES
[1] R. Biswas, “An application of fuzzy sets in students' evaluation,” Fuzzy Sets and Systems, vol. 74, no. 2, pp. 187-194, 1995.
[2]D. F. Chang and C. M. Sun, “Fuzzy assessment of learning performance of junior high school students,” Proceedings of the 1993 First National Symposium on Fuzzy Theory and Applications, Hsinchu, Taiwan, Republic of China, pp. 10-15, 1993.
[3] S. M. Chen and C. H. Lee, “New methods for students’ evaluating using fuzzy sets,” Fuzzy Sets and Systems, vol. 104, no. 2, pp. 209-218, 1999.
[4] C. H. Cheng and K. L. Yang, “Using fuzzy sets in education grading system,” Journal of Chinese Fuzzy Systems Association, vol. 4, no. 2, pp. 81-89, 1998.
[5] T .T. Chiang and C. M. Lin, “Application of fuzzy theory to teaching assessment,” Proceedings of the 1994 Second National Conference on Fuzzy Theory and Applications, Taipei, Taiwan, Republic of China, pp.
92-97, 1994.
[6] J. R. Echauz and G. J. Vachtsevanos, “Fuzzy grading system,” IEEE Transactions on Education, vol. 38, no. 2, pp. 158-165, 1995.
[7] L. Frair, “Student peer evaluations using the analytic hierarchy process method,” Proceedings of 1995 Frontiers in Education Conference, vol.
2, pp. 4c3.1-4c3.5, 1995.
[8] V. G. Kaburlasos, C. C. Marinagi, and V. T. Tsoukalas, “PARES: A software tool for computer-based testing and evaluation used in the Greek higher education system,” Proceedings of the 2004 IEEE International Conference on Advanced Learning Technologies, pp.
771-773, 2004.
[9] A. Kaufmann and M. M. Gupta, Fuzzy Mathematical Models in Engineering and Management Science. Amsterdam: North-Holland, 1988.
[10] C. K. Law, “Using fuzzy numbers in education grading system,” Fuzzy Sets and Systems, vol. 83, no. 3, pp. 311-323, 1996.
[11] J. Ma and D. Zhou, “Fuzzy set approach to the assessment of student-centered learning,” IEEE Transactions on Education, vol. 43, no. 2, pp. 237-241, 2000.
[12] F. McMartin, A. Mckenna, and K. Youssefi, “Scenario assignments as assessment tools for undergraduate engineering education,” IEEE Transactions on Education, vol. 43, no. 2, pp. 111-119, 2000.
[mn1, mn2] =
4 3 2 1
1
w w w w
w + +
+ × [fn1(L), fn1(U)] +
4 3 2 1
2
w w w w
w + +
+ × [fn2(L), fn2(U)] +
4 3 2 1
3
w w w w
w + +
+ × [fn3(L), fn3(U)] +
4 3 2 1
4
w w w w
w + +
+ × [fn4(L), fn4(U)], (8)
[13] J. R. Nolan, “An expert fuzzy classification system for supporting the grading of student writing samples,” Expert Systems with Applications, vol. 15, no. 1, pp. 59-68, 1998.
[14] A. Pears, M. Daniels, A. Berglund, and C. Erickson, “Student evaluation in an international collaborative project course,”
Proceedings of the 2001 Symposium on Applications and the Internet Workshops, pp. 74-79, 2001.
[15] M. H. Wu, A Research on Applying Fuzzy Set Theory and Item Response Theory to Evaluate Learning Performance, Master Thesis, Department of Information Management, Chaoyang University of
Technology, Wufeng, Taichung County, Taiwan, Republic of China, 2003.
[16] H. Xingui, “Weighted fuzzy logic and its applications,” Proceedings of the 12th Annual International Computer Software and Applications Conference, Chicago, Illinois, pp. 485-489, 1988.
[17] L. A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp. 338-353, 1965.
[18] H.-J. Zimmermann, Fuzzy Set Theory and Its Applications. Dordrecht:
Kluwer-Nijhoff, 1991.
