Chapter 3 Methodology
3.1 Research Design
3.2.3 Logistic Regression for RaschGSP IRT
Call N is the number of students taking the test, test results are reported in interval data by range of Gamma values s (localized grey relational grade), the greater the s the higher the proficiency level on the test content.
Suppose that the students are graded into two types: pass and fail, for example; pass is denoted by 1 and fail is denoted by 0. Logistic regression is applied to determine the relationship between test scores and ability if a student passed the exam, as follows: In the 0XY plane, 0X-axis describes the order of students sorted by ascending s value, 0Y-axis shows only two states of pass and fail (Fig. 3-2).
Fig. 3-2 Test results of the students plotted against the pass-fail categories With Y is called the event of pass-fail of a student, these results are transformed into probabilities. The probabilities of event Y are plotted at each value of X, the curve is plotted to fit into logistic regression curve, the graph is presented in Figure 3-3.
According to that basis, logistic regression model is determined as follows:
Y: event of pass-fail of a student, Y=1 corresponding to s 0.5 : pass, Y=0 corresponding to s 0.5: fail
x: the order of students’ abilities corresponding to their s value increasing 0
Y 1
Order of students’ abilities xxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
X
52
Fig. 3-3 Test results plotted against probability of allocation to pass-fail categories )
s(x
: the probability of event Y bound by the condition x, 0s(x)1 The form of the logistic regression equation is:
x x
53
It is guessed that Y 1 when s(x)0.5 corresponding to (x)0, and Y 0 when s(x)0.5 corresponding to (x)0.
Obtained:
1
Y when s(x)0.5 at locations x
0
Y when s(x)0.5 at locations x
Note. For the problems, the procedure is also performed the same with procedure above, the similar formula is obtained for p(x).
The graph is plotted again for clarity with some important points
Fig. 3-4 Logistic regression curve of test results
Referring to and selecting the advantages of nonparametric IRT model and kernel smoothing - nonparametric technique for estimating regression functions which was proposed by Ramsay (Lee, 2007; Ramsay, 1991; T. Oguz & Tuncay, 2014), the results above lead to the following definitions and properties.
Definition 3.1: (RaschGSP IRT function)
Let 1 exp
( )
) 1
(
x D x
y be the two-parameter logistic regression
function, where ,R are regression coefficients; D1.702 is the scaling constant.
When x is the order of students’ abilities or the order of items’ difficulties and y is the localized grey relational grade, the above function y(x) is called RaschGSP IRT function. If x is the order of students’ abilities and y is LGRG-S then the above function
1
0.5
0
1 x
54
) (x
y is called RaschGSP IRT function for students, similarly, if x is the order of items’ difficulties and y is the LGRG-P then y(x) is called RaschGSP IRT function for problems.
Similarly, the three-parameter RaschGSP IRT function is established, its definition is presented as follows (Sheu, Nguyen, Pham, Nguyen, et al., 2014):
Definition 3.2: (RaschGSP IRT function)
Let 1 exp
( )
) 1
(
x D x
y (3-6)
be the three-parameter logistic regression function, where ,, R are regression coefficients; D1.702 is the scaling constant. When x is the order of students’ abilities or the order of items’ difficulties and y is the localized grey relational grade, the above function y(x) is called RaschGSP IRT function. If x is the order of students’
abilities and y is LGRG-S then the above function y(x) is called RaschGSP IRT function for students, similarly, if x is the order of items’ difficulties and y is the LGRG-P then y(x) is called RaschGSP IRT function for problems.
Definition 3.3: (RaschGSP IRT curve)
RaschGSP IRT curve is the graph of RaschGSP IRT function. The graph of RaschGSP IRT function for students is called RaschGSP IRT curve for students and the graph of RaschGSP IRT function for problems is called RaschGSP IRT curve for problems.
Definition 3.4: (test intermediate value)
The abscissa of the intersection point between the RaschGSP IRT curve and the straight line y0.5 is called test intermediate value and denoted by 0.5.
In Figure 3-4, 0.5 is clearly shown in the graph.
Definition 3.5: (Ability level of class)
Ability level of student class only based on test result (abbreviated as ability level of class) is determined by the ratio O of portion 10.5 of students getting high test score to portion 0.5 of students getting low test score in that class, the greater the ratio O is the higher the ability level of class reaches, and vice versa.
55
In the RaschGSP IRT model, the relationship between the test intermediate value (0.5) and the ability level of class (O) is the decreasing relationship. That means the greater the 0.5 is the lower the O reaches, and vice versa, the smaller the 0.5 is the higher the O reaches.
Proof:
For a fixed group of students taking a test, suppose that the data of this test result are fitted for the logistic regression model which is expressed by the RaschGSP IRT word, the relationship between 0.5 and O is the decreasing relationship.
(Q.E.D) Illustration: Figure 3-5 shows the case of the ability level of class (O) being high that corresponds to the high percentage of high Gamma students:
56
Fig. 3-5 RaschGSP IRT curve for students of class with high ability level
In contrast, Figure 3-6 shows the case of the ability level of class (O) being low that corresponds to the low percentage of high Gamma students.
Application: For assessment of the ability level of class through its test result, its RaschGSP IRT curve for students will be plotted to determine 0.5, if the 0.5 is small then the ability level of class will be appreciated. In contrast, if the 0.5 is large then the ability level of class will be underestimated. (See example 3.4 in page 81)
57
Fig. 3-6 RaschGSP IRT curve for students of class with low ability level Theorem 3.1: (Value of test intermediate value)
Suppose that
then the test intermediate value 0.5 is determined by the following formula:
)
58
Taking the natural logarithm of both sides with the supposition of right side is greater than zero. This supposition is always satisfied due to (3-10).
(3-11)
Illustration: The value of test intermediate value is determined based on regression coefficients (parameters) ,, . In practice (in this dissertation), the value of test intermediate value is calculated and outputted from program written by MATLAB programing language. This value is applied to evaluate ability level of class – one of evaluation criteria mentioned in this dissertation (see definition 3.5).
Application: (Please see example 3.4 in page 81).
59
then the value of test discrimination 0.5 is calculated by the following formula:
The three-parameter RaschGSP IRT function is
( )
Illustration: The value of test discrimination is determined based on regression coefficients (parameters): ,, . In practice (in this dissertation), the value of test discrimination is calculated and outputted from program written by MATLAB
60
programing language. This value is applied to evaluate quality of test – one of evaluation criteria mentioned in this dissertation.
Application: (Please see example 3.3 in page 77).
Property 3.2: (Meaning of test discrimination)
The test discrimination 0.5 expresses the discrimination of test for the students in class. The higher the polarization between group of high Gamma students (Gamma 0.5) and group of low Gamma students (Gamma 0.5) is, the larger the slope of RaschGSP IRT curve for students gets, that means the greater the value of 0.5 gets, and vice versa.
Proof: (This property is self-evident true, need not to be proved)
Fig. 3-7 RaschGSP IRT curve for students in case of high test discrimination
61
Fig. 3-8 RaschGSP IRT curve for students in case of low test discrimination Illustration: Figure 3-7 shows that, for this class the test is very good for classifying students into two large groups: high and low achievement because the test discrimination 0.52.28 (very high). And Figure 3-8 shows that the test is not very good in classifying students into two groups: high and low achievement (0.50.86).
Application: The test discrimination for each class represented by 0.5 (abbreviated ) is compared to each other, for example, if 12 then test for class number 2 is better than for class number 1 in classifying students into two groups. (See example 3.3 page 77)
Theorem 3.3: (Comparing test discrimination for classes)
Suppose that each data set Dk (xki,yki), k1,2,,K; i1,2,,M , where K is the number of classes, and M is the number of students in a class, satisfies RaschGSP
IRT function 1 exp
( )
) 1
(
f x D x
y . If and
are set to be constants62
(, const), then the test discriminations for the classes represented by k for the data sets Dk are significantly comparable to each other.
Proof:
Consider the following three-parameter RaschGSP IRT function:
( )
RaschGSP IRT curve of the k-th class is a regression curve of the function expressed by (3-15) for the data set Dk (xki,yki),k 1,2,,K; i1,2,,M. Let it be proven
Taking the natural logarithm of both sides:
)
63
Taking the natural logarithm of both sides, we get 1 0 satisfies the RaschGSP IRT function given. In Figure 3-9, the test discriminations for class number 1 and class number 2 corresponding to 1,2 are significantly compared to each other, in detail 12. This is used to evaluate the quality of test.
Application: The test discrimination for each class represented by 0.5 (abbreviated ) is compared to each other, for example, if 12 then the test for class number 2 is better than for class number 1 in classifying students into two groups. (See example 3.3 in page 77)
64
Fig. 3-9 Test discrimination for each class represented by 1, 2 is comparable to each other
Theorem 3.4: (Comparing test intermediate values for classes)
Suppose that each data set Dk (xki,yki),k1,2,,K; i1,2,,M where K is the number of classes, and M is the number of students in class, satisfies RaschGSP IRT function y f (x) 1exp
1D(x0)
. If and are set to be constants (, const), then the test intermediate values of classes represented by k found for the data sets Dk are significantly comparable to each other.Proof:
Consider the following three-parameter RaschGSP IRT function:
( )
exp 1 ) 1
(
f x D x
y , where ,, Rare parameters
RaschGSP IRT curve of the k-th class is regression curve of the above function for the k-th data set Dk (xki,yki), k 1,2,,K; i1,2,,M . It is proven that
65
Taking the natural logarithm of both sides:
significantly comparable together, and the RaschGSP IRT curves are congruent.
The above result is still true for the case of parameter 0.5.
(Q.E.D) Illustration: For every data set Dk, it is always found out one and only one value of
5 .
0 of it satisfies the RaschGSP IRT function given. Therefore, each class has a test intermediate value respectively, they can be compared to each other. This result is used to evaluate the ability level of a class.
Application: The test intermediate value of each class represented by 0.5 is compared to each other, that means, the ability level of each class is compared to each other. For example, if the test intermediate value of class number 1 is greater than the one of class number 2 then the ability level of class number 2 is better than the class number 1. In specific, please see example 3.4 page 81.
Corollary 3.1: (Comparing ability levels of classes)
66
Suppose that each data set Dk (xki,yki),k 1,2,,K; i1,2,,M where K is the number of classes, and M is the number of students in class, satisfies RaschGSP IRT function y f(x) 1exp
1D(x0 )
. If and are set to be constants (, const), then the ability levels of classes represented by Ok values for the data sets Dk are significantly comparable to each other.Proof:
This corollary is demonstrated by the proofs of property 3.1 and theorem 3.4.
Application: Ability levels of classes represented by Ok are compared together via comparing test intermediate values k together. For example, in the Figure 3-10, test intermediate values of classes number 1 and 2 corresponding to 1 ,2and 12, so the ability level of class number 1 is higher than ability level of class number 2 in evaluating the ability of class. (See example 3.4 in page 81)
Fig. 3-10 Test intermediate values of three classes are comparable to each other
67
In order to meet the above model, the data have to satisfy some requirements. Three basic assumptions of the model for data:
(1) A unidimensional latent trait;
where R is regression coefficient, D=1.702 is the scaling constant.
Two-parameter logistic regression function:
where ,R are regression coefficient, D=1.702 is the scaling constant.
And three-parameter logistic regression function: