The Rasch model has the identification problem that different sets of parameters give rise to the same distribution of Uij. For example, the two sets of (bj, θi) = (0, 0.5) and (bj, θi) = (−0.5, 0) result in the same probability of getting a correct answer under
the Rasch model (1). Similar identification problems exist in our current model. In current model, similar indeterminacy exists between the item difficulty bj and mean ability µθ parameters. That is, the sets of parameters (bj, µθ) and (bj+ c, µθ+ c) result in the same probability of getting a correct answer under the Rasch model such that
p(uij = 1) =
In other words, different parameter sets could satisfy the same distribution of Uij. That is, based on the data, we are unable to distinguish between the two sets of parame-ters and therefore it is necessary to constrain some parameter values for identification purpose.
In the context of multiple-group factor models for ordered-categorical measures, Millsap and Tein (2004) propose a set of parameter identification constraints in con-generic and dichotomous case. Concon-generic denotes the factor structure including any single-factor models, or any multiple-factor model where each indicator loads on only one latent variate. In the multiple-group factor models, the ordered-categorical out-come Yij yields the response s where
Yij∗c = ajc
θic+ εij, (16)
Yijc = s if τj,sc< Yij∗c≤ τj,s+1c,
where c is the group membership of examinee i, τj,s is the sth threshold of item j, aj
is the loading of item j, and E(ϵij) = 0, ∀i = 1, · · · , N and j = 1, · · · , J.
The set of identification constraints proposed by Millsap and Tein (2004) is:
For some c0 ∈ {1, · · · , C}, E(θic0) = 0, and Var(ϵcij0) = 1,∀j = 1, · · · , J;
∀j = 1, · · · , J, τj,s1 1 = τj,s2 1 =· · · = τj,sC1 for some s1 ∈ {1, · · · , S};
for some j0 ∈ {1, · · · , J}, a1j0 = a2j0 =· · · = aCj0 = 1, and
τj10,s2 = τj20,s2 =· · · = τjC0,s2 for some s2 ∈ {1, · · · , S}(s2 ̸= s1).
In the two-group case of the focal (f ) and reference (r) groups, Hwang (2012) proposes a set of identification constraints for model (16) as
E(θri) = 0;
∀j = 1, · · · , J, Var(ϵrij) = Var(ϵfij) = 1;
ar1 = af1 = 1, and τ1,1r = τ1,1f .
An alternative set of equivalent identification constraints based on Millsap and Tein (2004) in this case is
E(θir) = 0, and Var(ϵrij) = 1,∀j = 1, · · · , J;
∀j = 1, · · · , J, τj,1r = τj,1f ; ar1 = af1 = 1, and τ1,2r = τ1,2f .
Although the above identification constraints are proposed for multiple-group models, similar principles apply to the cases of mixture or latent class models. Our present model (8) uses the logit link and the variance of the standard logistic distribution is
π2
3 . That is, if we formulate our model with the εij terms in (16), the variances of εij’s are fixed for all items and all classes and therefore Hwang’s approach is more applicable. Moreover, we only consider Rasch models for dichotomous responses and therefore the complexity of identification issue is greatly reduced. To be more specific, our study adopts the identification constraints that
For some c0 ∈ {1, · · · , C}, E(θi|η, c0) = µθc0 = 0;
for some j0 ∈ {1, · · · , J}, b1j0 = b2j0 =· · · = bCj0.
3 Simulation Studies
In this section, we examine the estimation performance under the mixture SEM framework for the MRM-RT model. Furthermore, the advantage of using the informa-tion of response time is evaluated by comparing results from simultaneously analyzing item responses and response time to those from simply using item responses alone.
3.1 Data Generation
First, we introduce the simulation settings. There are three latent classes and 25 multiple-choice items, each with four options. Parameters of the three classes are designed to characterize examinees whose behaviors are rapid-guessing (RG, class 1), solution behavior (SB, class 2), high ability and/or respond with familiarity (HARF, class 3). The proportions of examinees in the RG, SB, and HARF classes are 0.15, 0.55 and 0.3, respectively.
The parameter values are mainly based on the simulation settings used in Meyer (2010). Examinees in the SB class generally spend the most time on each item among the three classes. In contrast, examinees in the HARF class spend less time on items than those in the SB class due to their smartness or familiarity with the items from practice. Examinees in the RG class only take a few seconds on each question, for the circumstances that they simply read through item quickly and guess without too much thinking. In the RG class, mean and variance of response log-time are fixed to be -0.5 and 0.01, respectively. For the SB and HARF classes, mean of response log-time increases linearly as item difficulty increases. Response time spent on either the easier or the more difficult items are likely to be similarly shorter or longer among examinees, and therefore the variances of response log-time for both the easier and the more difficult items are set to be small. In the SB class, mean and variance of response log-time range from -0.3 to 0.9 and 0.32 to 0.41, respectively. In the HARF class, mean and variance of response log-time range from -0.47 to 0.25 and 0.21 to 0.28, respectively. In other words, the variability in response time is considered smaller for examinees in the HARF class than those in the SB class. The parameters in Table 1 are in the log scale, and in order to better understand the difference among the classes, the response time distribution on item 13 for each class are plotted in Figure 2. The
response time for examinees in the RG class is generally much shorter than that for other classes.
Table 1: Mean and Variance Parameters of Response Log-Time for RG, SB and HARF Classes
RG SB HARF
mean variance mean variance mean variance
-0.5 0.01 -0.3 0.32 -0.47 0.23
RG = rapid-guessing; SB = solution behavior;
HARF = high ability and/or respond with familiarity.
As for the distribution of the ability, we assume that examinees in the RG class simply guess one of the options without thinking, and therefore each option of an item has the same probability of being chosen. In other words, the probability of getting a correct answer on each four-option item by guessing is .25 in the RG class.
Therefore, the mean and variance of the ability distribution are both conveniently
Figure 2: The response time distribution on item 13 for the RG, SB and HARF classes.
set to be 0 and we simply use the appropriate item difficulty parameter to ensure such a probability of answering correctly for each item. The SB class stands for the more general population, and therefore the ability distribution is assumed to follow the standard normal distribution N(0, 1). In contrast, the mean of the ability is higher for the HARF class, characterizing that the faster response is partly due to the smartness of examinees in this class. Moreover, its variance of ability is smaller than that of the SB class, indicating that examinees in the HARF class are more homogeneous in terms of their ability. The mean and variance of the ability distribution in the HARF class are respectively 0.5 and 0.65.
The characteristics of the three classes are not only reflected in the response time and the ability distribution, but also in the item difficulty parameters. In the RG class, item difficulty parameters bj’s are fixed to 1.099 for all items such that the probability of getting a correct answer on each item is .25. In both the SB and HARF classes, item difficulty parameters bj’s range from -2 to 2. To capture the feature of the HARF class such that examinees in this class might be more familiar with test items through more practice and therefore some items may appear easier to them, the difficulty parameters of those items are considered to be smaller for the HARF class than the SB class. Here, we randomly select ten out of the 25 items to have smaller difficulty parameters for the HARF class than those for the SB class. The chosen items are 3, 7, 9, 11, 15, 16, 19, 20, 23, and 25. These items are considered to exhibit differential item functioning (DIF) with respect to the latent groups of SB and HARF (Maij-de Meij, Kelderman &
van der Flier, 2010). All the item difficulty parameter values are summarized in Table 2.
Table 2: Item Difficulty Parameters for RG, SB, and HARF Classes
item RG SB HARF
RG = rapid-guessing; SB = solution behavior;
HARF = high ability and/or respond with familiarity.
The estimation performance under various sample sizes, i.e., the total numbers of examinees, is also of interest. In addition to the sample sizes of 500 and 2000 used in Meyer (2010), we also take 250 and 1000 into consideration to better represent the small, medium and large sample sizes. For each case, 100 sets of independent repli-cations are simulated. To further examine the information brought from the response time, two fitted models, MRM-RT and MRM, are considered for each replication.
MLR is applied to both fitted models. All the estimations are done using Mplus 5.
Necessary identification constraints are imposed to both the SB and HARF classes to
ensure identification of item parameters in both fittings. More specifically, the ability mean is fixed to 0 for the SB class, and item 1 is considered the anchor item with no differential item functioning between the SB and HARF classes. In addition, with regard to the RG class, the ability mean and variance are fixed to 0, and item difficulty parameters is fixed to 1.099 without estimating.