The aim of Study 3 was to examine stability in 2 × 2 achievement goal endorsement over time in a Taiwanese student panel sample.
Method
Participants and Procedure
This study included only secondary students drawn on from the sample of Study1. 784 students (378 middle school students and 405 high school students) participated in two consecutive years. They attended the seventh and tenth grades in 2007; one academic year later, they attended the eighth and eleventh grades. The students were instructed to complete the AGQ-C at mid-spring semester 2007 (Time 1). Again, they were instructed to complete AGQ-C at mid-spring semester 2008 (Time 2).
Measures
The AGQ-C was again used to investigate the participating students’ achievement goals in their Chinese language course. Reliability coefficients in the present data for the four achievement goal subscales were .88/.88, .85/.88, .91/.92, and .83/.87 at Time 1 and Time 2, respectively.
Statistics Analysis
The structural stability, differential stability and mean-level stability of secondary students’
goal endorsements were examined. Confirmatory factor analysis was used to compare the fit indexes for a series of nested models with increasing constraints. I conducted Pearson product–moment correlations to examine differential continuity in achievement goal endorsement across the two time points. Descriptive statistics for each achievement goal were computed for data collection periods, and paired t tests were computed to test mean-level stability.
Results
Descriptive statistics and intercorrelations of achievement goal items across Time 1 and Time 2 are presented in Table 5-1. Table 5-2 offers descriptive statistics, alpha coefficients of and zero order correlations of indicators of achievement goals over Time 1 and Time 2.
Time1
1.Ma1 3.65 1.06 - 2.Ma2 3.70 1.10 .70 -
3.Ma3 3.46 1.11 .66 .65 - 4.Mv1 3.00 1.14 .22 .23 .20 -
5.Mv2 3.03 1.18 .26 .22 .18 .67 -
6.Mv3 3.00 1.16 .29 .24 .21 .60 .71 -
7.Pa1 3.42 1.14 .47 .52 .49 .25 .20 .18 -
8.Pa2 3.35 1.14 .51 .58 .50 .27 .22 .19 .82 -
9.Pa3 3.27 1.13 .47 .51 .50 .20 .15 .15 .67 .72 -
10.Pv1 2.80 1.19 .07 .06 .05 .16 .25 .24 .15 .17 .15 -
11.Pv2 2.91 1.22 .09 .12 .05 .21 .28 .27 .14 .15 .15 .61 -
12.Pv3 2.82 1.27 .07 .10 .03 .21 .25 .26 .13 .14 .15 .53 .69 - Time 2
13.Ma1 3.60 .97 .39 .36 .36 .05 .03 .08 .30 .32 .31 -.02 -.01 -.05 - 14.Ma2 3.64 1.01 .37 .43 .38 .08 .06 .09 .33 .35 .32 -.02 -.01 -.05 .71 - 15.Ma3 3.48 1.04 .34 .35 .41 .13 .09 .11 .27 .29 .30 -.08 -.03 -.08 .65 .69 - 16.Mv1 3.01 1.08 .12 .14 .11 .21 .22 .22 .10 .13 .09 .10 .10 .11 .25 .26 .25 - 17.Mv2 3.08 1.08 .12 .10 .12 .18 .22 .23 .10 .09 .07 .10 .10 .12 .22 .23 .24 .73 - 18.Mv3 3.02 1.12 .09 .12 .13 .18 .22 .25 .10 .12 .08 .09 .10 .14 .25 .23 .24 .62 .72 - 19.Pa1 3.41 1.06 .33 .37 .35 .14 .09 .11 .43 .47 .42 .03 .03 .03 .52 .59 .50 .25 .22 .21 -
20.Pa2 3.34 1.06 .31 .36 .35 .14 .11 .10 .44 .47 .42 .02 .01 .00 .51 .60 .53 .28 .25 .22 .84 -
21.Pa3 3.26 1.07 .32 .35 .34 .12 .06 .07 .42 .46 .43 .04 .05 .04 .48 .51 .46 .24 .20 .17 .71 .74 -
22.Pv1 2.94 1.14 -.05 .01 .00 .09 .12 .12 .01 .02 .05 .22 .27 .24 .07 .04 .10 .28 .29 .29 .09 .12 .14 - 23.Pv2 2.92 1.16 -.04 -.01 -.01 .08 .07 .10 .01 .02 .03 .20 .27 .25 .06 .02 .09 .24 .27 .29 .09 .11 .11 .66 - 24.Pv3 2.83 1.20 -.04 -.01 -.01 .13 .11 .14 .03 .01 .04 .22 .25 .26 .05 .04 .09 .27 .30 .32 .10 .11 .12 .58 .74
Note. When the correlation coefficients were above .06, they were statistically significant at .05.
Ma1-3 = Mastery-approach goal items; Mv1-3= Mastery-avoidance goal items;
Pa1-3 =Performance-approach goal items; Pv1-3 =Performance-avoidance goal items
Table 5-2 Descriptive statistics, alpha coefficients of and zero order correlations of indicators of achievement goals over Time 1 and Time 2 (N = 784)
M SD 1 2 3 4 5 6 7 8
T1 1. Ma 3.48 .944 (.88)
2. Mv 3.05 .942 .35** (.85)
3. Pa 3.22 .998 .63** .25** (.91)
4. Pv 2.81 .975 .08* .28** .14** (.83)
T2 5. Ma 3.50 .875 .49** .15** .44** -.06 (.88)
6. Mv 3.07 .932 .21** .29** .20** .16** .37** (.88)
7. Pa 3.29 .950 .45** .14** .58** -.02 .67** .34** (.92) 8. Pv 2.88 .993 -.01 .13** .03 .37** .06 .35** .10** (.87)
Note. ( ) : alpha coefficients of internal consistency
Ma = Mastery-approach goal indicator; Mv= Mastery-avoidance goal indicator;
Pa =Performance-approach goal indicator; Pv =Performance-avoidance goal indicator
Structural stability
Confirmatory factor analyses were used to compare the fit indexes for a series of four nested models with increasing constraints: configural invariance (Model 1), weak measurement invariance (Model 2), strong MI (Model 3), and strict MI (Model 4). As can be seen from Table 5-3, model 1 achieved an acceptable fit according to the GFI, CFI, and IFI; although the chi-square-test indicated significant departures of the model from the data—which is also owed to the high power of this test in conjunction with many degrees of freedom. As a consequence, I considered the configural invariance model as adequately describing the data.
Subsequently, weak MI (model 2) was imposed by requiring the factor loadings to be equal at Time 1 and Time 2. Doing so did not significantly reduce model fit (Δχ2 (8) = 17.83, p = .023), implying that weak measurement invariance holds. The scaling of the latent variables was equal, which allows variance and covariance comparisons of the factors across time. In model 3 (Strong MI), intercepts of the observed indicators were constrained to be equal across time, thus imposing strong MI. According
to Table 5-3, the fit of this model was not statistically inferior to that of the previous one (Δχ2(12) = 21.40, p = .045), from which one might conclude that strong MI holds across Time 1 and Time 2.
Consequently, factor mean differences can be calculated across time, because all mean differences of the indicators are due to differences in latent variable means in the strong invariance model.
Finally, strict MI (Model 4) was imposed by requiring residual variances of the 12 indicators to be equal at Time 1 and Time 2. As Table 5-3 shows, model fit decreased significantly compared to the previous model (Δχ2(12) = 97.71, p < .005). Hence, it appeared as if at least some of the residual variances were different at the two measurement occasions. However, according to the GFI, CFI and IFI, these differences did not seem to be very pronounced. I thus concluded that strict measurement invariance did not hold, which implied that not all differences in the variances of the observed indicators were due to differences in factor variances. Note that for examining the three different types of stability, strict MI does not represent a prerequisite. It is sufficient to establish strong MI (Meredith, 1993; Meredith & Horn, 2001) for other stability examinations.
Table 5-3 Invariance analyses of four measurement invariance models over time
Model df χ2 GFI CFI IFI Δχ2 Δdf p Δχ2/Δdf
Model 1 Configural
invariance 212 437.72 0.955 0.990 0.990 ---
Model 2 weak factorial
invariance 220 455.55 0.954 0.989 0.989 17.83 8 0.02254 2.229
Model 3 strong factorial
invariance 232 476.95 0.954 0.989 0.989 21.40 12 0.04482 1.783 Model 4
strict factorial
invariance 244 574.66 0.944 0.985 0.985 97.71 12 0.00000* 8.143
***P<0.005
Differential stability
Cross time Pearson product–moment correlations were calculated to examine differential stability.
Table 5-4 showed that 4 latent achievement goals at Time 1 were significantly positive related to their
respective goals at Time 2. The correlation coefficients 0.579 (performance-approach), .489 (performance-approach), .373 (performance-avoidance) to 0.291 (mastery-avoidance) are relative of middle to small magnitude. This implies that the rank order of secondary students changed profoundly in two avoidance-goal endorsement across a year. By contrast, it appears as if two approach-goal endorsements were more stable with regard to interindividual differences across time.
Mean-level stability
Table 5-4 also shows means and standard deviations for each latent achievement goals. Paired t tests were performed to test cross time differences of means for each achievement goal. Three of four latent factor means (in the sample level) remained quite stable across a year; only performance-approach goals increased significantly from Time 1 to Time 2.
Table 5-4 Descriptive Statistics, mean-level stability (paired t test), and differential stability T1 T2 T1 to T2
Achievement goals M SD M SD t r
Mastery-approach goals 3.48 .944 3.50 .875 -.504 .489**
Mastery-avoidance goals 3.05 .942 3.07 .932 -.491 .291**
Performance-approach goals 3.22 .998 3.29 .950 -2.302* .579**
Performance-avoidance goals 2.81 .975 2.88 .993 -1.804 .373**
* p< .05; ** p < .01.