3.5 Measurements
3.5.2 Reading ability tests
3.5.2.1 Word recognition test. Word identification is the process of determining the pronunciation and some meaning of a word encountered in print (Gentry, 2006; Harris &
Hodges, 1995). Readers employ a variety of strategies to accomplish this. Ehri (2005) identified four of them: decoding, analogizing, predicting and recognizing whole words by sight. The word recognition test contains 150-word lists ranging in difficulty in terms of the structure of Thai words for fourth grade and ranging from two- to four-word syllables (see Table 5). The lists were developed by randomly sampling the grade-level lists from basic reading vocabularies for fourth grade (Bureau of Academic Affairs and Educational
Standards, Thailand, 2008). The word structures of the word recognition test and an example test item of each word structure was presented in Table 5.
44 Table 5
Example of a word recognition test.
Word structures
Number of syllables
2 3 4
Without a final consonant กระแส พระด าริ ระเกะระกะ
The final consonant is the same as the main alphabet’s sound
คานหาบ หอยทับทิม ก าลังวังชา
The final consonant is different from the main alphabet’s sound
สาบสูญ สุภาษิต บุญญาธิการ
Reduced vowel character พนัน กติกา สถิตเสถียร
Changing vowel character ฝึกหัด สรรพสิ่ง นิทรรศการ
Diphthong ปลากัด กระจายเสียง กระเสือกกระสน
Changing initial consonant sound เหลวไหล พระต าหนัก หมายเลขรหัส Using a tone marker ลุล่วง ผ้าเช็ดหน้า สุรุ่ยสุร่าย
Vowel: ไ- ใ- ยั- ไส้กรอก ไปรษณีย์ รถจักไอน ้า
Using orthography รถทัวร์ วาจาสิทธิ์ ลายลักษณ์อักษร
3.5.2.2 Reading comprehension. Comprehension occurs as a reader builds a mental representation of the message contained in a text. Comprehension depends on the activation of relevant background knowledge and is strongly related to oral language comprehension and vocabulary growth. Along with explicit vocabulary instruction, meta-cognitive strategies such as questioning, predicting, making inferences, clarifying misunderstandings and
summarizing while reading should be included in comprehension instruction (Birsh, 2005).
45
To measure reading comprehension, a Thai reading comprehension test was developed based on the original test created by Worasri (2007). The original test was created to measure the Thai reading ability for fourth grade students in the Lana dialect in the north part of Thailand.
In order to measure reading in this study, a modified version was used, as some of the items from the original test used the local language. The modifications made the test more
appropriate for students in general. Seven passages were used in this test. Each passage ranged from 45 to 306 Thai words in length, and 35 comprehension questions were developed for this test. The comprehension questions were developed based on three sub-domains: vocabulary knowledge (nine items), syntactic awareness (eight items) and pragmatic awareness (18 items). Passage and comprehension questions were presented on paper in multiple-choice format. Each correct answer counted as one point, and the time for the test was 60 minutes. The test was administered in groups.
Example of a reading comprehension question:
ให้นักเรียนอ่านข้อความต่อไปนี้และตอบค าถามข้อ
1 – 2“
พ่อถือขวานออกไปไหนแต่เช้าจ้ะแม่
”นิดถามแม่ขณะที่ก าลังช่วยแม่จัดโต๊ะอาหาร
“
ออกไปที่สวนจ้ะ เมื่อคืนฝนตกหนัก
”แม่ตอบ
“
ท าไมพ่อต้องเอาขวานออกไปด้วยละจ้ะ
”หนูน้อยอายุ 7 ขวบยังสงสัย
“
ต้นมะขามหวานต้นใหญ่ถูกฟ้าผ่า ถ้าเอามันไว้ก็ไม่ออกฟักอีก พ่อเขาก็เลยจะไปโค่นมันทิ้ง
” 1. ในบทสนทนา ผู้ถามมีความสัมพันธ์อย่างไรกับผู้พูด
ก. เป็นน้อง ข. เป็นแม่
ค. เป็นลูก ง. เป็นพ่อ
2
. เหตุการณ์ในตอนนี้เป็นเวลาใด
ก. เวลากลางคืน ข. เวลาเช้ามืด
ค. เวลาเช้า ง. เวลาค ่า
46 3.6 Data collection
The researcher recruited 15 volunteers to do data collection. The volunteers were 15 teachers chosen from the schools sampled in this study. One day of intensive training was provided to the volunteers. The researcher designed the contents of the training for all volunteers. The training covered the following:
- Why is data collection needed?
- What are the cognitive components and reading abilities of reading in Thai?
- Why do we need to administer the tests individually?
- What is step-by-step data collection?
- How should we explain the directions of each sub-test?
- Why are item trials so important?
- How do you judge whether an answer is right or wrong?
- How do you score each sub-test?
- How do you build rapport and create a relaxing situation?
After intensive training, data were collect for the first time in January – February, 2014, and for the second time in November 2014 – January 2015. The tests were administered in the first semester of fourth grade school year in quiet rooms in the schools with parental consent.
3.7 Data analysis
The hypothesized relationships were examined using the two-wave cross-lagged structural equation model (SEM) (Anderson & Gerbing, 1988) to simultaneously address the directions of the relationship between cognitive component and reading ability over time. To achieve the goal of this study, two steps of data analysis.
47
First, the preliminary results, descriptive statistics comprising the means and standard deviations of the study variables as well as the zero order bivariate correlations between them were calculated using the SPSS package (version 17) (SPSS Inc., 2008). Next, a structural equation model (SEM) was used for second of data analysis. Second step, a cross-lagged SEM was used to examine the hypothesized relationships between students’ cognitive components and reading abilities. For conducting the SEM, the Amos 18.0 software (Arbuckle, 2009) was used.
3.7.1 Cross-lagged panel analysis. Cross-lagged panel analysis, a SEM technique, was used to test the hypotheses. As a way of ameliorating measurement error, a common threat to validity in causal analysis, a latent variable approach was used (Shadish, Cook, & Campbell, 2002). Panel data consist of at least two variables measured at two or more time points in the same set of subjects. The analysis of panel data has been recognized for its advantages in testing for causal effects because it can provide evidence regarding all three conditions of causality: (a) covariation of the two variables; (b) time precedence of the causal variable; (c) non spuriousness (i.e. the association of the two variables must not be produced by a joint association with a third variable or set of variables). Figure 4 shows the model for the main analysis. Cross-lagged analyses include three types of relationships. The first relationship is synchronous correlations - the relationship between variables within each time point (e.g., cognitive component at Time-1 and reading ability at Time-1). These indicate the cross-sectional relationships between variables. The second relationship is auto-regressive the relationship among the same variables over time (e.g., cognitive component at Time-1 and Time-2). These provide an indication of stability across time. The third relationship is the cross-lagged correlations – the relationship between one variable on another over time (e.g., cognitive component at Time 1 and reading ability at Time 2).
48
3.7.2 Statistical analysis. The SEM analysis was conducted with maximum likelihood estimation using Amos 18.0 (Arbuckle, 2009). The evaluation of the hypothesized model fit included a measurement phase and a structural phase. In SEM, a variance-covariance matrix is generated from observed data. The structural phase involves a path-analysis approach, in which the SEM program determines estimates that will most nearly reproduce the variance-covariance matrix. The fit indices selected a priori to determine goodness of-fit included indices of absolute fit, the chi-square statistic ( 𝜒2), the comparative fit index (CFI) , and the root mean square error of approximation (RMSEA) (Hu & Bentler, 1999).
The Chi-Square value is the traditional measure for evaluating overall model fit and 'assesses the magnitude of discrepancy between the sample and fitted covariance matrices' (Hu & Bentler, 1999). In case of small samples, the Chi-Square statistic lacks power and because of this may not discriminate between good fitting models and poor fitting models (Kenny & McCoach, 2003). Due to the restrictiveness of the Model Chi-Square, the current study have sought alternative indices to assess model fit, that minimizes the impact of sample size on the Model Chi-Square, which was introduced by Wheaton, Muthén, Alwin, and Summers (1977) that is the ratio of relative/normed chi-square (𝜒2/df). The
recommendations of 𝜒2/df is range from as high as 5.0 (Wheaton et al, 1977) to as low as 2.0 (Tabachnick & Fidell, 2007). In this study, to justify model fit the criterion ratio of
relative/normed chi-square of Tabachnick & Fidell (2007) was applied. The Goodness-of-Fit statistic (GFI) was created by Jöreskog and Sörbom as an alternative to the Chi-Square test and calculates the proportion of variance that is accounted for by the estimated population covariance (Tabachnick & Fidell, 2007). The GFI values can range between 0 and 1, and a GFI value of 0.90 or larger is generally accepted as a cut-off criterion for indicating well-fitting models (Miles & Shevlin, 1998).
49
The Comparative Fit Index (CFI) represents the ratio between the discrepancies of this target model to the discrepancy of the independence model (Bentler, 1990). The CFI assumes that all latent variables are uncorrelated (null/independence model) and compares the sample covariance matrix with this null model. The CFI statistic can have values ranging between 0.0 and 1.0 with values closer to 1.0 indicating good fit. A cut-off criterion of CFI 0.95 is
presently recognized as indicative of good fit (Hu & Bentler, 1999).
The RMSEA was first developed by Steiger and Lind (1980, cited in Steiger, 1990). It is a statistic which explains how well the model, with unknown but optimally chosen
parameter estimates, would fit the population covariance matrix (Byrne, 1998). RMSEA values below 0.01 indicated excellent fit; values between 0.01 and below 0.05 good fit;
values between 0.05 and 0.08 mediocre fit, and values above 0.10 poor fit (MacCallum, Browne & Sugawara, 1996).
50
CHAPTER FOUR
RESULTS
As previously stated, a two-wave cross-lagged panel design was used to determine the relationship between cognitive component and reading ability over time. This chapter presents the results on the measurement development, descriptive statistics, zero order correlations, and results from the cross-lagged structural equation modelling (SEM).
4.1 Measurement development
The tests were developed following the theoretical principles that explain reading processes, and content validity as assessed by expert judgment. In particular, results of the measurement development including item fit in terms of mean-square statistics (MNSQ) and the reliability of the scales are presented.
4.1.1 Item mean-square statistics and item difficulty. In Rasch analysis, item fit indices are reported for individual items. An item that has an MNSQ of 1 indicates a perfect fit. A value that ranged between 0.75 and 1.33 generally points to a good fit (Adams & Khoo, 1996). Table 6 shows the MNSQ at time 1 and at time 2 of the morphological structure test, morphological production test, phoneme isolation test, reading comprehension test and word recognition test. The MNSQ for all the items in the five tests ranged from 0.80 to 1.22, indicating that the items had a good fit to the Rasch model (see the Appendix for the detailed MNSQ values of each test). Table 6 also showed that the item difficulties of the five tests at time 1 and time 2 ranged from –2.989 to 2.393.
4.1.2 Reliability. The reliability of the tests on morphological structure,
morphological production, phoneme isolation, reading comprehension and word recognition were examined in terms of Rasch reliability coefficient (EAP/PV) using the ConQuest
51
software. The EAP/PV reliability pertains to the variance explained in accordance with the estimated model divided by the total variance in individual ability (Adams, 2006). For both the time 1 and the time 2 data, the tests exhibited an EAP/PV reliability between 0.647 and 0.953, which indicated that most of the observed total variance is accounted for by the model variance.
Cronbach’s alpha was used to determine the reliability of the rapid word segmentation test. According to Gable and Wolf (1993), the alpha values of good cognitive measures usually ranged from high .80s to low .90s, but the authors also indicated that good affective instruments frequently exhibit Cronbach’s alpha reliability values ranging in the .70s. The Cronbach’s alpha reliability of the test for the time 1 data was .874, whereas that for the time 2 data was 0.836. On the basis of the values of the Cronbach’s alpha, the overall scale of the rapid word segmentation test exhibited adequate internal consistency.
Test–retest reliability was determined for the rapid number naming (RNN) and rapid colour naming (RCN) tests. Test–retest reliability is the most transparent approach to investigating change over time (Baltes, Reese & Nesselroade, 1978). It involves the use of the same test repeatedly over time and is defined as the extent to which test material can be relied on to consistently measure a characteristic over time with the same test material.
Pearson’s correlation coefficient measures the direction and strength of the linear relationship revealed by test–retest reliability. The coefficient can in theory range from –1 to +1. A
correlation coefficient larger than 0.80 is usually considered high, whereas a correlation coefficient .00–.20 are considered slight, .21–.40 fair, .41–.60 moderate, .61–.80 substantial, and .81–1.0 almost perfect (Smith et al., 1990). The correlation coefficients of the RNN and RCN tests over the two datasets were higher than 0.90, indicating that the tests are test-retest reliable.
52
Since, all tests were collected data in two time point, therefore the test-retest reliability were applied to measure each characteristic consistently over time. The results show that all tests have correlation coefficient of the test-retest ranged from 0.298 to 0.853, indicated that all tests have characteristic consistently over time, in other word all tests are reliable.
Table 6
Item fit with MNSQ item difficulty and reliability.
Measurement
Reading comprehension test T1 0.86; 1.22 –1.220; 2.305 0.832 .853**
Word recognition test T1 0.80; 1.18 –2.985; 2.393 0.965 .596**
Rapid colour naming test T1 - - 0.931 .454**
Rapid number naming test T1 - - 0.981 .410**
Rapid word segmentation test T1 - - 0.874 .438**
Phoneme isolation test T1 0.90; 1.12 –1.744; 1.712 0.647 .395**
Morphological production test T1 0.88; 1.10 –1.704; 1.700 0.687 .350**
Morphological structure test T1 0.82; 1.12 –2.425; 1.656 0.710 .298**
Reading comprehension test T2 0.92; 1.09 –1.231; 0.527 0.775 .853**
Word recognition test T2 0.79; 1.13 –2.989; 2.220 0.953 .596**
Rapid colour naming test T2 - - 0.924 .454**
Rapid number naming test T2 - - 0.902 .410**
Rapid word segmentation test T2 - - 0.836 .438**
Phoneme isolation test T2 0.89; 1.16 –1.806; 0.032 0.665 .395**
53 Table 6 (continued)
Item fit with MNSQ item difficulty and reliability.
Measurement
Morphological production test T2 0.89; 1.14 –1.522; 0.204 0.723 .350**
Morphological structure test T2 0.88; 1.17 –2.207; 0.689 0.656 .298**
Note. T1 = data from time 1, T2 = data from time 2
4.2 Preliminary results
4.2.1 Descriptive statistics. The results on cognitive component and reading ability are shown in Table 7. The frequency distribution of the eight tests from the first- to the second-time data was evaluated in term of skewness and kurtosis statistics. Ability with skewness and kurtosis statistics that exceeded an absolute value of 2 were flagged as potentially problematic (Hildebrand, 1986). As shown in Table 7, the distribution of ability across the eight tests aligned with the skewness and kurtosis statistics that were lower than absolute 2 (range, –2 to 2). This alignment indicates that all the tests on cognitive component and
reading ability are characterised by normal distribution. The minimum and maximum scale of cognitive component consist the rapid colour naming test (RCN), rapid number naming test (RNN), rapid word segmentation test (RWS), phoneme isolation test (PHI), morphological production test (MAP), and morphological structure test (MAS) at time-1 and time-2 were ranged from 0.133 to 0.448 for minimum scale and 0.180 to 0.924 from maximum scale, with the mean ranged from 0.151 to 0.774 and the standard deviation (SD) ranged from 0.011 to 0.143. The minimum and maximum scale of reading ability consist the reading
comprehension test (REC) and the word recognition (WOR) at time-1 and time-2 were
54
ranged from 0.138 to 0.372 for minimum scale and 0.809 to 0.882 from maximum scale, with the mean ranged from 0.487 to 0.651 and the standard deviation (SD) ranged from 0.132 to 0.164.
Table 7
Descriptive statistics for cognitive component and reading ability.
Minimum Maximum Mean
Standard
Note. REC = reading comprehension, WOR = word recognition, RCN = rapid colour naming, RNN = rapid number naming, RWS = rapid word segmentation, PHI = phoneme
55
isolation, MAP = morphological production, MAS = morphological structure. T1 = data from time 1, T2 = data from time 2.
4.2.2 Bivariate correlations. The correlation coefficients of the study variables are presented in Table 8. Overall, the measures of the cognitive and reading components of reading development in the first- and second-time data exhibited a consistent relationship.
The correlation results on cognitive component and reading ability agree with the correlation structure in Messick’s (1989) study, which involved an examination of
intercorrelations to determine how strongly each item correlates with the other items from its respective subscale. The cut-off criterion used by the author was a correlation of at least 0.20 between the item and most of the other items from its respective subscale. In the current study, cognitive components were measured on the basis of morphological production, morphological structure, rapid colour naming, rapid number naming, rapid word
segmentation and phoneme isolation, over both the datasets, were correlated significantly with reading comprehension and word recognition (range, 0.286 to 0.964). Rapid colour naming and rapid number naming were negatively correlated with morphological production, morphological structure, rapid word segmentation, phoneme isolation, reading
comprehension and word recognition at times 1 and 2 (Table 8), indicating that if students are highly competent in certain skills (morphological production, morphological structure, rapid word segmentation, phoneme isolation, reading comprehension and word recognition), then they require less time to name objects (colour and number in this study).
56 Table 8
Bivariate correlations amongst the cognitive and reading ability tests.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1. RECT1 1
2. WORT1 .750** 1
3. RNCT1 –.792** –.634** 1
4. RNNT1 –.735** –.600** .905** 1
5. RWST1 .746** .637** –.778** –.797** 1
6. PHIT1 .745** .628** –.756** –.751** .886** 1
7. MAPT1 .747** .588** –.742** –.742** .783** .739** 1
8. MAST1 .732** .579** –.742** –.729** .770** .730** .964** 1
9. RECT2 .853** .678** –.895** –.874** .796** .765** .749** .752** 1
10. WORT2 .767** .596** –.790** –.750** .698** .662** .646** .656** .884** 1
11. RCNT2 –.406** –.295** .454** .417** –.369** –.336** –.320** –.317** –.456** –.445** 1
12. RNNT2 –.404** –.285** .401** .410** –.364** –.340** –.297** –.286** –.458** –.434** .668** 1
13. RWST2 .518** .377** –.532** –.495** .438** .406** .423** .409** .559** .539** –.729** –.717** 1
14. PHIT2 .463** .317** –.497** –.468** .408** .395** .367** .379** .518** .512** –.633** –.631** .714** 1
15. MAPT2 .491** .378** –.457** –.402** .415** .383** .350** .348** .469** .429** –.693** –.732** .701** .615** 1
16. MAST2 .422** .288** –.411** –.358** .353** .353** .298** .298** .428** .428** –.613** –.583** .667** .798** .674** 1
57
** p <0.01
Note. 1 = reading comprehension (REC), 2= word recognition (WOR), 3= rapid colour naming (RCN), 4 = rapid number naming (RNN), 5 = rapid word segmentation (RWS), 6 = phoneme isolation (PHI), 7= morphological production (MAP), 8= morphological structure (MAS). T1 = data from time 1, T2 = data from time 2.
58 4.3 Structural model: Cross-lagged modelling
The cross-lagged structural equation model (SEM) was analysing to answer the research question. The idea behind cross-lagged modelling is to identify the causal priority of variables by comparing the crossed and lagged paths between constructs. The crossed path coefficient represents the longitudinal prediction of one construct by another, above and beyond the autoregressive prediction of that construct from an earlier measure of itself (Curran & Bollen, 2001). For model estimation, SEM with the time 1 and time 2 data was conducted. The hypothesised model is shown in Figure 5. This model yielded an acceptable fit with the data (χ2/df = 346.142/92, GFI = 0.894, CFI = .962, RMSEA = .088). Figure 5 shows the complete model used for hypothesis testing.
Figure 5. Model for hypothesis testing.
Following the examination of the overall model fit indices, individual parameter estimates were assessed to analyse the feasibility of their estimated values. No individual parameters with unreasonable estimates ‘falling outside of acceptable ranges’ (i.e. standard errors that are extremely large or small, correlations greater than 1 and negative variance)
59
(Byrne, 1994) were found. For significance tests, parameter estimates were divided by standard error to determine whether the parameter estimates of the SEM statistically differ from zero. As indicated by an alpha level (p) of the .05 two-tail standard, if test statistics are greater than ±1.96, data are in the critical region and that the value of sample statistics differs significantly from zero.
4.4 Hypothesis testing
The aim of this study was not to build the best model of cognitive component and reading ability but to examine the hypothesized relationships between cognitive component and reading ability. Because the model fit is satisfactory, its regression coefficients were examined to validate hypotheses 1–6.
Table 9
Standardized path coefficients of the relationships.
Estimate
SE CR P
Standardized Unstandardized
COG T2 <--- COG T1 0.332 0.221 0.083 2.672 **
COG T2 <--- Read T1 –0.274 –0.032 0.014 –2.218 *
Read T2 <--- Read T1 0.201 0.208 0.062 3.385 ***
Read T2 <--- COG T1 –0.768 –4.581 0.371 –12.363 ***
COG T1 <--> Read T1 –0.883 –0.002 0.000 –10.353 ***
COG T2 <--> Read T2 –0.157 0.000 0.000 –2.226 * Note 1. REC = reading comprehension, WOR = word recognition, COG = cognitive component, READ = reading ability, T1 = data from time 1 and T2 = data from time 2.
Note 2. *** = p< .001, ** = p< .01, * = p< .05
60
Table 9 reports the standardised path coefficients of the relationships amongst the major variables without covariates in three types of the relationship including;
(1) Auto-regressive effects of the two variables.
The auto-regressive effect (C1C2) of the cognitive component at time-1 (COG T1) on the cognitive component at time-2 (COG T2) was significantly correlation, with the standard coefficient 0.332 (p< .01). And the auto-regressive effect (R1R2) of the reading ability at time-1 (REC T1) on the reading ability at time-2 (REC T2) was significantly correlation, with the standard coefficient 0.202 (p< .001).
(2) Synchronous correlations between the two variables.
The unconditional zero order correlation (C1R1) between the cognitive component (CCG T1) and the reading ability (REC T1), both at time-1 was significantly negative correlation, with the standard coefficient -0.883 (p< .001). And the zero order correlation (C2R2) between the cognitive component (COG T2) and the reading ability (REC T2), both at time-2 was significantly negative correlation, with the standard coefficient -0.157 (p< .05).
These negative correlations indicate an inverse relationship between the cognitive component time-1 and reading ability time-1, and also inverse relationship between the cognitive time-2 and reading ability time-2.
(3) Reciprocal effects between the two variables over time.
The Reciprocal effects (C1R2) of the cognitive component at time-1 (COG T1) on reading ability at time-2 (REC T2) was significantly negative relationship, with the standard coefficient -0.768 (p< .001). And the effect (R1C2) of reading ability at time-1 (REC T1) on cognitive component at time-2 (COG T2) was significantly negative relationship, with the standard coefficient -0.274 (p< .05). These negative effected indicate an inverse relationship between the cognitive component time-1 and reading ability time-2, and also inverse
relationship between the reading ability time-1.and cognitive time-2
61
Figure 6 illustrates the standardised path coefficients of the relationship between
cognitive component and reading ability for hypothesis testing by cross-lagged SEM analysis.
Figure 6. Standardised path coefficients of the relationships determined for hypothesis testing by cross-lagged SEM.
The negative correlations of C1R1, C2R2, R1C2, and C1R2 may involve with the negative correlation between rapid naming tests (rapid colour naming and rapid number naming) and the other tests. In order to assess the causal relationship from three cognitive
The negative correlations of C1R1, C2R2, R1C2, and C1R2 may involve with the negative correlation between rapid naming tests (rapid colour naming and rapid number naming) and the other tests. In order to assess the causal relationship from three cognitive