With the evolvement of the language learning theories in recent years, questions have arisen with regard to the psychometric properties of the SILL. These studies (Nyikos & Oxford, 1993; Oxford & Burry-Stock, 1995; Hsiao & Oxford, 2002) focused on: a) the adequacy of categorizing strategies into six divisions; b) the psychological assumptions underlying the suggestion of strategic paradigms; c) the relationship of strategy research among the current language teaching paradigms; d) the relationships between the identified strategies and intercorrelations among these identified strategies; e) the adequacy of various measurements of strategy use and awareness (Brown, 2007).
Oxford and Burry-Stock (1995) examined the reliability and validity of the ESL/EFL version of SILL as well as its confirmed relationship to sensory preferences.
Their study included an estimated 40-50 major studies, including a dozen dissertations and theses which used the SILL, involving 8000-8500 language learners. The psychometric qualities of the SILL can be summarized in the follow aspects of utility, reliability, validity.
In the aspect of utility, many researchers around the world have employed the SILL as an instrument to discover the relationship between strategy use and language performance in the classroom. The most important purpose of this goal is that if there is a strong relationship between these two variables, language performance might be improved by enhancing strategy use. Other uses of the SILL have included assessing strategy use at a given point, to be compared with strategy use later; comparing the learning strategy use of men and women, making the linkage between strategy use and underlying learning styles conceptually, and based on the strategy use of different students, providing individualized classroom instruction. The SILL is very useful in real-world settings for making decisions relevant to people’s lives.
24
In the aspect of reliability, the Cronbach alpha reliability coefficient is used on continuous data such as the Likert-type in the SILL. The reliabilities of the ESL/ EFL SILL have been high across many culture groups, such as 0.94 of Yang (1992), using the Chinese translation in Taiwan; 0.92 of Watanabe (1990), using the Japanese translation; 0.91 of Oh (1992), using the Korean translation; 0.93 of Park (1994), using the Korean translation and 0.91 of Oxford and Green, using Puerto Rican Spanish translation. Although some reliabilities became lower when the study was conducted in English instead of learners’ native language in the ESL/EFL settings, they were still very acceptable.
In the aspect of validity, both content-validity and criterion-related validity resting on the predictive and correlative links with language performance (course grades, standardized test scores, ratings of proficiency) in a number of studies (Rossi-Le, 1989; Phillips, 1990, 1991; Chang, 1991; Wen and Johnson, 1991; Green
& Oxford, 1993; Park, 1994; Mullins, 1991; Oxford et. al., 1993; Oxford and Nyikos, 1989; Watanabe, 1990) showed the strongest support demonstrations of the SILL. For example, Rossi-Le (1989) found that language proficiency level could predict strategy use in multiple regression analysis. More proficient ESL students used self-management strategies like planning and evaluation (p < .006) and formal practice (p < .02) significantly more frequently than less proficient ESL students.
Although doubts still exist such as Robson and Midorakawa’s (2001) study, questioning the reliability of the SILL in their research that showed a low level of test-retest reliability when the SILL was conducted twice to the same Japanese learners of English, Hsiao and Oxford (2002) compared classification theories of language learning strategies by using confirmatory factor analysis (CFA) of the data collected from 517 college EFL learners to measure ESL/EFL version of the SILL. In their study, fifteen rival models were compared, and seven research hypotheses were
25
derived and tested. They first compared Rubin’s (1981) dichotomous direct and indirect classification of language learning strategies with Oxford’s and found some major differences (Table 2). While Rubin classified clarification/ verification and monitoring as direct strategies, Oxford classified their counterparts as indirect strategies (asking question for clarification/ verification = social strategy; monitoring
= metacognitive strategy). Production tricks which were classified as Rubin’s indirect strategies corresponded to a subcategory of Oxford’s compensation strategies.
Table 2.
The differences between Rubin’s taxonomy and Oxford’s taxonomy of language learning strategies.
strategies Rubin’s Taxonomy Oxford’s Taxonomy
clarification/ verification
direct indirect→ socialmonitoring
direct indirect→ metacognitiveproduction tricks
indirect direct→compensationFurthermore, Hsiao and Oxford (2002) pointed out the differences between O’Malley and Chamot’s (1990) system and Oxford’s system (Table 3): a) O’Malley Chamot’s system categorized inferencing in cognitive strategies whereas Oxford listed it as a compensation strategy to make up for missing knowledge; b) Oxford separated memory strategy from the category of cognitive strategies unlike O’Malley and Chamot who put both together in cognitive strategies; c) Communication strategies were not specifically included in the O’Malley and Chamot system; d) O’Malley and Chamot grouped affective and social strategies together to form a small category called socioaffective strategies, but, by contrast, Oxford classified both as separate categories.
26 Table 3.
The differences between O’Malley & Chamot’s taxonomy and Oxford’s taxonomy of language learning strategies.
strategies O’Malley & Chamot’s Oxford’s
inferencing
cognitive compensationmemory
cognitive memorycommunication
not included includedaffective
socioaffective affectivesocial
socioaffective socialTo test viability of the factorial structure underlying different strategy classification schemes proposed in the literature, Hsiao and Oxford (2002) undertook a series of confirmatory factor analysis to compare Oxford’s six-factor solution against Rubin’s (1975) two-factor classification and three variants (three-factor, four-factor, and five-factor solutions) of O’Malley and Chamot’s (1990) strategy classification (four factors by subdividing socio-affective strategies into social and affective strategies; five factors: by subdividing cognitive strategies into memory, cognitive and compensation strategies). The results of Hsiao and Oxford’s studies suggested that Oxford’s (1990) classification of strategies into six broad dimensions accords with the learner’s use of strategies and constitutes a valid categorization scheme. In addition, the 50-item version of SILL, designed to measure learners’ use of L2 or foreign learning strategies, has a clear factor structure based on Oxford’s six-taxonomy learning strategies (Green & Oxford, 1995).
According to the present researcher’s review of studies of language learning strategies, the SILL appears to be the most frequently used instrument. The popularity may be explained by the specificity and convenience of a questionnaire with good psychometric properties that make it easy for researchers to adopt the framework.
Griffiths and Oxford (2014) suggested the instrument, SILL, tailored to the needs and characteristics of the group would measure students’ actual strategy use more reliably.
27
Thus, in the present research, the most important criteria for a study to be subsumed in the meta-analysis is adopting the SILL as the main instrument to investigate learners’ language learning strategy use. Even though a study adapted the SILL according to the needs of the particular learners, situations, goals and research purpose, it would be considered as a SILL-based study.