An Investigation of the Relationships between Strategy Use and GEPT Test Performance

(1)

ࡻᄬିጯ English Teaching & Learning 32. 3 (Fall 2008): 35-69

An Investigation of the Relationships between

Strategy Use and GEPT Test Performance

Jessica R W Wu

The Language Training and Testing Center (LTTC) jw@lttc.ntu.edu.tw

Abstract

This study investigated language learner strategy use reported by 567 Taiwanese EFL learners through a questionnaire and the relationships of strategy use to language performance in the General English Proficiency Test (GEPT)—Listening and Reading, at the intermediate level. Statistical analyses were employed to explore the complex relationships between test-takers’ cognitive and metacognitive strategy use and L2 test performance. Major findings include: Learners used more metacogntive strategies than cognitive strategies in general. Some strategies were found to be significant predictors of test performance, accounting for 14% of the variance. Variations in the effect of learner strategy use may be associated with L2 proficiency and the language skill (listening vs. reading) assessed in the test being investigated. Some strategies were used more frequently by the higher achievers, which suggests that particular strategies are useful for EFL learners. Despite the limitations, the findings have a number of implications for pedagogy and future research.

Key Words: strategy use, cognitive and metacognitive processing, GEPT

(2)

INTRODUCTION

Influenced by individual characteristics, some learners are more successful than others in their language learning and even in taking a language test. Since the 1970s, many second language acquisition (SLA) researchers have been concerned with the learning process itself, and much research has been devoted to investigating the relationships between learner strategy use and cognitive processing and the products of SLA. Many researchers (e.g., O’Malley, Chamot, Stewner-Manzanares, Kupper, & Russo, 1985; Politzer & McGroarty, 1985; Purpura, 1998, 1999) compared the use of cognitive processing (CP) related strategies between successful and unsuccessful language learners, through which the taxonomies of language learning strategies were also established. The results of these studies suggest the existence of a positive relationship between L2 learners’ strategy use and their learning outcome.

A similar trend has emerged in language testing (LT) research as researchers have expressed growing interest in providing a model of communicative language ability which aims to describe the language ability of an individual and to identify the individual characteristics that may influence variation in learners’ performance in a language test (Bachman, 1990; Bachman & Palmer, 1996). Strategy use is one of the many variables that were identified as the individual characteristics, including age, gender, background knowledge, motivation, attitude, anxiety, and the like, that would likely affect test performance. According to Canale and Swain (1980), strategic competence, along with grammatical competence and sociolinguistic

(3)

Wu: Strategy Use and GEPT Test Performance

competence, form the three major components in communicative competence. In their model of communicative competence, strategic competence, including both verbal and nonverbal strategies, may be called into action by learners to compensate for communication breakdowns due to insufficient competence or performance variables. Also, in the model proposed by Bachman (1990), metacogntive strategies involving assessment, planning, and execution were defined as strategic competence, which is a mediator between the language use context and internal knowledge in communicative language use. However, the depiction of metacognitive strategies in the earlier models involving language testing has been criticized as lacking sufficient support from empirical evidence (McNamara, 1996; Purpura, 1999). Not until recent years have a few researchers empirically investigated this issue (e.g., Phakiti, 2003; Purpura, 1997, 1998, 1999; Song & Cheng, 2006). Therefore, more empirical research on the effect of strategic competence on language test performance is desirable.

Inspired by the empirical research into the relationships between learners’ reported strategy use and their performance in English language tests (Phakiti, 2003; Purpura, 1997, 1998, 1999; Song & Cheng, 2006), this study investigated strategy use as reported by Taiwanese EFL learners through a questionnaire validated by Purpura (1999), and its relationships to the GEPT, a test that is widely taken in Taiwan. In addition, the present study echoed Phakiti’s (2003) recommendation that replication of the past research into the relationships of strategy use to language performance in L2 testing should be carried out, aiming to examine whether consistency in the use of cognitive and metacognitive strategies in L2 testing is

(4)

observable. To enable comparisons between the findings of the present study and the earlier research into this area, the present study adopted the definitions used in Purpura (1999, p.6), in which cognitive strategy use relates to “specific actions or behaviors that students invoke during learning, use or testing” and metacognition is the notion of thinking about thinking. Metacognition involves deliberate, planned, goal-directed mental processing. Thus, metacogntive strategy use has “an executive control over the cognitive strategy use.”

The research questions in this article are as follows:

1. What types of language strategies did the GEPT-Intermediate test-takers under investigation report using?

2. What relationships, if any, existed between reported strategy use and language test performance in the GEPT-Intermediate? Which strategy use was the best predictor of GEPT performance at the intermediate level?

3. Were there any differences between higher achievers and lower achievers in reported strategy use and in the relationships between strategy use and test performance?

LITERATURE REVIEW

Under the influence of research and theory in cognitive psychology and information processing, there has been a growing interest in studying language learner strategies since the 1970s. Rubin (1975) first made a number of observations of strategies used by

(5)

“good language learners.” Numerous studies (O’Malley et al., 1985; Politzer & McGroarty, 1985; Oxford, 1986) further established the taxonomies for the behaviors used by successful language learners by analyzing the type, variety, and frequency of strategy use. In 1990, Oxford developed the Strategy Inventory for Language Learning (SILL), which was used or adapted as an instrument in the subsequent SILL studies to investigate learning strategies and the relationships between strategy use and language performance. According to Oxford (1990), language learning strategies help learners to enhance their language learning and to develop their communicative competence. For instance, cognitive strategies, such as analyzing, and certain memory strategies, like the keyword technique, are useful for understanding and recalling new information in the process of learning a new language. Metacogntive strategies help learners to regulate their own cognition, plan their learning, monitor their progress, and evaluate their achievements.

In most of the SILL studies, the results generally showed that learning was affected in a positive manner when such strategies were used and higher frequency of strategy use was associated with more proficient learners (e.g., Green & Oxford, 1995). Therefore, it is believed that language teachers should help their students to increase awareness of learning strategies and to develop their ability to use the strategies. In line with this, both O’Malley and Chamot (1990) and Oxford (1990) provided teachers with practical recommendations for developing students’ L2 learning strategies in the classroom and training students to apply such strategies to each of the four language skills.

(6)

strategies could be divided into two broad types, i.e., learning strategies and use strategies. Learning strategies are those used purposefully by learners to enhance L2 learning and acquisition, whereas use strategies are those used to enhance L2 performance in a communicative language task or in a language test. Phakiti further discussed the differences between these two. In a non-testing situation, learning strategies help learners acquire language knowledge and store it in long-term memory; however, when in a test or target language use (TLU) situation, strategy use is related to the working memory that is associated with the short-term memory regarding the TLU and helps learners to retrieve necessary knowledge from the long-term memory. Therefore, LT researchers are more interested in looking at use strategies than learning strategies. This review will now turn to three empirical LT studies on the relationships of test-takers’ strategy use to L2 test performance.

Purpura (1997, 1998, 1999) investigated the psychometric characteristics of an 80-item cognitive and metacognitive strategy questionnaire and used it to investigate strategy use and test performance of 1,382 EFL test-takers, using the First Certificate of English Anchor Test that included 20 items of grammar and vocabulary, 10 items of passage comprehension, 10 items of word formation, 20 items of cloze, and 10 items of sentence formation. In Purpura’s work, a three-factor model of cognitive strategy use (comprehending, storing/memory, and using/retrieval processes) and a one-factor model of metacognitive strategy use (assessment) were proposed through structural equation modeling, which is more effective than other statistical procedures for simultaneously

(7)

investigating a number of independent and dependent variables. The results demonstrated that metacogntive processing (MP) had no direct effect on test performance but did have a significant, positive, and direct influence on learners’ cognitive processing (CP), particularly on both memory and retrieval processes. The results showed that MP seemed to exert an executive function over CP. The results also indicated that CP did not have a significant influence on reading ability or grammar ability. However, memory processes were reported to have a significant, negative effect on grammar ability as it was found that the more frequently reported use of memory strategies was associated with poorer performance; that is, the less frequently the test-takers used the memory strategies, the better they performed.

Phakiti (2003) examined the strategy use of 384 Thai university students and its relationships to a reading comprehension achievement test that included gap-filling cloze and reading comprehension items. In the study, a 35-item strategy questionnaire derived from Purpura’s study (1999) was used. To supplement the quantitative data with qualitative analysis, eight of the students were selected for retrospective interviews. Phakiti reported a positive but weak relationship between the cognitive and metacogntive strategy use and reading test performance. The results also indicated that the more proficient students reported more frequent use of metacogntive strategies.

Song and Cheng (2006) explored the relationships between strategy use reported by 121 Chinese EFL learners and their performance on College English Test-Band 4, a national English proficiency test in China. Song and Cheng also used a 64-item questionnaire that was derived from Pupura’s study (1999). The

(8)

questionnaire consisted of 34 items of cognitive strategy use and 30 items of metacognitive strategy use. The cognitive strategy items were grouped into 10 strategy-type variables (strategy use at the subscale level), measuring three underlying process-type scales (strategy use at the scale level); the metacognitive items were grouped into four strategy-type variables, measuring one underlying scale. The results generally revealed that this group of learners used more metacogntive strategies than cognitive strategies. Specifically, inferencing (one subscale of cognitive strategies) was the most frequently used strategy. At the scale level, memory and retrieval was the only significant predictor of test performance, accounting for 8.6% of the variance in the test. At the subscale level, inferencing and practicing

naturalistically were the best predictors of test performance.

The three empirical studies reviewed above all suggest that there are complex relationships between learners’ strategy use and their performance in L2 testing. The findings of these studies have added new input to the theory and research of cognitive and metacogntive strategy use and L2 test performance. Based on the results of the three empirical studies, more research in this area should be conducted to address the need for a better understanding of strategic competence in L2 test performance across cultural and geographical settings.

METHOD

Participants

(9)

GEPT-Intermediate practice test, which is normally given two weeks prior to the operational GEPT administration with an aim to providing prospective test-takers with a simulated test under the genuine GEPT test conditions. A total of 567 test-takers were invited to take the GEPT-Intermediate practice test in January 2007. They were asked to fill out the strategy use questionnaire before the practice test started. Among them, 345 learners were female (61%) and 222 were male (39%). The ages of the learners ranged from 14 to 19, with a mean of 17.09, close to the profile of actual GEPT-Intermediate test-takers.

Instruments

Song and Cheng’s questionnaire (2006), which was developed based on Purpura’s work (1999), was translated and employed in this study to elicit information on strategy use (see Appendix). Following the procedures employed in Song and Cheng (2006), 34 items of cognitive strategy use were grouped into 10 subscales for strategy-type variables, which were to measure three scales or underlying process-type scales. Thirty items of metacognitive strategy use were grouped into four subscales for strategy-type variables, which were to measure two underlying scales. Tables 1 and 2 present the cognitive processing (CP) variables and metacognitive processing (MP) variables, respectively. The questionnaire used a 5-point Likert scale (A to E) to indicate the frequency of each strategy use (A for ‘never’; E for ‘always’). For the purpose of the present study, all the questionnaire items were translated into Chinese. The translated questionnaire was piloted among a small sample to check whether the statements were clearly worded and understood. Adjustments were made on the basis of the pilot results.

(10)

Table 1

A Taxonomy of the CP Variables (34 items)

Cognitive Processing (CP) No. of Items (Items Used) 1 Comprehending Processes (COMP)

z Analyzing inductively z Clarifying/verifying z Inferencing 3 (30, 33, 34) 2 (15, 32) 2 (28, 31) 2 Storing/Memory Processes (MEM)

z Associating

z Linking with prior knowledge

z Repeating/rehearsing z Summarizing 4 (7, 8, 9, 10) 4 (1, 2, 4, 17) 5 (3, 22, 23, 24, 25) 2 (5, 27)

3 Using/Retrieval Processes (RET) z Applying rules z Practicing naturalistically z Transferring 3 (6, 13, 26) 5 (16, 18, 20, 21, 29) 4 (11, 12, 14, 19) Table 2

A Taxonomy of the MP Variables (30 items)

Metacognitive Processing (MP) No. of Items (Items Used) 1 Online Assessment Processes

z Assessing the situation (before the event)

z Monitoring (during the event)

8 (35, 36, 45, 50, 52, 53, 54, 59)

4 (3, 44, 47, 55)

2 Post-Assessment Processes (after the event)

z Self-Evaluating

z Self-Testing

10 (39, 40, 46, 49, 51, 56, 58, 60, 63, 64) 8 (37, 38, 41, 42, 48, 57, 61, 62)

(11)

The GEPT

The GEPT is a five-level criterion-referenced EFL testing system implemented in Taiwan to assess the general English proficiency of EFL learners. The aim of the GEPT is to promote the concept of life-long learning and to encourage the use of the communicative approach in English teaching and learning.

Each level of the GEPT consists of four components of assessment: listening, reading, writing, and speaking. Those test-takers who pass the first-phase test, which includes listening and reading, are entitled to register for the second-phase test, which includes speaking and writing. The GEPT-Intermediate is the second level of the five-level EFL testing system. A learner who passes the GEPT-Intermediate “can use basic English to communicate about topics in daily life. His/her English ability is roughly equivalent to that of a high school graduate in Taiwan.” (LTTC, 2007).

A number of studies related to the GEPT have been conducted on parallel-form reliability (Weir & Wu, 2002), concurrent validity of the GEPT Intermediate Level and GEPT High-Intermediate Level (LTTC, 2003), mapping the GEPT to the Common English Yardstick for English Education in Taiwan (LTTC, 2005), written language of test-takers (Kuo, 2005), test impact (Wu & Chin, 2006; Wu, 2008), test form and individual task comparability (Weir & Wu, 2006), and mapping the GEPT reading tests to the CEFR (Wu & Wu, 2007). The results of the studies show that the test employs a set of standardized procedures in the administration and has maintained high reliability and validity.

(12)

Data Collection and Analysis

The participants were asked to respond to the questionnaires before they performed the GEPT practice test. After the questionnaire responses were collected, the participants’ responses were keyed in to the computer manually and were double-checked later. To perform statistical analysis, the responses coded in alphabet (A for ‘never’; E for ‘always’) were recoded to numbers (1 for ‘never’; 5 for ‘always’). The participants’ test ID numbers were used to link the scores on the GEPT-Intermediate with the questionnaire responses. The questionnaire responses and test scores were then analyzed using SPSS 14.0 (SPSS, 2006).

Various statistical analyses were conducted. First, descriptive statistics at the item level were obtained to examine how the participants reported using each of the strategies. Second, descriptive statistics at the subscale level for both cognitive strategy use and metacogntive strategy use were calculated. Additionally, a paired t test was run to investigate whether there was a significant difference between cognitive strategy use and metacogntive strategy use. Third, regression analysis was performed to explore the statistical relationships between strategy use and language proficiency. Finally, ANOVA analysis was performed to investigate the extent to which the strategy use employed by the higher achievers in test performance differed from that used by lower achievers. For this purpose, the participants were divided into three ability groups according to their GEPT performance by the sum score and the part scores for reading and listening.

(13)

RESULTS AND DISCUSSION

The major findings of this study will be summarized and discussed according to the research questions stated earlier.

Research question 1: What types of language strategies did the

GEPT-Intermediate test-takers under investigation report using?

The distributions for the 64 items of strategy use were first calculated. The means of these items ranged from 1.73 to 4.34, and the standard deviations ranged from .74 to 1.25. The majority of skewness and kurtosis values ranged between 1 and –1. All values for skewness and kurtosis were within the acceptable limits ( ± 2), indicating that the responses for the individual items were normally distributed. Therefore, all the items were retained for the subsequent analyses.

Forty-six strategies (72% of the total items) were used at a high rate, ranging from 3.01 to 5.0. Seventeen strategies (27% of the total items) were used at a medium rate, ranging from 2.0 to 3.0, and only one item was used at a lower rate. Table 3 presents the five most frequently used strategies, and Table 4 presents the five least frequently reported strategies. Among the five most frequently used strategies, four involved metacogntive strategy use and only one involved cognitive strategy use. As for the five least frequently used strategies, the opposite pattern was found in that four involved cognitive strategy use and only one involved metacogntive strategy use.

(14)

Table 3

The Five Most Frequently Reported Strategies (rank orders)

Rank Item Subscales Scales M/

SD 1 When I am taking an English test, I

try to concentrate on what I am doing (Item 51).

Self-Evaluating Post-Assessment (Metacognitive)

4.34/ 0.74

2 Before I write a composition in English, I plan my work (Item 64).

4.19/ 0.89 3 When I listen to English, I

recognize when I have not understood something (Item 40).

4.06/ 0.88

4 When someone is speaking English, I try to concentrate on what the person is saying (Item 46).

4.01/ 0.82

5 I try to improve my reading in English by guessing the meaning of new words from context (Item 31).

Inferencing Comprehension processes

(Cognitive)

3.95/ 0.82

(15)

Table 4

The Five Least Frequently Reported Strategies (rank orders)

Rank Item Subscales Scales M/

SD 1 I try to improve my English by

spending time with English-speaking people (Item 16). Practicing naturalistically Using/Retrieval processes (Cognitive) 1.73/ 0.95

2 Before I begin an English test, I think about how the test will be scored (Item 45). Assessing the situation (before) On-Line assessment processes (Metacognitive) 2.19/ 1.13

3 I try to improve my English by analyzing the ways that other writers show relationships between ideas (Item 34). Analyzing inductively Comprehending processes (Cognitive) 2.34/ 0.92

4 I try to improve my English by analyzing how other writers organize their paragraphs (Item 33).

Analyzing inductively Comprehending processes (Cognitive) 2.41/ 0.91

5 Before I begin an English test, I decide how important it is for me to get a good grade on the test (Item 53).

Assessing the situation On-Line assessment Processes (Metacogntive) 2.42/ 1.19

Further, a paired-sample t test was conducted for a comparison between cognitive strategy use and metacognitive strategy use. The results showed there was a significant difference between these two types (t=-13.846, p=.000), indicating that the learners reported using more metacognitive strategies than cognitive strategies.

(16)

This observed phenomenon is consistent with the findings reported in an earlier study in which Chinese EFL learners’ strategy use was also examined (Song & Cheng, 2006). The similarity may be due to the fact that the EFL learners in both China and Taiwan are all Chinese speakers and share some commonality in terms of L1 and cultural background. In addition, given that both studies were conducted in an L2 testing situation where the learners usually tend to focus on performance (product-oriented) rather than on learning (process-oriented), it seems reasonable that metacogntive strategies, particularly those engaged with post-assessment processes such as

self-evaluating and self-testing, were employed more often by

learners.

Looking closer at the least-used cognitive strategies at the subscale level, it was found that practicing naturalistically and

analyzing inductively were used least often. Two reasons might

account for the low employment of these strategies. First, given that Taiwanese learners tend to be shy in conversing with people in English and have relatively fewer practice opportunities to improve English by talking with English-speaking people than EFL or ESL learners in other countries where English speakers and learning resources in English are more accessible, it seems reasonable for the learners in the present study to report less use of strategies relating to practicing naturalistically. Second, among the four macro-skills, writing is often perceived by Taiwanese EFL learners as the most difficult skill to develop. The perception of Taiwanese EFL learners was well supported by the GEPT score data analysis (LTTC, 2008), which showed that the test-takers’ writing performance was limited

(17)

by poor ability in organizing ideas or paragraphs, which is exactly the task requiring learners to demonstrate whether they can analyze

inductively or not. Therefore, it is not surprising to see the result

indicating that the strategies involving analyzing inductively were rarely employed.

Research question 2: What relationships, if any, existed between

reported strategy use and language test performance in the GEPT-Intermediate? Which strategy use was the best predictor of GEPT performance at the intermediate level?

To answer the research question addressing the relationships between strategy use and test performance on the GEPT-Intermediate, stepwise multiple regression analysis was performed to estimate the prediction equation between these two variables. The subscales were treated as independent variables, and the sum score, reading score, and listening score were treated as the dependent variables.

In the previous research of this kind, the dependent variables under investigation were only the score for the reading test (Purpura, 1997; Phakiti, 2003) or the sum score of the whole test resulting from the consolidated performance of test-takers on listening, reading, and writing (Song & Cheng, 2006). However, to further explore the relationships between strategy use and test performance, it is more appropriate to treat different aspects of language performance as separate dependent variables. In light of this, in this study regression was performed three times with three different GEPT-Intermediate scores, with sum score, reading score, and listening score as dependent variables, and with the measures of cognitive strategy use and metacognitive strategy use as independent variables. The major

(18)

findings are summarized and discussed below based on the figures presented in Tables 5-7.

1. Five contributory variables were found (accounting for 14% of the sum score variance), including linking with prior knowledge,

assessing the situation, monitoring, inferencing, and applying rules. Among them, it was noted that both assessing the situation

(in relation to online assessment processes) and applying rules (in relation to using/retrieval processes) had a negative effect on the overall GEPT-Intermediate test performance. These findings are consistent with Paris’s suggestion (2002) that strategies can have “positive, negative, and neutral consequences” (cited in Song & Cheng, 2006).

2. Looking further at strategy use as predictors for GEPT performance, two predictors (assessing the situation and

monitoring) were identified repeatedly in three different test

results in terms of sum score, reading score, and listening score. Moreover, it is worth noting that assessing the situation had a negative consequence on test performance (with a minus B value of -13.292 to -5.398), whereas monitoring had a positive consequence (with a positive B value of 4.120 to 10.092). The unstandardized regression coefficient B indicated that 1 scale point of increase on assessing the situation might predict a decrease in the sum score of approximately 13 points, whereas 1 point of increase on monitoring might predict an increase in the reading score of approximately 4 points.

The question of why the strategy use relating to assessing the

(19)

answered when we examine some of the statements under

assessing the situation in the questionnaire. The statements were

mostly about what learners normally do either before or during a language performance situation, for example, Item 36 (Before I

begin an English test, I try to see which parts will be easy and which parts will be difficult) and Item 45 (Before I begin an English test, I think about how the test will be scored). It seems

logical to infer that strong learners with more confidence in their language proficiency may be concerned about the test items themselves or about how to answer all of the test items; whereas weaker learners, who may not be able to answer all of the test items, may feel the test is too difficult for them to complete, leading them to assess the situation in order to decide how to optimize their performance on the test. Thus, the strategy use relating to assessing the situation was found to be associated with poorer test performances.

Unlike assessing the situation, strategy use related to

monitoring was found to have a positive relationship with the

GEPT-Intermediate performance. The statements for monitoring, such as Item 43 (When I am speaking English, I know when I have

not pronounced something correctly) and Item 55 (When I listen to English, I recognize other people’s grammar mistakes) were

about the extent to which learners are aware of errors made in using English. When learners can identify errors in grammar or pronunciation, it shows that they have acquired the accurate form or pronunciation and are more likely to produce more accurate language. This, therefore, may explain why the use of

(20)

GEPT-Intermediate performance.

To sum up, the results of regression analyses suggest that learners’ use of strategies can explain only a limited portion of test performance and many other variables remain to be explored to explain the rest of the variation in the complexity of L2 proficiency. In addition, it is interesting to find that not every type of strategy use helps test performance and the variation in the degree or direction of the effect may be caused by different abilities or skills assessed in the test being investigated. The implications of these findings for teaching and learning will be discussed in the following section.

Table 5

Multiple Regression: Strategy Use on Sum Score

R2 B Beta t p

(Constant) 137.200 14.587

Linking with prior knowledge .045 7.428 .147 2.885 .004 Assessing the situation .100 -13.292 -.273 -6.620 .000 Monitoring .126 10.092 .202 4.311 .000

Inferencing .134 4.651 .107 2.411 .016

(21)

Table 6

Multiple Regression: Strategy Use on Reading Score

R2 B Beta t p

(Constant) 66.324 12.819

Assessing the situation .044 -7.764 -.270 -6.464 .000 Linking with prior knowledge .093 4.818 .161 3.580 .000

Monitoring .107 4.120 .139 2.967 .003

Table 7

Multiple Regression: Strategy Use on Listening Score

R2 B Beta t p

(Constant) 71.701 15.393

Practicing naturalistically .044 2.375 .102 2.050 .041 Assessing the situation .076 -5.398 -.218 -5.236 .000

Monitoring .109 5.361 .211 4.208 .000

Inferencing .120 3.051 .138 3.118 .002

Self-testing .131 -4.580 -.172 -3.221 .001

Repeating/rehearsing .139 2.689 .117 2.213 .027

Research question 3: Are there any differences between high

achievers and lower achievers in reported strategy use and in the relationships between strategy use and test performance?

To further explore the relationships between strategy use and test performance, ANOVA analysis was performed to compare learners’ strategy use at the subscale level when proficiency was controlled as a

(22)

moderate variable. Participants in this study were divided into three groups (High, Medium, and Low) by the GEPT sum score, reading score, and listening score (see Table 8).

Table 8 Ability Groups

Ability Groups Scores Range N

High Sum Reading Listening ＞186 ＞93 ＞93 122 92 165 Medium Sum Reading Listening 150-186 75-93 75-93 217 201 257 Low Sum Reading Listening ＜150 ＜75 ＜75 228 274 145

According to the ANOVA results, significantly different strategy use between the stronger and the weaker test-takers was found repeatedly in five of the subscales across sum score, reading score, and listening score. The five common subscales are inferencing,

linking with prior knowledge, repeating/rehearsing, assessing the situation, and monitoring. The first three are related to CP and the

other two are related to MP. The results clearly indicated that the stronger learners reported a higher frequency of strategy use, except for those related to assessing the situation. In other words, the

(23)

stronger learners actually used strategies relating to assessing the

situation less frequently than the weaker learners. This again supports

the data indicating that assessing the situation had a negative relationship with test performance according to the results of the regression analysis reported earlier in the paper. By grouping the strategy subscales into two broad dimensions, cognitive and metacognitive strategy use, it was noted that on the whole the stronger learners employed both cognitive and metacogntive strategies significantly more often than the weaker ones, which is consistent with the results reported by Phakiti (2003).

Based on the findings about the relationships between strategy use and test performance reported earlier in this paper, we have noted that the variation in learners’ strategy use may be caused by different abilities or skills assessed in the test (listening vs reading). As a follow-up to this finding, when comparing strategy use between higher achievers and lower achievers, it is worthwhile to trace the relationships within each area of performance (listening vs reading) in which particular strategies may be used more often by a higher achiever, but less often by a lower achiever, and vice versa. For example, it was found that

1. Analyzing inductively (under the level of CP) failed to differentiate higher and lower achievers in the listening performance as it did in the reading performance.

2. Self-Testing and self-evaluation (under the level of MP)

also failed to differentiate higher and lower achievers in the listening performance as they did in the reading performance.

(24)

the process during a listening test. First, analyzing inductively is more closely associated with reading skills, which require learners to look for the ways that writers show relationships between ideas, rather than with listening skills, which normally require more decoding and encoding of input. Therefore, a good reader rather than a good listener is likely to be able to employ a more frequent use of analyzing

inductively. Second, by the same token, when a listening activity is

performed, learners are required to concentrate on what is being heard, as one of the statements for self-evaluation (Item 40) says, ‘When I

listen to English, I recognize when I have not understood something.’

Under this circumstance, it is therefore reasonable to suggest that learners, irrespective of their proficiency, are equally capable of evaluating their listening comprehension.

LIMITATIONS AND IMPLICATIONS

The present study aimed to explore the complex relationships between Taiwanese EFL learners’ cognitive and metacogntive strategy use and their performance on the GEPT test. This work has revealed a number of interesting findings, yet they should not be considered conclusive or comprehensive due to some limitations in this study. For example, methodologically, in terms of the data collection procedures, the questionnaire was conducted in Chinese to ensure that the participants understood it. However, though the translated questionnaire was piloted and necessary adjustments were made before it was used in the study, the translated text of each strategy

(25)

might still not have had the exact same meaning to the readers in this study as that intended in the original English version. The possible differences in meaning due to translation might have had an impact on what was reported in this study. Besides, for the sake of efficiency, the present study used a questionnaire to gather learners’ self-reports of strategy use. Basing research only on a quantitative method has its limitations in obtaining more in-depth reports on strategy use from learners. Follow-up interviews with some of the participants should be added in future studies of this kind. In regard to statistical procedures, although regression analyses were conducted to explore the relationships between strategy use and test performance, more sophisticated statistical analysis, such as structural equation modeling (Kunnan, 1998) has yet to be employed.

Despite the limitations, the findings in this study suggest a number of implications for pedagogy and future research as follows: 1. Strategy use varied due to learners’ proficiency of English. On the

whole, the higher achievers in the GEPT-Intermediate performance reported more use of both cognitive and metacognitive strategies. Furthermore, the strategies reported as used more often by the higher achievers in the GEPT-Intermediate were related to inferencing, linking with prior knowledge,

repeating/rehearsing, and monitoring. Therefore, learners should

be made aware of the importance of the active use of such strategies. As suggested by Green and Oxford (1995), it is hoped that sharing results such as those in the study will be useful in encouraging L2 learners to use such strategies. Relating this to the Taiwanese context, as Yang (1999) noted, Taiwanese EFL learners’ self-efficacy beliefs were strongly related to their use of

(26)

learning strategies; therefore, encouraging appropriate beliefs about L2 learning should also be included in strategy training programs for learners.

2. A few strategies such as analyzing inductively and self-evaluation failed to differentiate higher and lower achievers in the listening performance as they did in the reading performance, which suggests that the use of strategy may also be related to language skills. Therefore, it is also useful for teachers to recognize that some strategies may be more suited to a particular language skill than others so that more effective strategy choices can be introduced to their students.

3. The results of regression analyses indicated that learners’ use of strategies could explain only a limited portion of the test performance. This finding tells us that there are other strategies (e.g., affect and self-efficacy) and many other variables (e.g., task types, difficulty, test delivery) to be explored to obtain a more comprehensive view of learners’ cognitive and metacogntive strategy use. It is therefore recommended that more research into the effect of these variables, alone or in combination, on learners’ strategy use should be conducted.

4. Learners’ strategy use was investigated in a testing situation in this study. However, learners’ actual strategy use in a non-testing situation is likely to be different from that in a testing situation (Phakiti, 2003). Therefore, to better understand the complexity of learners’ psychology in using learning strategies under a testing operation, a combination of quantitative and qualitative methods and more refined statistical procedures are desirable in future research.

(27)

ACKNOWLEDGEMENTS

This study was supported by the Language Training and Testing Center (LTTC). I would like to extend my gratitude to my colleagues on the LTTC Research Team for providing necessary assistance to this study.

REFERENCES

Bachman, L. F. (1990). Fundamental considerations in language

testing. Oxford: Oxford University Press.

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.

Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied

Linguistics, 1, 1-47.

Cohen, A. D. (1998). Strategies in learning and using a second

language. New York: Addison-Wesley.

Green, J. M., & Oxford, R. (1995). A closer look at learning strategies, L2 proficiency, and gender. TESOL Quarterly, 2, 261-297.

Kunnan, J. A. (1998). An introduction to structural equation modeling for language assessment research. Language Testing, 15(3), 295-332.

Kuo, G. (2005). A preliminary corpus study on EFL test takers’

writing proficiency. Paper presented at the Eighth International

Conference on English Language Testing in Asia, Hong Kong.

(28)

studies of the GEPT intermediate level, GEPT high-intermediate level, CBT TOEFL, CET-6, and the English Testing of the R.O.C. College Entrance Examination. Retrieved November 17, 2007, from http://www.lttc.ntu.edu.tw/academics/testmain.htm

Language Testing and Training Center [LTTC]. (2005). Mapping the

GEPT to the common English yardstick for English education in Taiwan (CEY). Taipei: Language Training and Testing Center.

Language Testing and Training Center [LTTC]. (2007). General English

Proficiency Test. [Brochure]. Retrieved November 17, 2007, from

http://www.lttc.ntu.edu.tw/E_LTTC/gept_eng_main.htm

Language Testing and Training Center [LTTC]. (2008). GEPT score data

report. Retrieved December 10, 2008, from

http://www.lttc.ntu.edu.tw/academics/testmain.htm

McNamara, T. F. (1996). Measuring second language performance. London and New York: Longman.

O’Malley, J. M., & Chamot, A. U. (1990). Learning strategies in

second language acquisition. Cambridge: Cambridge University

Press.

O’Malley, J. M., Chamot, A.U., Stewner-Manzanares, G., Kupper, L., & Russo, R. (1985). Learning strategies used by beginning and intermediate ESL students. Language Learning, 35, 21-46. Oxford, R. (1986). Development of the strategy inventory for

language learning. Paper presented at the Language Testing

Research Colloquium. Monterey, CA.

Oxford, R. (1990). Language learning strategies: what every teacher

should know. Boston: Heinle and Heinle.

(29)

benign? In P. Chambres, M. Izaute, & P. Marescaux (Eds.),

Metacognition: Process, function, and use (pp. 105-120).

Norwell, MA: Kluwer.

Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test performance. Language Testing, 20, 26-56.

Politzer, R. L., & McGroarty, M. (1985). An exploratory study of learning behaviors and their relationship to gain in linguistic and communicative competence. TESOL Quarterly, 19, 103-123. Purpura, J. E. (1997). An analysis of the relationships between

test-takers’ cognitive and metacogntive strategy use and second language test performance. Language Learning, 47, 289-325. Purpura, J. E. (1998). Investigating the effects of strategy use and

second language test performance with high- and low-ability test-takers: a structural equation modeling approach. Language

Testing, 15, 333-379.

Purpura, J. E. (1999). Learner strategy use and performance on

language tests: a structural equation modeling approach.

Cambridge: University of Cambridge Local Examinations Syndicate and Cambridge University Press.

Rubin, J. (1975). What the “good” language learner can teach us?

TESOL Quarterly, 9, 41-51.

SPSS (2006). SPSS Version 15.0 (Computer software). Chicago, IL: SPSS.

Song, X., & Cheng, L. (2006). Language learner strategy use and test performance of Chinese learners of English. Language

Assessment Quarterly, 3, 244-266.

(30)

of the GEPT spoken performance test. Proceedings of the Fifth

International Conference on English Language Testing in Asia,

Tokyo, Japan.

Weir, C. J., & Wu, J. (2006). Establishing test form and individual task comparability: a case study of a semi-direct speaking test.

Language Testing, 23(2), 167-197.

Wu, J. (2008). Views of Taiwanese students and teachers on English language testing. Research Notes, 34, 6-9.

Wu, J., & Wu, R. Y. (2007). Using the CEFR in Taiwan: the

perspective of a local examination board. Paper presented at The

Fourth Annual EALTA Conference, Sitges, Spain.

Wu, R. Y., & Chin, J. S. (2006). An impact study of the intermediate

level GEPT. Paper presented at the Ninth International Conference on English Testing in Asia, Taipei. Taiwan.

Yang, N. D. (1999). The relationship between EFL learners’ beliefs and learning strategy use. System, 4, 515-536.

ABOUT THE AUTHOR

Jessica Wu is Head of the Testing Editorial Department at the LTTC where she handles the issues of validation, research and test development for the various foreign language testing programs. She received her PhD from the University of Surrey, U.K. She is currently a member of the Editorial Board of Language Assessment Quarterly.

(31)

APPENDIX

Questionnaire on Learner Strategy Use

本問卷的目的是希望瞭解 LTTC 全民英檢考生所使用之學習策略，因此問卷的答案並沒有絕對的對或錯，請盡量據實填寫問卷；本調查結果僅作為研究之用，個人資料完全保密，也絕對不會影響您的成績。請利用約 20 分鐘的時間填寫問卷，答完後，請在原位靜候，待試場內其他考生全部填寫完畢，監試老師收完問卷，再開始今天的抽樣測驗。謝謝您的協助。請將最適合您目前狀況的敘述於 A、B、C、D 或 E 圓圈內塗黑。當我學習新的英語教材時，從來不偶爾有時候經常總是 1. 我會嘗試把過去所學的和新學的作結合。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 2. 我會嘗試把所學的東西在心裡自己先整理出一些脈絡。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 3. 我會重複唸新學的字，確定我已經學會並瞭解。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 4. 我會用過去的經驗，來幫助我學到更多的東西。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 5. 我會摘要記下我聽到或讀到的東西。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 6. 先學習規則，對我的學習最有幫助。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 當我學習新的英文單字時，從來不偶爾有時候經常總是 7. 我會把新學單字的發音和我熟悉的字的發音聯想在一起。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 8. 我會回想第一次看到（例：在書裡面的哪一頁）或聽到那個字的地方。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 9. 我會聯想那些字所代表的外觀、感覺、氣味、聲音或者味道。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 10.我會把新的單字和曾學過且發音相近的單字聯想在一起。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ

(32)

當我學習英文文法時，從來不偶爾有時候經常總是 11.我會用中文的文法來幫助我學習英文的文法規則。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 12.我會比較中文和英文的文法規則。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 13.我會熟記英文文法規則，並運用它。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 為了讓我自己的英文進步，從來不偶爾有時候經常總是 14.我會嘗試在中文裡面找出和英文類似的字詞。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 15.我會請別人告訴我，我對英文的理解或我所說的英文是不是正確。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 16.我會找時間和美/英國人聚會。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 17.我會嘗試使用我學到的英文。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 18.我會盡量找機會說英文。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 19.我會嘗試找出並且使用和中文類似的英文字詞。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 為了讓我自己的英文聽力進步，從來不偶爾有時候經常總是 20.我會看英語電視節目。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 21.我會聽英語廣播節目。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 為了讓我自己的英文口說能力進步，從來不偶爾有時候經常總是 22.我會複誦所學到的東西。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 23.我會持續練習新學單字的發音，一直到我的發音像美/英國人一樣。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 24.我會持續複誦新學的英文句子，一直到我能夠輕鬆的把這些句子說出來。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 25.聽到美/英國人所說的話後，我會持續複誦。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 26.我會運用所學到的英文文法規則來造句。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 為了讓我自己的英文閱讀能力進步，從來不偶爾有時候經常總是 27.我會摘要記下讀到的東西，以便記住它。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 28.我會盡量不要一遇到新的單字就查字典，試試自己的閱讀能力。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ

(33)

Wu: Strategy Use and GEPT Test Performance 30.我會留意作者如何組織不同的想法。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 31.我會從前後文猜我本來不認識的單字。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 為了讓我自己的英文寫作能力進步，從來不偶爾有時候經常總是 32.我會請別人看我寫的東西。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 33.我會分析別的作者如何組織他們的文章段落。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 34.我會分析別的作者如何組織不同的想法。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 其他從來不偶爾有時候經常總是 35.用英文表達前，我會先想：我的文法有沒有好到可以表達我的想法。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 36.考英文的時候，我會先看一看哪些部分簡單、哪些部分難，再開始作答。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 37.我會思考如何有效地學習英語。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 38.學到新的英文單字或片語的時候，我會確定自己是不是已經記下這些單字或片語。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 39.英文考試交卷前，我會先檢查答案。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 40.聽英文時，我知道自己有沒有完全聽懂。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 41.我會想運用所學到的英文文法規則，來確定我是否已經學會了。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 42.我會想運用所學到的英文單字，來確定我是否已經學會了。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 43.我說英文時，自己知道有些字的發音不正確。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 44.我說英文時，自己知道哪些地方說得像美/ 英國人一樣。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 45.考英文的時候，我會先想一想考試分數如何計算，再開始作答。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 46.別人說英文時，我會盡量專心聽他/她在說什麼。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 47.我說英文時，自己知道哪些地方犯了文法錯誤。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 48.考完英文後，我會檢討下次應如何改進。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ

(34)

49.學英文的時候，我會從自己的錯誤中學習。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 50.考英文的時候，我會先想一想哪些部分最重要，再開始作答。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 51.考英文的時候，我會盡量專心作答。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 52.開始寫英文作業前，我會先想：以我的英文能力，能不能完成這個作業。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 53.考英文的時候，我會先想一想這個考試的分數的重要性，再開始作答。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 54.用英文表達前，我會先想：如果我沒辦法表達清楚或我不知道該用哪個字的時候該如何處理。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 55.聽別人說英文時，我會注意到別人哪些地方犯了文法錯誤。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 56.用英文和別人交談後，我會檢討如何說得更好。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 57.我會竭盡所能的從書報雜誌的文章中，找出學好英文的方法。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 58.說英語時，我會確認對方是不是真得聽懂我得意思。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 59.和別人用英文交談前，我會先想一想對方對我要說的東西有多少瞭解。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 60.當別人聽不懂我說的英文時，我會想辦法瞭解錯在哪裏。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 61.學了新的英文文法規則後，我會確認自己是不是真的知道如何使用這些文法規則。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 62.學到新的東西後，我會確認自己是否真的學會了。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 63.開始寫英文作業前，我會先準備好字典或其他參考書或工具書。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ 64.開始寫英文作文前，我會先想一想該怎麼寫。 Ⓐ Ⓑ Ⓒ Ⓓ Ⓔ

(35)

全民英檢考生之學習策略使用與測驗成績的關係

摘要本文報告全民英檢考生之學習策略使用調查並探討策略使用與測驗成績的關係。研究參與對象是 567 位全民英檢中級初試(含聽力、閱讀兩項測驗)考生，資料蒐集含學習者策略使用調查問卷與全民英檢中級初試成績(含總分、聽力與閱讀分項成績)。隨後藉由統計分析了解英語學習者在認知策略與後設策略的使用情形，並進一步探討其與測驗成績之間的關係。主要的研究發現含：後設策略使用較認知策略頻繁；部分策略之使用顯著與英語能力相關(對測驗成績具有 14%的解釋力)；學習者策略使用的效果因英語能力、技能 (聽、讀)的差異有所不同；英語能力較強的學習者使用的策略有別於英語能力較弱者，顯示有些策略確實有助於英語學習。本文最後除提出本研究的限制外，也提出教學建議及未來研究的方向。關鍵詞：學習策略使用認知策略與後設策略全民英檢