This study aims to investigate the effects the compulsory standardized examination may have on first-year college students’ motivation to learn English before and after they took the required GEPT. First-year students in National Chiao Tung University were recruited for this study. They completed two sets of
questionnaires before and after partaking in GEPT. Besides, they were asked to give their opinions on two open-ended questions for the purpose of understanding how they prepared for this compulsory standardized examination and how they viewed this policy of GEPT as a threshold for graduation.
In the following sections, we described our participants, instruments, procedures of data collection, and the statistical methods for data analysis.
Participants
Our sample comprised of five hundred and ninety first-year college students in National Chiao Tung University who took GEPT on May 12th 2007. These
participants came from thirteen Freshman English courses, with two hundred and sixty-nine coming from five reading courses, one hundred and nine from four
conversation courses, and another two hundred and twelve from four listening courses.
Students in each class came from diverse colleges, including colleges of electrical and computer engineering, computer science, engineering, science, biological science and technology, management, humanities and social science, and Hakka studies.
For the college students in National Chiao Tung University, English is a required course in the first academic year. They are required to register in two of three
Freshman English courses in the first two semesters. The English courses are
32
inclusive of reading, conversation and listening courses. Regulated by the school policy, it is stipulated that the first-year students have to take the first stage of the High-intermediate level of GEPT at the end of the first academic year. Additionally, GEPT is incorporated into the requirements of the English courses the students take.
It is a must for the students to take the listening and reading sections of the same level of GEPT as a mock, whose grades they obtain will be counted as 15-20% of their total scores of the courses they take.
Students who successfully pass the first stage (listening and reading sections) of GEPT can go on taking advanced English courses or second language courses of Japanese, French, and German, with a maximum of 4 credits. As for those who do not pass the examination, they can still take these courses but at the same time, they have to take another year’s study of remedial English before they graduate from the
university. The remedial English courses aim at improving listening and reading skills.
Students meet for an hour every week. The textbook “Read and Think” is the essential reading. Some teachers may add other supplemental materials such as Advanced magazines to their course reading. If students do not plan to take remedial English, they can choose to take GEPT again. Once they pass, it is not necessary for them to take remedial English courses.
Instruments
Questionnaires, including Likert-type statements and open-ended questions (Appendices D and E) were employed in the present study. Below, the instruments used for measuring variables were elaborated.
Self-efficacy beliefs
Duncan and McKeachie (2005) made full investigations into empirical studies
33
addressing the nature of motivation and the use of learning strategies and they found that entire or partial items in the Motivated Strategies for Learning Questionnaire (MSLQ) were used in fifty-six studies from 2000 to 2004. Since MSLQ is an instrument frequently applied to measure learners’ motivation across numerous disciplines, we adapted eight items (Questions 1 to 8 in Appendices D and E) for the measurement of self-efficacy beliefs, trying to investigate how our participants perceived their ability to learn English. The responses to these items ranged from strong disagreement to strong agreement, with a 5-point scaling.
Motivational intensity
In our questionnaires, ten items (Questions 9 to 18 in Appendices D and E) were intended to measure our participants’ motivational intensity, i.e., how much effort the students made when they learned English. These items were adapted from
Attitude/Motivation Test Battery (AMTB) (Gardner, Tremblay, & Masgoret, 1997). A 5-point Likert scale, with responses ranging from strongly disagree to strongly agree, was provided for our participants to choose what best suits their situation.
Intrinsic value
Seven items contained in the questionnaires administered before and after GEPT looked into the students’ interest in learning English (Questions 19 to 25 in
Appendices D and E). They were adapted from Remedios et al.’s research (2005). To maintain the consistency of our questionnaire and conform to the context of this study, we modified the interrogative sentences in Remedios et al.’s study (2005) into
declarative ones. Moreover, two questions (How interesting do you find English? &
How fun do you find English?) in the original version were merged into one sentence (I find English interesting) for we thought that interesting and fun mean the same
34 thing to Chinese students.
Actually, MSLQ (Duncan & McKeachie, 2005) comprises four items for measuring learners’ intrinsic goal orientation. Many researchers drew on the subscale of MSLQ in their studies (Campbell, 2001; Davenport, 2003; Green, 2001; Hancock, 2002; Hargis, 2001; Jacobson, 2000; Karabenick, 2004; Livingston, 2000; Longo, 2000; McManus, 2000; Polleys, 2001; Seibert, 2002; Vanzile-Tamsen, 2001). Instead of using the four items in the MSLQ, however, we adapted the items in Remedios et al.’s research (2005). The reason lies in the fact that the four items in the MSLQ mainly evaluate the effects of instructional strategies and course structures on students’ learning (Duncan & McKeachie, 2005). Nevertheless, the items developed by Remedios et al. (2005) are prone to explore the influences of tests or examinations on students’ interest in language learning, which is more pertinent to our study.
Therefore, we decided to adopt the items developed by Remedios et al. (2005) in our research.
Test motivation
Eight items in the questionnaire examined the extent to which the participants stressed this compulsory standardized examination and whether they were motivated to perform well on it (Questions 26 to 33 in Appendix D). These items were adapted from Wolf and Smith’s (1995) study. The researcher made minor changes on the wording, replacing “this test” with “GEPT” as a reminder for the participants that the focus in this study was GEPT instead of the midterm, final examinations or quizzes they took in class.
Students’ attitudes toward GEPT as a threshold for graduation
Inspired by Etten, Freebern, and Pressley’s research (1997) which reported on an
35
investigation into one hundred and forty-two college students’ beliefs about the process of examination preparation in the northern United States, we came up with two open-ended questions to elicit our participating students’ opinions regarding the issue being concerned in the second questionnaire survey (Appendix E). More specifically, we asked them whether the policy of passing a standardized examination works for their improvement of English proficiency. We also solicited ideas from our participants about their ways to prepare for and their attitudes to deal with a
standardized examination.
Procedures
As displayed in Table 3.1, this study was composed of a two-stage data collection. In Stage 1, three weeks before GEPT was administered, we made use of questionnaires to measure three motivational variables, including self-efficacy beliefs, motivational intensity (effort), and intrinsic value. In addition, we measured how motivated our participants were in preparing for GEPT, i.e., their test motivation.
In the second stage, two weeks after our participants obtained their GEPT grades, questionnaire survey was utilized again to examine the same variables of self-efficacy beliefs, motivational intensity and intrinsic value. Aside from the Likert-type items, two open-ended questions were included in the questionnaire to collect information concerning students’ attitudes over the school policy and their effort spent on the compulsory standardized examination. Both the questionnaires in the present study were written in the students’ native language -- Chinese.
The application of a two-stage data collection lay in two reasons. First, we made references to the past literature and discovered that most studies conducted pre- and post-tests and made comparisons in between. Secondly, due to administrative
concerns, we intended to execute two questionnaire surveys so as not to interfere with
36
teachers’ teaching and exhaust our participants. As to the timetable, Stage 1 was administered three weeks before GEPT since we thought that the compulsory
examination was approaching and may start to impact on our participants’ L2 learning motivation. Stage 2 was administered two weeks after our participants received their GEPT grades because we wanted to finalize our second questionnaire survey before our students’ upcoming final examinations.
Table 3.1 Instruments and Variables in the Two Stages of Data Collection Two-stage Data Collection
Stage 1 Stage 2
Instruments Questionnaire 1 Questionnaire 2 Variables - self-efficacy beliefs - self-efficacy beliefs
- effort - effort - intrinsic value - intrinsic value - test motivation - perspectives
towards GEPT
In the ensuing passages, we describe how the questionnaire items were translated from English into Chinese. Then, the process of pilot study was depicted. The
two-stage data collection was eventually addressed.
Translation and back translation of the questionnaire items
The items included in our questionnaire were originally English-written sentences. To conserve the time of questionnaire administration and ensure our participants’ understanding of all the items, we translated them from English into Chinese. At first, the researcher translated the English-version questionnaire into a Chinese-version one. Then, a graduate student not involved in this project was invited to translate the Chinese-version questionnaire back into another English version.
37
Following that, the researcher compared the two English versions of questionnaire items to ascertain the translated items were truthful to the original ones in meaning.
For the items that did not quite match in the two versions, the researcher modified their Chinese items and asked the outside student to translate these items again.
Among the questionnaire items, two questions measuring motivational intensity were discussed more often than the rest. The characteristic these two items had in common was that they began the sentences with “I don’t bother…” or “I can’t be bothered…”
This kind of sentence structure was more confusing when we tried to translate it into Chinese. The procedures for back translation of the questionnaire items were repeated twice. Until all the items were considered feasible were our questionnaires finalized.
Pilot study and its results
After we made sure the contents of our questionnaires, we did a pilot study to test its reliability. For items of self-efficacy beliefs, motivational intensity, and intrinsic value, they were distributed to seventy-three sophomores in three Advanced English courses in National Chiao Tung University. As for the items of test motivation, another eighty-three first-year college students in two Freshman English courses in the same university completed our questionnaire. Those who were recruited for the pilot study did not take part in our main study. The reason why we separated our questionnaire items and distributed them to different students is that the items
belonging to test motivation intended to elicit first-year college students’ responses to the level of importance of GEPT as perceived and consequently, it is better if we could have first-year college students involved in the pilot study. The rest of items were then distributed to the sophomores since we did not have so many participants and hoped that all of them got involved in the formal questionnaire survey, not the pilot study.
38
Table 3.2 demonstrated the reliability of each variable drawn from the pilot study.
As can be seen, the Cronbach alpha values for the items of self-efficacy beliefs, motivational intensity, intrinsic value and test motivation were .80, .70, .83, and .77.
The pilot results manifested that these items were acceptable in our study.
Table 3.2 The Cronbach Alpha Values for Subscales Self-efficacy
beliefs
Motivational intensity
Intrinsic value
Test motivation
Number of items 8 10 7 8
α value .80 .70 .83 .77
As for the open-ended questions in the second questionnaire, two college students not involved in this study were invited to go over them and provided suggestions on their wording. They indicated that the wording was appropriate and would not result in misunderstanding.
Two-stage data collection
In order to administer the questionnaires in the Freshman English courses, it is essential for the researcher to get permission from the English teachers who are responsible for teaching the first-year English reading, listening and conversation courses. Therefore, an email was forwarded to nineteen English teachers in National Chiao Tung University before the spring term 2007 started (Appendix C). The content of the email conveyed the purpose of this study and told these teachers the possible time for questionnaire administration. At the end of this email, these teachers were asked to give a short reply if they were willing to open their classes to the researcher.
Finally, we got responses from twelve English teachers who indicated that they agreed with the research in their classes.
39
After we were allowed to distribute questionnaires in these Freshman English classes, we administered questionnaire surveys in two time frames. The reason why we did our survey twice was that this study was intended to investigate how the
compulsory standardized examination may interfere with college students’ motivation.
To address the issue, we collected the data for two times and compared our
participants’ motivational changes in terms of the variables of self-efficacy beliefs, motivational intensity and intrinsic value. Moreover, the relationships between our participant’s GEPT grades and the variables mentioned above are another concern in this study. Accordingly, the second survey was administered after our participating students gained their scores on GEPT.
As noted in the research framework proposed in the first chapter (p. 6), we specified the time for questionnaire distribution. The first questionnaire survey was administered in the ninth week of the spring term 2007, i.e., three weeks before GEPT was held. The researcher went to each class and explained to the students the purpose of distributing the questionnaire. Following that, our participants spent ten minutes completing all the questions.
The second questionnaire survey was administered in the fifteenth week of the same semester, two weeks after our participants were notified of their GEPT grades. It was estimated that the questionnaire took our participants ten minutes or so to be finished.
As for the open-ended questions in the second questionnaire, we told our participants that their responses would be beneficial for the present study and they were encouraged to describe their experiences and express their opinions. The participating students were given sufficient time to complete this part.
40 Data Analysis
The collected data were analyzed by using SPSS 12.0. We drew on descriptive and inferential statistics to account for the concerned issues in this research. First of all, we presented the means and standard deviations of all the measured variables, including self-efficacy beliefs, motivational intensity, intrinsic value, test motivation and GEPT grades. Following that, we ran the analysis of correlation and
within-subject t-tests to answer the first four research questions. Correlation analysis was applied to answer the first research question, examining the relationships between our participants’ test motivation and the three motivational variables. For the second research question, we intended to explore whether our recruited undergraduates’
pre-test motivation correlated with their GEPT grades, and as a result, correlation analysis was used to probe the associations. After that, we made an investigation into the interrelatedness among the three motivational variables before and after the compulsory standardized examination by using correlation analysis. For the fourth research question, we observed the motivational changes of self-efficacy beliefs, motivational intensity and intrinsic value before and after the compulsory
standardized examination by running within-subject t-tests. Finally, the participants’
responses obtained from the open-ended questions were classified in terms of common themes, which offered information to help us investigate the last research question.
Through this research, we may identify the influences of compulsory
standardized examinations on students’ motivation to learn English. Moreover, we may get a better understanding of how the students perceive this examination as a requirement in their first year’s English learning in university.
41