Chapter 3 Research Methodology
Quantitative research methods were applied in this study. The results of a motivation questionnaire were analyzed to address the three research questions proposed in the first chapter. A pilot study was conducted first to test the reliability of the questionnaire items and to examine which items needed revision. The revised questionnaire was then distributed to students from Grade 3 to Grade 9 in Taiwan.
At least 150 students from each grade level were surveyed. Several statistical methods, including t-test, analysis of variance (ANOVA), and Pearson correlation, were applied to analyze results of the questionnaire.
Instruments
A motivation questionnaire was used as the major instrument in this study.
This questionnaire contains two sections. The first section enquires about participants’ background information, including their school and grade level (item 1), age (item 2), gender (item 3), and amount of exposure to English in different contexts, such as school (item 4), English education outside school (item 5), and family (item 6).
Items 7 to 10 were added to measure the socioeconomic status (SES) of the students’
family. All these items were selected based on several researchers’ (Chang et al., 2003; Chen, 1984; Pan, 2002) questionnaires on the part of background information so as to address the research questions.
Socioeconomic status (SES) in this study was assessed by the education level and occupation of students’ parents. Although SES is usually measured from three dimensions— education, occupation, and income (Cooper & Tom, 1984), considering that elementary and junior high students may not know the income of their parents, only two dimensions were resorted to, education and occupation, to place the
participants’ family socioeconomic status into three strata— high, mid, and low. It is also parents’ education level and occupation that were used as the indicator in another two studies (Pan, 2002; Wang, 1981) where the SES of six graders’ and junior high school students’ families were investigated. This ranking method is based on Hallingshead’s (1957) two-factor index of social position. Education degree is distributed to five levels—post graduate and above being the top level, assigned five points. The second is university and college level, assigned four points. Then follow high school level (three points), primary school level or literate without any schooling (two points), and illiterate level (one point). Parents’ occupations are classified into five categories as well. The top one is high-level professional and is given five points. This category includes occupations like university principals, university professors, legislators, doctors, etc. The second category, assigned four points, is labeled as professional, including principals and teachers in primary and high schools, judges, lawyers, accountants, engineers, etc. The third is semi -professional, given three points. Examples are like policemen, firefighters, secretaries, wholesalers, etc.
The fourth is skilled, assigned two points, and the last is unskilled, given one point.
Skilled occupations include barbers, cooks, drivers, tailors, postmen, etc., and unskilled include housewives, guards, waiters/waitresses, etc. The higher level of education and occupation of students’ parents will be selected into the calculation of SES. According to Hallingshead (1957), SES is calculated by the summation of the point of education times four and that of occupation times seven. SES is ranked as high if the total score is between 41 and 55, as mid between 30 and 40, and as low between 11 and 29, as shown in Table 1.
Table 1 The Calculation of Socioeconomic Status Ranks Education
level
Index points
Occupation rank
Index
points Index points of SES SES rank
I 5 I 5 5×4＋5×7＝55
II 4 II 4 4×4＋4×7＝44 High (41-55)
III 3 III 3 3×4＋3×7＝33 Mid (40-30)
IV 2 IV 2 2×4＋2×7＝22
V 1 V 1 1×4＋1×7＝11 Low (29-11)
The second section of the questionnaire aims to investigate different aspects of English learners’ motivation to learn English. It is divided into two parts. The first part examines students’ strength of motivation (referred to as “motivational intensity”
hereafter) and is designated with a total of 13 items . Items 2, 3, 4, 7, 8, 9, and 10 are indicative of strength. They were either translated or adapted from Gardner’s AMTB (1985b). Peng’s (2002) version of translation was taken as a reference. In addition to items directly assessing intensity, items enquiring persistence and attention were placed into this part as well. In Tremblay and Gardner’s (1995) questionnaire items on motivational intensity, persistence and attention along with intensity in itself were added up as an indicator of motivational intensity. In the present questionnaire, items 1, 6 and 11, selected from Pan’s (2002) questionnaire, measure learners’
attention, and items 5, 12, and 13, modified from Peng’s (2002) and Zeng’s (2002), assess learners’ persistence in learning English. The score of the thirteen items were summed up to indicate students’ motivational intensity.
The second part in section two inquires why students learn English. Several types of motives were put into investigation. Two major constructs applied are Gardner’s instrumental/integrative orientation construct (1985a) and Deci and Ryan’s self-determination theory (1985). Therefore, the two major instr uments adapted are the Attitude/ Motivation Test Battery (AMTB) by Gardner (1985b) and Language
Learning Orientations Scale— Intrinsic Motivation, Extrinsic Motivation, and Amotivation Subscales (LLOS-IEA) (Noels, Pelletier, Clément, & Vallerand, 2000).
Items developed from these two instruments explore students’ motives in learning English on the following dimensions:
1.Integrative Motive: Items 9 to 13 measure students’ Integrative Motive for learning English. Four items (item 9 to item 12) are translated from AMTB. Item 13 is a negative item regarding Integrative Motive, and should be reverse coded. The addition of item 13 was the result of the preliminary exploratory survey (see Chapter 3) on students’ reasons to learn English. That statement was found in a number of students’ answers. Thus, to reflect the result of the preliminary survey, this item was added. A high score (maximum=20) reflects strong Integrative Motive and a low one (minimum=5) reflects weak Integrative Motive.
2. Instrumental Motive: Items 14 to 18 examine students’ Instrumental Motive for learning English. These five items were modified from both Gardner’s AMTB (1985b) and Peng’s motivation questionnaire (2002). A high score (maximum=20) reflects strong Instrumental Motive while a low one (minimum=5) reflects weak Instrumental Motive.
3. Amotivation: Items 1 to 3 were translated and modified from LLOS-IEA, assessing whether students are amotivated to learn English. A high score (maximum=12) represents that students do not know why they are learning English and are amotivated; a low score (minimum=3) shows students clearly know about the reason why they are learning English.
4. Intrinsic Motive: Items 19 to 29 investigate students’ Intrinsic Motive, including Stimulation (items 19-21), Accomplishment (items 22-26), and Knowledge (items 27-29). Most items were translated and adapted from LLOS-IEA with two (items 22 and 26) taken from Peng’s study (2002) because these two are quite related
to Accomplishment. A high score in each (maximum value in Stimulation is 12, in Accomplishment 20, and in Knowledge 12) suggests the Intrinsic Motive plays a strong part in students’ learning English whereas a low score (minimum value in Stimulation is 3, in Accomplishment 5, and in Knowledge 3) means this motive is not a powerful factor that makes students learn English.
5. Extrinsic Motive: Items 30 to 44 inspect how students are extrinsically motivated to learn English. The fifteen items were mainly translated from LLOS-IEA, with some added and revised from either Peng’s study (2002) (e.g. item 42) or students’ responses in the preliminary survey (e.g. items 36 and 39). The first four items (items 30 to 33) measure the most self-determined form of Extrinsic Motive, namely, Identified Regulation. Items 34 to 40 assess Introjected Regulation and the last four (items 41 to 44) estimate the least self-determined form, External Regulation. A high score in each (maximum value in Identified Regulation is 16, in Introjected Regulation 28, and in External Regulation 16) means the Extrinsic Motive plays an important role in stimulating students’ English learning. On the contrary, a low score (minimum value in Identified Regulation is 4, in Introjected Regulation 7, and in External Regulation 4) says that this Extrinsic Motive is not a strong reason for the students’ learning of English.
There is an addition of Requirement Motive (items 4 to 8) besides Gardner’s (1985a) and Deci and Ryan’s (1985) constructs. These items were taken from Peng’s (2002) motivation instrument. A high score (maximum=20) indicates that requirement motivates students to learn English while a low one (minimum=5) implies that requirement is not an important factor that drives students to learn. The organization of the questionnaire and the distribution of all the items are shown in Table 2.
Table 2 Organization of the Questionnaire and Distribution of Items Section II Section I:
Background information Part I: Motivational
Intensity Part II: Motive Types
Content Item
number Content Item
number Content Item
number
Ÿ School and Grade level 1 Ÿ Amotivation 1-3
Ÿ Age 2 Ÿ Strength 2, 3, 4, 7,
8, 9, 10 Ÿ Requirement 4-8
Ÿ Gender 3 Ÿ Integrative 9-13
Ÿ Attention 1, 6, 11 Ÿ Instrumental 14-18 Ÿ Amount of exposure to
English 4-6
Ÿ Intrinsic 19-29
-School 4 Ÿ Persistence 5, 12, 13 -Stimulation 19-21
-Outside school 5 -Accomplishment 22-26
-Family 6 -Knowledge 27-29
Ÿ SES 7-10 Ÿ Extrinsic 30-44
-Identified 30-33
-Introjected 34-40 -External 41-44
Participants
Students from Grades 3 to 9 were selected via convenient cluster sampling to participate in this study. This study set third graders as the starting point of investigation because the Minister of Education announced that English education in Taiwan would be officially implemented from Grades 3 to 9 beginning from the 94th school year. Moreover, a great majority of schools have begun their English instruction since the third grade, and some even from the first grade (Liao, 2003;
Chen, 2002) despite the present stipulation that English should be taught since the fifth grade in elementary school according to Grades 1-9 Curriculum. In light of this common phenomenon, it was decided that this investigation began with third graders so that it will provide implications for the English education in Taiwan before the new policy is administrated.
Procedures
In the very beginning before which construct to be included in the instrument was decided, an informal preliminary exploratory survey was conducted in April, 2003, to provide a rough glance at the English learning motives of students nowadays.
The results of the preliminary survey were the basis of the questionnaire design.
After the questionnaire was completed, three experts in EFL field were invited to review the questionnaire in order to examine whether any item was invalid. It was then revised according to the experts’ feedback. Next, in order to ensure the readability of the questionnaire for younger students, it was given to three elementary school students (two third graders from Taipei Municipal Xin-He Elementary School and Jy-Shei Elementary School in Taipei County, respectively, and one fourth grader from Taipei Municipal Fu-Shing Elementary School). These three students were asked which items they did not understand after they finished answering the questionnaire. Modification was then made according to their responses. A pilot study was conducted in March, 2004, to test the reliability of all the items. The final formal study was carried out from May to June in 2004 after some further revisions of the questionnaire were made according to the pilot study.
Preliminary Exploratory Survey
A total of 353 students (39 sixth graders from Tainan Municipal An-Ping Elementary school; 314 seventh and eighth graders—174 from Tainan Municipal Ho-Shun Junior High and 140 from Taipei Municipal Chen-Yuan Junior High) were asked to write down at least three reasons for what they thought they learned English.
Their written responses were analyzed to find out Taiwanese students’ major English learning motives. The results were used to decide the motivation constructs to be applied in later questionnaire design.
Pilot Study
In the pilot stud y the questionnaire was administered to a total of 262 students selected via convenient cluster sampling in March, 2004 after expert validity and readability of the questionnaire were established. The 262 students were mainly from the beginning two levels of the two stages of formal education, third and fourth grades from the primary school and seventh and eighth grades from the junior high school. Students of the youngest age at the two stages were chosen to participate in the pilot study because it was assumed that students of an older age would have no problem understanding the questionnaire if those of a younger age could understand it.
Distribution of students participated in the pilot study is listed in Table 3.
Table 3 Distribution of Students in the Pilot Study G3 ~ G8— 1 class/per grade
Primary schools Junior high schools
North
Taipei County:
Lu-Chou Elementary School G3 = 35; G4 = 35
Taipei County :
Hsin-Tai Junior High School G7 = 36; G8 = 38
South
Tainan County:
Wen-Xian Elementary School G3= 27; G4= 24
Chia-yi County:
Mei-Shan Junior High School G7 = 36; G8 = 31
Out of feasibility considerations, the subjects were recruited from the classes of the researcher’s acquaintances who were in-service teachers. Students in the pilot study came from both northern and southern parts of Taiwan. Questionnaires were sent to teachers in the southern parts of Taiwan and the teachers were told the procedures of questionnaire administration. As to the administration in Taipei, the researcher went to the schools and conducted it herself. Students were encouraged to point out anything they did not understand while they were filling out the questionnaire. Their comments and questions were taken as a reference for
rephrasing questionnaire items before the formal study. The first item in Part Two, Section Two was rephrased with reference to students’ feedback. Three items in the first section enquiring students’ exposure to English at and outside school were also revised in reflection to the problems they posed when answering the items.
The reliability of each motivation dimension was calculated with SPSS 11.0 for Windows after the 262 questionnaires were retrieved. The Cronbach’s alpha is .88 for motivational intensity, and for motive types, it is .77 for Integrative Motive, .85 for Instrumental Motive, .86 for Requirement Motive, and .82 for Amotivation. On the part of Intrinsic Motive, the Cronbach’s alpha is .86 for Stimulation, .92 for Accomplishment, and .90 for Knowledge; on the part of Extrinsic Motive, it is .89 for Identified Regulation, .88 for Introjected Regulation, and .72 for External Regulation.
The alpha value of all the dimensions is considered acceptable so all the items were kept in the formal study.
Formal Study
After the pilot study from May to June, 2004, the finalized questionnaire was administered to a total of 1,292 students from Grade 3 to Grade 9. These students came from Taipei Municipal Fu-Lin Elementary School, Yong-Ji, and Shi-Lin Junior High Schools as well as from Tainan Municipal An-Ping Elementary School, and Chong-Ming Junior High School. Again, due to feasibility reasons, these students were invited to participate because their teachers were the researcher’s acquaintances.
The sampling method is thus convenient cluster sampling, as in the pilot study. The numbers of students from each grade and each school are listed in Table 4.
Table 4 Distribution of Students in the Formal Study
Primary schools Junior high schools Taipei City
North
Fu-Lin Elementary School G3 ( 3 classes)＝77 G4 ( 4 classes)＝89 G5 ( 3 classes)＝92 G6 ( 3 classes)＝88
(1) Yong-Ji junior high school (2) Shi-Lin Junior High School
( 2 classes/ per grade at per school) G7＝ 51＋56＝107
G8＝ 54＋64＝118 G9＝ 57＋53＝110 Tainan City
South
An-Ping Elementary School ( 2 classes/ per grade) G3＝70
G4＝71 G5＝75 G6＝69
Chong-Ming Junior High School ( 3 classes/ per grade)
G7＝98 G8＝119 G9＝109
The process of administration was the same as that in the pilot study. For students in Tainan, questionnaires were sent to their teachers. Those teachers were told the procedures before they helped the administration of the questionnaire. They also sent the answered questionnaires back to the researcher. The researcher herself went to the three schools in Taipei to administer the questionnaires. Participants were told to answer each item according to their real feelings and situations.
Reliability of items was calculated again in the formal study before performing further statistical analyses, such as t-test and ANOVA. Results of reliability analysis suggested that most of the motivation subscales in this questionnaire were reliable.
With all of the students as a group Cronbach’s alpha for the subscale of motivational intensity is .89. In fact, Cronbach’s alpha of each motivational dimension was over .80 except External Regulation. The Cronbach’s alpha of External Regulation was .78, quite close to .80. For the remaining motive types, their Cronbach’s alpha was as follows: Introjected Regulation .88, Identified Regulation .87, Stimulation .88,
Accomplishment .91, Knowledge .87, Instrumental, .83, Integrative .81, Requirement .90, and Amotivation, .83.
On the part of Integrative Motive, result of item analysis showed that Cronbach’s alpha would increase from .81 to .86 if item 13 was deleted. The deletion of item 13 could also raise the reliability of the Integrative Motive subscale separately calculated for third to ninth graders. The reliability for third graders raised from .75 to .85, fourth from .80 to .83, fifth from .79 to .81, sixth from .79 to .83, seventh from .82 to .85, eighth from .85 to .88, and ninth from .82 to .89. Therefore, item 13 was not included in later statistical analyses.
The values of Cronbach’s alpha on the three subscales of Intrinsic Motive were all approaching .90, with that on Accomplishment exceeding .90. Reliability analysis of the three subscales combined together gave the Cronbach’s alpha .95, representing the reliability of Intrinsic Motive. On the part of Extrinsic Motive, though one of its subcategories had a Cronbach’s alpha of .78 only (External Regulation), the reliability of the whole Extrinsic Motive group was still high, Cronbach’s alpha being .92. Cronbach’s alphas on Intrinsic ( .95) and Extrinsic Motives ( .92) were higher than those of Integrative ( .86 if item 13 deleted) and Instrumental Motives ( .83). It seems that to this group of students, the intrinsic/extrinsic motive subscale is more reliable than the integrative/instrumental one.
Data Analysis
Both descriptive and inferential statistics were performed to analyze data.
Descriptive statistics, such as means and standard deviations (SD), were computed to understand the background information of the participants and their performance on questionnaires. One-way ANOVA (analysis of variance) was conducted to answer
the first research question, exploring differences or changes from Grade 3 to Grade 9.
Whenever appropriate, t-test and one-way ANOVA were carried out to solve research questions two and three, examining the relationships between grades and motive types, and between motive types as well as motivational intensity and background variables, including gender, exposure to English, and SES. In addition, Pearson correlation analysis was conducted to explore the connection of different motive types as well as the association between motive types and motivational intensity.
