國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
35
CHAPTER THREE METHODOLOGY
In the previous chapter, the definitions and theories of second language learning motivation have been discussed. The motivational components have also been
introduced. Research findings have shown that there is some relationship among them but it is not that clear. Therefore, the present study was designed to explore their relationship. The methodology of the study will be presented in this chapter, which includes five parts: the structure and hypothesis of this study, subjects, instrument, procedure, and data analysis.
The Structure and Hypothesis of This Study
The present study intended to investigate Taiwanese non-English major’s English learning motivation and to propose an empirically grounded construct of language learning motivation by using structural equation modeling. More specifically, this study was designed to answer the following question:
Specifically, this study aimed to develop a model that can be supported by empirical data, and effectively explain the relationship among all the components of Taiwanese EFL learners’ motivation.
Five latent variables were taken into consideration in the structural model as below:
1. Integrativeness: Integrative Orientation (ITO), Attitude toward English Speaking Countries (AEC), and Interest in Foreign Languages (IFL).
2. Instrumentality: Instrumental Orientation (ISO).
3. Intrinsic Orientation: Intrinsic Orientation (IO).
4. Extrinsic Orientation:
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
36
Identified Regulation (IDR): performing an activity out of personal choice for self-related reasons.
Introjected Regulation (INR): performing an activity because of externally imposed rules or pressure that individual has incorporated into the self and accepted as norms.
External Regulation (ER). performing activities determined by means external to the person, such as tangible benefits or punishments.
5. Motivational Behaviors: Motivational Intensity (MI), Attitude toward Learning English (ALE), and Desire to Learn English (DLE).
Among these variables, Integrativeness, Instrumentality, Intrinsic Orientation and Extrinsic Orientation are exogenous variables; Motivational Behaviors are
endogenous variables. The hypothesized causal relationship in the proposed model (see Figure 7) based on previous literature (Chen& Sheu, 2005; Csizer & Dornyei, 2005; Gardner, Tremblay & Masgoret, 1997; MacIntyre et al., 2001; Tremblay and Gardner, 1995; Yashima, Zenuk-Nishide & Shimizu, 2004; Noels, 2003) can be described as follows:
1. Integrativeness, instrumentality, intrinsic orientation, and extrinsic orientation all directly influence motivational behaviors.
2. Integrativeness, instrumentality, intrinsic orientation, and extrinsic orientation are mutually correlated.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
37
Figure 7. The Proposed Model
Participants
The study adopted convenience sampling to select participants. In the pilot study, the participants were 90 non-language major freshman students in National Taipei University of Education. They were 20 males and 50 females, aged 18-23, having at least 6 years of English learning experience. The pilot study was conducted in February, 2011. The questionnaire used in the pilot study was sent to the participants through e-mails, and 90 participants finished the questionnaire. After excluding incomplete or blank responses, 70 valid questionnaires were retrieved.
In the formal study, the participants were 302 non-language (i.e. English or foreign language) major freshmen from 8 universities in Taipei city, National
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
38
Cheng-Chi University (n=50), Taipei Medical University(n=45), National Taiwan University of Science and Technology(n=38), National Yang-Ming University(n=30), Shih Hsin University(n=45), Shih Chien University(n=29), Ming Chuan
University(n=31), and Chinese Culture University(n=34). These universities were considered to represent the population in that both national and private university, comprehensive university and university with a single orientation (e.g. the medical university), and university of technology are all included. The sample includes 140 male students and 162 female students. They majored in humanities (n=42), social science (n=121), and science and technology (n=139). Most of them had at least 6 years of English learning experience.
It could be concluded from literature that the results might be most stable if the sample is between 200 to 500 in SEM analysis (Wu, 2010). Considering the return rate of the questionnaire, the number of questionnaire distributed was increased to 450, and the link to the on-line version of the questionnaire was sent to all the freshman students in National Cheng-Chi University to increase returning number of the questionnaire.
The formal study was conducted in November, 2011. After incomplete questionnaires were removed, 302 valid questionnaires were retrieved.
‧
A questionnaire written in Chinese was used to collect information on participants’
motivation for English learning (See Appendix B). The items in the questionnaire were adapted from Dörnyei (1990), Gardner (1985), Gardner, Tremblay & Masgoret (1997), Noels, Clément & Pelletier (1999), Peng (2000), and Wen’s (1997)
motivation/attitude scales. Some items were revised to make them appropriate to the context and participants in the current study. The questionnaire consisted of three parts (the structure of the questionnaire is shown in Table 1). The participants were asked to respond to a 4-point scale with choice ranging from strongly disagree, disagree, agree and strongly agree.
Table 1
The Structure of the Questionnaire in Pilot Study Part Content
Part 1 Background Information (6 questions)
Age, gender, English learning experience, school, department, and year Part2 Motivational Behaviors (24 questions)
Motivational Intensity (8 questions)
Attitude toward Learning English (6 questions) Desire to Learn English (6 questions)
Part3 Integrativeness (15 questions) Integrative Orientation (3 questions)
Attitude toward English Speaking Countries (5 questions) Interest in Foreign Languages (7 questions)
Instrumentality (5 questions) Instrumental Orientation
Intrinsic Orientation (6 questions) Intrinsic Orientation
Extrinsic Orientation (19 questions) External Regulation (13 questions) Introjected Regulation (3 questions) Identified Regulation (3 questions)
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
40
Before the pilot study, the questionnaire was given to three experts specializing in English language teaching for the validity, in terms of its logic, wording, and structure, and then it was revised accordingly. After retrieving the questionnaire in the pilot study, item analysis with SPSS for Windows 12.0 was conducted to select items for the formal study. The steps of item analysis were as follows.
First, independent t-test was performed between higher-scored group and lower-scored group. Items lower then the significance level (p>.05) and their critical ratio lower then 3.00 were considered to be deleted or revised (Wu, 2011). Then, the correlation coefficients between the total score of each scale and the score of each item were calculated. The items lower than the significance level (p>.05) and with correlation coefficients lower than 0.3 were considered to be deleted or revised.
Finally, the Cronbach’s α was calculated to examine the internal consistency. Some items were considered to be deleted if the α value of the scale it belongs to improves when it was deleted. After item analysis, the Structural Equation Modeling program Amos 18 was applied to conduct confirmatory factor analysis to analyze the
measurement model of each scale. The questionnaire was revised according to the statistical result. The questionnaire for the formal study was shown in Appendix C.
The results of item analysis and confirmatory factor analysis of each scale were presented as follows.
Motivational Behaviors Item analysis
The result of item analysis in the Motivational Behaviors scale was shown in Table 2.
‧
Item Analysis of Motivational Behaviors Scale (α=.908)
Dimension Item
Critical
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
42
Item 2 was revised and kept because the critical ratio was lower than 3.00 but deleting it would decrease the α-value of the scale. Item 4 and 5 were deleted because deleting them improved the α-value of the scale, and the critical ratio of item 5 was lower than 3.00. After formal study, the Motivational Behaviors scale was further analyzed using the samples of 302 participants to test the construct validity of the measurement model.
Confirmatory Factor Analysis
The Structural Equation Modeling program Amos 18 was applied to conduct confirmatory factor analysis to analyze the measurement model of motivational behaviors. Eighteen items measuring three components/dimensions of motivational behaviors was analyzed. The chi-square value was 595.646, with 132 degrees of freedom. RMSEA value was .096, suggesting the measurement model of motivational behaviors need to be modified.
Modification indices report and Par change was used to remove items or loosening constraints on certain model parameters in order to improve the overall model fit. The parameters were modified one at a time, and the model was
re-estimated right after modifying each parameter. Parameter modification was based on high “modification indices” and “par change” value (Wu, 2010). Item 4, 11, 12, 15, 16, and 18 was removed and item 13 and 14 were loosened constraint because of high modification index. The result that item 13 and 14 are correlated may be because both items measure participants’ willingness to communicate and to practice speaking skill.
The modified measurement model has 89.894 chi-square value and 50 degrees of freedom. RMSEA value was.046; NFI value was.963; CFI value was .983; PNFI value was.729; GFI value was .964; AGFI value was.944; PGFI value was.618, suggesting the modified measurement model was a good fit to the data. The items left
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
43
in Motivation Intensity (MI) scale/dimension were item 1, 2, 3, 5, and 6. The items left in Attitudes toward Learning English (ALE) were item7, 8, 9, 10. The items left in Desire To Learn English (DLE) were item13, 14, and 17. The modified measurement model of Motivational Behaviors was shown as Figure 8.The goodness of fit indices was shown in Table 3. The items in the modified scale, the factor loadings and squared multiple correlation of each item, the average variance extracted and the composite reliability of the scale were shown in Table 4. The composite reliability of the three dimensions was higher than .60, and the average variance extracted was higher than .60, indicating the modified measurement model had achieved good reliability and validity.
To develop the model, the latent variable Motivational Behaviors was designed to be measured by three observed variables, Motivation Intensity (which was the average score of item 1, 2, 3, 5, and 6), Attitudes Toward Learning English(which was the average score of item 7, 8, 9, and 10), and Desire To Learn English (which was the average score of item 13,14, and 17).
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
44
Standardized estimates Default model χ2=89.894(P= .000); df=50
AGFI= .944; GFI= .964 RMSEA=.046; CN= 287
Figure 8. The Measurement Model of Motivational Behaviors Table 3
The Goodness of Fit Indices of the Measurement Model of Motivational Behaviors
χ2 p df RMSEA GFI AGFI
Modified model
89.894 .000 50 .046 .964 .944
NFI RFI IFI TLI CFI PGFI PNFI PCFI CN χ2 /df Modified
model
.963 .951 .983 .977 .983 .618 .729 .745 287 1.798
‧
The Factor Loadings, Squared Multiple Correlations, Average Variance Extractedand Composite Reliability of Motivational Behaviors Scale
Dimension Item
Factor
Note. AVE=Average Variance Extracted; CR=Composite Reliability; SMC=Squared
Integrativeness Item Analysis
The result of item analysis in the Integrativeness Scale was shown in Table 5.
‧
Item Analysis of Integrativeness Scale (α=.910)
Dimension Item Critical
Ratio
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
47
Item 28 was revised and kept because deleting them improved the α-value of the scale, but its critical ratio was higher than 3.00. After formal study, the Integrativeness scale was further analyzed using the sample of 302 participants to test the construct validity of the measurement model.
Confirmatory Factor Analysis
The Structural Equation Modeling program Amos 18 was applied to conduct confirmatory factor analysis to analyze the measurement model of motivational behaviors. 15 items measuring three components/dimensions of Integrativeness was analyzed. The chi-square value was 351.642, with 87 degrees of freedom. RMSEA value was .089, suggesting the measurement model of motivational behaviors need to be modified.
Modification indices report and Par change was used to remove items or loosening constraints on certain model parameters in order to improve the overall model fit. The parameters were modified one at a time, and the model was
re-estimated right after modifying each parameter. Parameter modification was based on high “modification indices” and “par change” value (Wu, 2010). Item 24 and 26 were removed and item 47 and 48 were loosened constraint because of high
modification index, item 52 and 53 were loosened as well. The result that item 47 and 48 are correlated may be because both items measured participants’ evaluation about the importance of foreign languages. The correlation between item 52 and 53 may be because both of the items measured participants’ positive feelings and experiences about foreign languages. The modified measurement model has 164.705 chi-square value and 60 degrees of freedom. RMSEA value was.068, NFI value was.931, CFI value was .954, PNFI value was.716, GFI value was .941, AGFI value was.911, PGFI value was.620, suggesting the modified measurement model was a good fit to the data.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
48
The items left in Integrative Orientation (ITO) scale/dimension were item 19, 20, and 21. The items left in Attitude toward English Speaking Countries (AEC) were item 22, 23, 25. The items left in Interest in Foreign Languages (IFL) were item 47, 48, 49, 50, 51, 52, and 53. The modified measurement model of Integrativeness was shown as Figure 9. The goodness of fit indices was shown in Table 6. The items in the modified scale, the factor loading of each item, the average variance extracted and the
composite reliability of the scale were shown in Table 7.
To develop the model, the latent variable Integrativeness was designed to be
measured by three observed variables, Integrative Orientation (which was the average score of item 19, 20, and 21), Attitude toward English Speaking Countries (which was the average score of item 22, 23, and 25), and Interest in Foreign Languages (which was the average score of item 47, 48, 49, 50, 51, 52, and 53).
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
49
Standardized estimates Default model χ2=164.705(P= .000); df=60
AGFI= .911; GFI= .941; RMSEA=.068; CN= 184 Figure 9. The Measurement Model of Integrativeness
Table 6
The Goodness of Fit Indices of the Measurement Model of Integrativeness
χ2 p df RMSEA GFI AGFI
Modified model
164.705 .000 60 .068 .941 .911
NFI RFI IFI TLI CFI PGFI PNFI PCFI CN χ2 /df Modified
model
.931 .910 .955 .941 .954 .620 .716 .734 185 2.745
Table 7
The Factor Loadings, Squared Multiple Correlations, Average Variance Extractedand Composite Reliability of Integrativeness Scale
‧
Dimension Item
Factor
Note. AVE=Average Variance Extracted; CR=Composite Reliability; SMC=Squared Multiple Correlations
Instrumentality Item Analysis
The result of item analysis in the Instrumentality Scale was shown in Table 8.
‧
Item Analysis of Instrumentality Scale (α=.821)
Dimension
Item Critical
Ratio
deleted Result
Instrumental
Item 32 was revised and kept because deleting them improved the α-value of the scale, but its critical ratio was higher than 3.00. After formal study, the
Instrumentality scale was further analyzed using the sample of 302 participants to test the construct validity of the measurement model.
Confirmatory Factor Analysis
The Structural Equation Modeling program Amos 18 was applied to conduct confirmatory factor analysis to analyze the measurement model of motivational behaviors. Five items measuring three components/dimensions of motivational
behaviors was analyzed. The chi-square value was 23.004, with 5 degrees of freedom.
RMSEA value was .097, suggesting the measurement model of motivational behaviors need to be modified.
Modification indices report and Par change was used to remove items or loosening constraints on certain model parameters in order to improve the overall model fit. The parameters were modified one at a time, and the model was
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
52
re-estimated right after modifying each parameter. Parameter modification was based on high modification indices and par change value (Wu, 2010). Item 30 was removed and item 28 and 29 were loosened constraint because of high modification index.
Although item 28 and 29 both dealt with the usefulness of learning English, item 28 described becoming more knowledgeable and item 29 was about having more chances of getting a better job. The negative correlation between the two items may be
because knowledge acquiring tends to be a long-term goal while job acquiring was not. The modified measurement model has 0.555 chi-square value and 1 degree of freedom. RMSEA value was .000, NFI value was.999, CFI value was 1.000, PNFI value was.167, GFI value was .999, AGFI value was.993, PGFI value was.100, suggesting the goodness of fit of the modified measurement was acceptable. The items left in Instrumental Orientation (ISO) scale/dimension were item 27, 28, 29, and 31. The modified measurement model of Instrumentality was shown as Figure10.The goodness of fit indices was shown in Table 9. The items in the modified scale, the factor loading and squared multiple correlation of each item, the average variance extracted and the composite reliability of the scale were shown in Table 10.
To develop the model, the latent variable Instrumentality was designed to be measured by the observed variable, Instrumental Orientation (which was the average score of item 27, 28, 29, and 31).
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
53
Standardized estimates Default model χ2=.555(P= .456); df=1 AGFI= .993; GFI= .999 RMSEA=.000; CN= 2644
Figure10. The Measurement Model of Instrumentality Table 9
The Goodness of Fit Indices of the Measurement Model of Instrumentality
χ2 p df RMSEA GFI AGFI
Modified model
0.555 .456 1 .000 .999 .993
NFI RFI IFI TLI CFI PGFI PNFI PCFI CN χ2 /df Modified
model
.999 .994 1.001 1.004 1.000 .100 .167 .167 2644 0.555
‧
The Factor Loading, Squared Multiple Correlations, Average Variance Extractedand Composite Reliability of Instrumentality Scale
Dimension Item
Factor
Note. AVE=Average Variance Extracted; CR=Composite Reliability; SMC=Squared Multiple Correlations
Intrinsic Orientation Item Analysis
The result of item analysis in the Intrinsic Orientation Scale was shown in Table 11.
Table 11
Item Analysis of Intrinsic Orientation Scale (α=.846)
Dimension Item Critical
Ratio
Item-total correlation
α after being
deleted Result
Intrinsic
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
55
The items in this scale were all kept because the critical ration, and item-total correlation reached the criterion, and theα-value decreased after deleting the items.
After formal study, the Intrinsic Orientation scale was further analyzed using the sample of 302 participants to test the construct validity of the measurement model.
Confirmatory Factor Analysis
The Structural Equation Modeling program Amos 18 was applied to conduct confirmatory factor analysis to analyze the measurement model of Intrinsic Orientation. Six items measuring three components/dimensions of motivational behaviors was analyzed. The chi-square value was 71.423, with 9 degrees of freedom.
RMSEA value was .135, suggesting the measurement model of motivational behaviors need to be modified.
Modification indices report and Par change was used to remove items or loosening constraints on certain model parameters in order to improve the overall model fit. The parameters were modified one at a time, and the model was
re-estimated right after modifying each parameter. Parameter modification was based on high “modification indices” and “par change” value (Wu, 2010). Item 33 was removed, and item 36 and 37 were loosened constraint because of high modification index. Item 36 was about satisfaction in suppressing the self and item 37 was about sense of achievement in conquering difficult practices. The correlation between these two items seemed reasonable because when an individual suppresses himself/herself, he/she usually has a sense of achievement. The modified measurement model has 1.087 chi-square value and 4 degrees of freedom. RMSEA value was .000, NFI value was.999, CFI value was 1.000, PNFI value was.399, GFI value was .999, AGFI value was.996, PGFI value was.266, suggesting the modified measurement model was a good fit to the data. The items left in Intrinsic Orientation (IO) scale/dimension were
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
56
item 32, 34, 35, 36and 37. The modified measurement model of Instrumentality was shown as Figure11. The goodness of fit indices was shown in Table 12. The items in the modified scale, the factor loading and squared multiple correlation of each item, the average variance extracted and the composite reliability of the scale were shown in Table 13.
To develop the model, the latent variable Intrinsic Orientation was designed to be measured by the observed variable Intrinsic Orientation (which was the average score of item 32, 34, 35, 36, and 37).
Standardized estimates Default model χ2=1.087(P= .896); df=4
AGFI= .996; GFI= .999 RMSEA=.000; CN= 3334
Figure 11. The Measurement Model of Intrinsic Orientation
‧
The Goodness of Fit Indices of the Measurement Model of Intrinsic Orientation
χ2 p df RMSEA GFI AGFI
The Factor Loading, Squared Multiple Correlations, Average Variance Extractedand Composite Reliability of Intrinsic Orientation Scale
Dimension Item
Factor
Note. AVE=Average Variance Extracted; CR=Composite Reliability; SMC=Squared Multiple Correlations
Extrinsic Orientation Item Analysis
The result of item analysis in the Extrinsic Orientation Scale was shown in Table 14 below.
‧
Item Analysis of Extrinsic Orientation Scale (α=.820)
Dimension Item Critical
Dimension Item Critical