The chapter describes the proposed model that characterizes the role of identity-based writing motivation and self-regulatory writing strategies in explaining L2 graduate students’ RA-abstract writing ability with L2 literacy serving as a covariate.
The chapter also details research procedure, participants, measures, and the structural equation modeling approach.
The Proposed Model
The present study aimed to investigate the role of identity-based writing motivation and the self-regulatory writing strategies in explaining L2 graduate students’
RA-abstract writing ability with L2 literacy serving as a covariate. Figure 1 shows the proposed model in the present study. By testing this proposed model, the aim was to achieve a better understanding of the interplay between cognitive factors (e.g., L2 literacy, use of self-regulatory writing strategies) and affective factor (i.e., identity-based writing motivation), in explaining L2 graduate students’ RA-abstract writing ability.
74
Figure 1. The motivated abstract-writing model (the MAW model). The model presents 4 latent variables and 12 observed variables (X1=reading ability, X2= writing ability;
X3=interest value, X4=utility value, X5=cost, X6=connectedness value, X7=ability self-concept; Y1=cognition regulation, Y2=motivation regulation, Y3=
socio-interactive resources regulation; Y4= global move, Y5= local pattern.).
Research Procedure
There are four phases involved in the research procedure of the present study, including phases of preparation, pilot studies, formal study, and evaluation. The four-phased research procedure is shown in Table 1.
75
Specify research purposes.
Propose a model of motivated abstract-writing (the MAW model).
Determine data collection methods.
Conceptualize the motivational constructs on the basis of identity-based motivation theory, expectancy-value theory, socio-educational theory, and genre theory.
Consult Experts and Compile Measures.
Consult experts in the fields of educational psychology and L2 learning to ensure the motivational constructs could be assessed by item measures.
Draft demographics questionnaire, Identity-based Writing Motivation Inventory (IWMI), and Self-regulatory Writing Strategy Inventory (SWSI) in English and translate the questionnaire items into Chinese.
Consult experts in the field of L2 research-article writing to ensure the genre theory could be applied to development of the two rating scales of RA-abstract writing.
Draft Research-Abstract Performance Assessment by two parts, First, draft a prompt in Chinese (participants’ L1) for a timed RA-abstract writing task on the basis of an empirical research article in applied linguistics; it was written in Chinese. Second, draft two rating scales for the RA-abstract writing task,
including a global move scale and a local pattern scale.
Compile an L2 literacy test by 20 multiple-choice items for reading comprehension and a timed essay test for writing ability. These test items were taken from the retired items in the advanced level of General English Proficiency Test (the
GEPT-advanced).
2.
Pilot study
& Measures development
Examine the Psychometric Properties of IWMI and SWSI.
Interview a focus group of 2 doctoral students from the fields of education and business to ensure the appropriateness of item wording and range in IWMI and SWSI.
Back-translate the inventories from Chinese to English to make the wording clear and precise.
76
Consult 2 experts in the fields of educational psychology and L2 learning (i.e., holding a Ph.D. in the respective field) to ensure the clarity and appropriateness of the two inventories.
Revise IWMI and SWSI for a subsequent pilot test.
Administer the two inventories to 255 graduate students in the fields of education and business in the pilot study.
Analyze the data collected from the pilot study by item analysis, internal consistency, and confirmatory factor analysis (CFA).
Establish preliminary validity for the two inventories.
Establish Feasibility of Research-Abstract Performance Assessment.
Pilot the RA-Abstract Writing Test to 5 graduate students in applied linguistics.
Interview the 5 graduate students to ensure that the prompt was clearly worded, and the test was accomplishable within the time limit.
Develop the two rating scales of the writing task via two stages:
In advisor-guiding stage, the two advisors guided the researcher through drafting of the two rating scales.
In expert-reviewing stage, 3 professors in applied linguistics reviewed the rating scales via semi-structured interviews and ensuing emails. The three experts were invited in sequence;
i.e., one at a time to approach the following questions :
Whether and how can the rating scales assess L2 graduate students’ ability to write an RA-abstract?
Should the scores be weighted?
How should the wording and criteria of the scale descriptions be presented?
3.
Formal study
Conduct the study.
Recruit 185 graduate students (both master’s and doctoral students) in applied linguistics.
Administer the measures in sequence, i.e., the RA-abstract writing task, the L2 literacy test, and finally the questionnaire set (i.e., demographics questionnaire, IWMI, and SWSI).
Specifically, administer the RA-abstract writing task and the L2 literacy test in a counter-balanced manner. Half of the participants took the RA-abstract writing task and then the L2 literacy test, whereas the other took the L2 literacy test and then the RA-abstract writing task.
77
4.
Evaluation
Analyze data.
Analyze background questionnaire, IWMI and SWSI.
Analyze the L2 literacy test: reading and writing tasks.
Specifically, for the writing task:
Recruit two raters.
Hold a rater training session.
The two raters rate independently.
Evaluate the inter-rater reliability.
Analyze the RA-abstract writing task.
Recruit two raters.
Hold a rater-training session.
The two raters rate independently.
Evaluate the inter-rater reliability.
Evaluate the proposed model via the structural equation modeling approach
Analyze correlations among the 12 observed variables.
Analyze correlations among the 4 latent variables.
By Anderson and Gerbing’s (1988) 2-step rule, test the measurement model and the structural model.
Further test the measurement model on its construct validity.
(1) The measurement model (confirmatory factor analysis (CFA).
Report the hypothesized model and the modified model by goodness-of-fit indices.
Report correlations between the latent variables.
(2)Construct validity testing.
Report factor loadings and error variances.
Calculate composite reliability for convergent validity.
Calculate the Average Variance Extracted (AVE) estimate for convergent validity.
Compare the AVE with the squared correlations of the four latent variables for discriminant validity.
(3) The structural model.
Report goodness-of-fit indices.
78
Report factor loadings and error variances.
Report regression coefficients.
In the preparation phase, important studies were reviewed and evaluated to propose a model of motivated abstract-writing, and to determine the data-collection methods. Then, a number of measures were drafted, including the demographics questionnaire, Identity-based Writing Motivation Inventory (IWMI), Self-regulatory Writing Strategy Inventory (SWSI), and Research-Abstract Performance Assessment (RAPA) that subsumes a prompt of RA-abstract writing task and two corresponding rating scales. Also, an English literacy test was taken from the retired items of General English Proficiency Test-advanced (GEPT), including 20 multiple-choice items for reading ability, and a timed essay test for writing ability.
In the phase of pilot studies, the measures were further revised, and the criteria designed for the rating scales of the RA-abstract writing were established. The measure revision was based on the content validity of IWMI and SWSI (see Appendix A for the demographics questionnaire, IWMI, and SWSI). In particular, the content validity of the two inventories was initially established by a few steps, including an interview of a focus group (IWMI and SWSI), backtranslation, and expert-consultation.
Also in the phase of pilot study, IWMI and SWSI were administered to 255 graduate students in the fields of education and business. Data in the pilot study was analyzed by item analysis, internal consistency, and confirmatory factor analysis.
Likewise, the RA-abstract writing task was administered to 5 graduate students in the field of applied linguistics. After the task, a series of follow-up interviews were conducted with the 5 graduate students to ensure that the writing task was clearly worded and accomplishable within the time limit. The criteria designed for the rating scales of the writing task were developed by two stages (the advisor-guiding and the expert-reviewing stages). Details regarding how to develop the rating scales will be
79
further discussed in the section entitled Measures.
In the phase of the formal study, 185 graduate students in applied linguistics (i.e., both master’s and doctoral students) were recruited. Before the study began, the graduate students had signed up the consent form (see Appendix B) to demonstrate their agreement to participate in the study. The participants took the measures in sequence, including the RA-abstract writing task (60 minutes), then the L2 literacy test (55 minutes), and finally a questionnaire set (15 minutes) that includes demographics information, Identity-based Writing Motivation Inventory (IWMI), and Self-regulatory Writing Strategies Inventory (SWSI). The RA-abstract writing task and the L2 literacy test were administered in a counterbalanced design. That is, half of the participants took the RA-abstract writing task, and then the L2 literacy test. The other took the L2 literacy test, and then the RA-abstract writing task.
Finally, in the evaluation phase, questionnaires, the L2 literacy test, and the RA-abstract writing task were analyzed, and the structural equation modeling (SEM) approach was applied to data analysis. First, all questionnaires were coded and computed to reveal participants’ profiles of demographics, identity-based writing motivation, and use of self-regulatory writing strategies. Second, the L2 litearcy test (i.e., reading and writing tasks) and the RA-abstract writing task were scored to show participants’ performances in this ability domain. Specifically, the writing task of the L2 literacy test was rated by a few steps: recruiting two raters, holding a rater-training session, rating, evaluating the inter-rater reliability (see more details in Rating of the GEPT-advanced writing). Likewise, the RA-abstract writing task was rated by the similar steps: recruiting two raters, holding a rater-training session, rating, evaluating the inter-rater reliability (see more details in Rating procedure of the RA-abstracts).
With the data of the questionnaires, the L2 literacy test, and the RA-abstract writing task, the SEM analysis was applied. In SEM analysis, the present study
80
examined the compatibility of the hypothesized model with the collected data.
Specifically, the present study followed Anderson and Gerbing’s (1988) two-step rule of measurement model and structual model, and further tested the measurement model by its construct validity. First, the measurement model was examined by goodness-of-fit indices (i.e., primary information on how well the proposed 4 latent variables could be represented by 12 observed variables) and by the correlations among the four latent variables. Furthermore, the measurement model was tested by its construct validity (i.e., convergent and divergent validity). The structural model was examined by goodness-of-fit indices and regression coefficients. Specifically, the regression coefficients were used to test the hypothesized interrelations among the 4 latent variables.
The four-phased research procedure presents an overview of the research steps taken in the present study. Details on the participants, measures (i.e., the L2 literacy test, questionnaires, RAPA, and the SEM approach will be described below.
Participants
Two studies were conducted, the pilot study and the formal study, involving two groups of participants.
The Pilot Study
The pilot study was conducted to finalize the identity-based writing motivation inventory (IWMI) and the self-regulatory writing strategy inventory (SWSI). The pilot study recruited 255 Taiwanese doctoral students in the fields of education and business.
There were two reasons to involve graduate students from a variety of academic fields:
concerns of practicality. The concerns of practicality stemmed from the limited number
81
of graduate students in the field of applied linguistics. It was less likely to find two groups of a large number of graduate students in applied linguistics (i.e., 100 to 200 students to participate in the pilot and formal studies respectively). In order to ensure to get enough graduate students in applied linguistics to participate in the formal study, graduate students from different academic fields were recruited for the pilot study.
Specifically, the participants in the pilot study were recruited from the fields of business and education, which fall into the category of social sciences, like the field of applied linguisitics. In particular, the participants in the pilot study were doctoral students in the fields of education and business. Doctoral students were chosen because they had more experiences in writing abstracts of research articles including abstracts of thesis, abstracts of journal articles, and abstracts of conference proceedings. Moreover, the two questionnaires were drafted not only by locating questionnaire items in related literature but also by interviewing two doctoral students in the fields of education and business respectively to reveal their motivation and strategies for RA-abstract writing.
In the pilot study, the participants majoring in education were recruited mainly from National Taiwan Normal University (NTNU, n=121, 47.5% of the 255 participants in total). There were two reasons for choosing this university. First, the university specializes in training pre-service teachers, and most graduate programs at NTNU were education-oriented, making most doctoral students at NTNU education majors. Second, NTNU was the researcher’ own university. The researcher had more access to participant recruitment, with the help either from individual doctoral students or from faculty members at the university. Additional doctoral students majoring in education at other universities (such as National Cheng Kung University, Taipei Municipal University of Education, n=19, 7.5% of the 255 participants in the pilot study) were recruited as well. The doctoral education majors totaled 140, accounting for 55 % of the 255 participants in the pilot study.
82
The participants majoring in business were recruited from National Taiwan University (n=75, 29.4% of the 255 participants in the pilot study) and National Cheng-Chi University (n=40, 15.6% of the 255 participants in the pilot study). These two universities were chosen because they were on the top-three rank of the business field in Taiwan, doctoral students of which were assumed to have more experiences in writing English RA abstracts, and to be more aware of their motivation and self-regulatory strategies for writing English RA-abstracts. The doctoral business majors totaled 115, accounting for 45 % of the 255 participants in the pilot study.
Table 2 presents demographics of the participants in terms of year in their doctoral programs and the number of thesis abstracts written. First, the participants were categorized as education majors or business majors, with each category accounting for 100 percent in the following data in terms of years in the doctoral programs and the number of thesis abstract written (Note that a few doctoral students held more than one MA degrees, and had written more than one thesis abstracts). For the year in program, missing data were found from 5 education majors (3.6%) and 7 business majors (6.1%).
Apart from the missing data, overall the doctoral students at years 1 and 2 accounted for around 50 percent of the participants (77 education majors = 55%, and 54 business majors = 46.9%), while the doctoral students ranging from years 3 to 7 accounted for the rest of the participants (58 education majors = 41.5% and 54 business majors = 46.9%).
For the number of thesis abstracts written, missing data were found from 34 education majors (24.3%) and 30 business majors (26.1%). Apart from the missing data, a majority of the participants reported that they had written 1 to 3 thesis abstracts in English (89 education majors = 63.6%, and 82 business majors = 71.3%). A few participants reported that they had written no thesis abstracts in English (14 education majors = 10%, and 3 business majors = 2.6%). A small number of participants reported
83
that they had written 4 to 6 thesis abstracts in English (3 education majors = 2.1%, and 0 business majors = 0.0%). As the majority of the participants reported that they had experiences in writing thesis abstracts in English, it was more meaningful and valid to ask them to report their motivation for English RA-writing and use of self-regulatory strategies for writing English RA-abstracts.
Table 2
Demographics of Doctoral Students of Education Majors and Business Majors in the Pilot Study.
Education majors Business majors
Year n % n %
1 45 32.1 23 20.0
2 32 22.9 31 27.0
3 20 14.3 20 17.4
4 21 15.0 12 10.4
5 9 6.4 11 9.6
6 4 2.9 6 5.2
7 4 2.9 5 4.3
Sum 135 96.4 108 93.9
Missing 5 3.6 7 6.1
Total 140 100.0 115 100.0
Thesis Abstracts written
N % n %
0 14 10.0 3 2.6
1~3 89 63.6 82 71.3
4~6 3 2.1 0 0.0
Sum 106 75.7 85 73.9
Missing 34 24.3 30 26.1
Total 140 100.0 115 100.0
The Formal Study
In the formal study, participants were recruited from graduate students who
84
enrolled in the universities in Taiwan in the fall semester 2011. Over two months (November, 2011 to January, 2012), the participants were recruited as a sample representative of the target population. The target population for the study was Taiwanese graduate students who majored in applied linguistics with a basic
knowledge of language acquisition (i.e., one of the required courses in their programs was language acquisition). The graduate students were under investigation because (a) they generally had better English literacy skills as prerequisite for research-article writing, than graduate students with other majors, (b) they had a need to learn writing of English research-articles in both master’s and doctoral programs so as to present their studies to their disciplinary community, and (c) they were more likely to be aware of their learning process of RA-abstract writing (e.g., writing motivation and strategies) due to their knowledge of language acquisition. Taken together, these characteristics of the L2 graduate students make it possible to collect meaningful data for testing the motivated abstract-writing model (the MAW model) that includes identity-based writing motivation and use of self-regulatory writing strategies in explaining RA-abstract writing ability with a covariate of L2 literacy.
However, this focus on the applied-linguistics majors in graduate programs made recruitment difficult to implement given the small number of the target graduate
students in Taiwan. For example, National Taiwan Normal University (NTNU) is one of the largest programs of applied linguistics in Taiwan. In the academic year of 2010 at NTNU, there were 128 students in the master’s program of applied linguistics and 40 students in the doctoral program. However, not all of these 168 graduate students were willing to participate in the study for two reasons. First, the study took around two hours to complete three tasks that were cognitively demanding. Second, a few MA students might have limited experiences in writing RA-abstracts (e.g., first-year MA students) and they were less confident in participating in the writing tasks. Therefore,
85
recruiting participants was of great difficulty in the formal study, making it less likely to recruit a large-sized sample.
When recruiting participants, I targeted both master’s and doctoral students in the programs of applied linguistics across 15 universities in Taiwan. These 15 universities were public and private universities in Taiwan, including (1) National Taiwan Normal University (NTNU), (2) National Cheng-Chi University (NCCU), (3) National
Tsing-Hua University (NTHU), (4) National Chiao-Tung University (NCTU), (5) Taipei Municipal University of Education (TMUE), (6) National Ocean University (NOU), (7) National Chuang-Hua University of Education (NCHUE), (8) National Kaohsiung Normal University (NKNU), (9) National Taiwan University (NTU), (10), Tamkang University (TU), (11) Chung-Yuan Christian University (CYCU) , (12) Kainan University (KNU), (13) Ming-chuan University (MCU), (14) Hsuan Chuang University (HCU), and (15) Tung Hai University (THU). Apart from these 15 universities targeted, a few other universities also have post-graduate programs of applied linguistics, such as National Taiwan University of Science and Technology (NTUST), National Cheng Kung University (NCKU), and Yuan-Ze University (YZU), National Taipei University of Technology (NTUT), and National Taipei University of Education (NTUE). These universities were not included because no volunteers from those universities could be reached over a period of two months when the survey proceeded. Although volunteers might become reachable in those universities at other time points, it was less likely to prolong the duration of the survey (e.g., from two months to three or four months). Such prolonging would result in the survey spanning two or more semesters (rather than one semester in the present study), making the participants’ performance relatively incomparable. Thus, I merely targeted the 15 universities stated above to recruit master’s and doctoral students over a period of two months.
86
Despite inclusion of 15 universities, the number of the participants (N=185) seems quite small in SEM research. With this small sample size, it is less likely to examine a complex model with various latent variables and indicators. An indicator requires a minimum of 10 participants in SEM analysis (Hair et al., 2010). This is why the study proposed a rather simple and parsimonious model that only includes 12 indicators when modeling the motivated abstract-writing.
The participants were recruited on class and individual basis. On a class basis, a few professors were contacted in the field across universities, and some of them
The participants were recruited on class and individual basis. On a class basis, a few professors were contacted in the field across universities, and some of them