In this chapter, the research design procedures are presented first, and then participants, instruments, classroom observation, data collection and analyses are reported in sequence. Finally, the reliability and trustworthiness are provided.
Research Design Procedures
The participants in an intact class were the third-grade senior high school students who would attend the college entrance examination around the end of this semester.
After getting the English teacher’s consent (see Appendices A & B), the researcher collected the students’ simulative examination on reading scores at the beginning of the semester and their college entrance examination on English reading scores around the end of the semester. The first questionnaire, including 16 items under two achievement goal types and 21 items under three reading strategy types, was administered after the simulative examination in the beginning week of the semester. Besides, the second questionnaire, containing the same items as the first one, was administered on the following day after the college entrance examination. During the semester, the researcher observed their English lessons for five weeks, two times per week. Before the end of the semester, the researcher looked for six students to do the think-aloud task.
They were selected randomly from High-, Mid- and Low-achievers based on their reading scores of the simulative examination at the beginning of the semester. Each level had two participants. A 187-word narrative English passage was prepared for the six students to verbalize their thoughts while they were performing the task.
Simultaneously, the researcher observed their reading behavior, audio-recorded their think-aloud process, and then their reading strategies were analyzed based on their transcripts. Besides, the researcher also interviewed every student briefly about their
GEPT and English learning experience during the break time. The research design procedures are showed in Table 1.
Table 1
Research Design Procedures
Sequence Instrument Participant
The first-half semester: 46
At the beginning of the semester The simulative examination 45 At the beginning of the semester
During the semester During the semester During the semester
The first questionnaire of motivations and reading strategies
Classroom observations A short interview
The reading material of the think- aloud task, and a reading quiz
Whole class
Whole class Whole class
6
Around the end of the semester The college entrance examination 45 Around the end of the semester The second questionnaire of
motivations and reading strategies
43
Total available samples: 42
Participants
Forty-six students (M = 29, F = 17) at their third grade from an intact class of a private senior high school in southern Taiwan were invited to participate in the current study. English was a compulsory subject for all the participants who had to take six hours of English courses every week in this semester. Among them, one student planned to study abroad and did not attend the college entrance examination. Besides, three students didn’t reply the second questionnaire. Therefore, 42 participants (M=26, F=16) were left available. The investigation in the first questionnaire learned that 26 students (62 per cent) had ever had GEPT experience and 34 students (81 per cent) attended extracurricular English class apart from the normal school curriculum. Besides, 20 students (48 per cent) went to English cram school to enhance their English ability (see
Table 2). From the survey and further interviews, it also revealed that only one student had ever lived abroad. The girl had stayed with her parents for three months in
Minnesota, U.S.A. when she was in the kindergarten. Moreover, she had lived with her family for six months in Singapore when she was at the second grade in a primary school, whereas her reading proficiency was intermediate (M-achiever). Of the 26 students who had GEPT experience, only eight received the certificate of GEPT Elementary level and two passed the initial test of GEPT Intermediate level.
Table 2
Descriptive Statistics for 42 Subjects Gender Participant Attending English
Cram School
In the following section, the pre- and post-questionnaires, the reading comprehension tests, and the reading material will be explained.
Questionnaires
The pre- and post-questionnaires consist of three sections (see Appendices C & D).
The first part contains the students’ background information. The second part is comprised with 16 motivational items developed by He (2008) originating from Ames and Archer (1988). A permission to use the scale was sought and granted by professor He. Items A1 to A8 estimate the magnitude of Mastery goals while Items A9 to A16 estimate Performance goals. This 16 five-point Likert items range from Point 1 (very unlike me), Point 2 (unlike me), Point 3 (partly like me), Point 4 (be like me) to Point 5 (very like me). A higher point indicates a stronger tendency of mastery goals or stronger
developed by Huang (2014) originating from Mokhtari and Sheorey (2002). This 21 five-point Likert items range from Point 1 (never), Point 2 (seldom), Point 3
(sometimes), Point 4 (usually) to Point 5 (always). A higher point indicates more frequency use of the reading strategy. Besides, the 21 items of reading strategies are grouped into three types: Global strategy category is estimated by nine items (B1, B2, B3, B6, B9, B14, B18, B19, B20); Problem-solving strategy category is measured by seven items (B4, B5, B7, B11, B13, B15, B17) and Support strategy category is assessed by five items (B8, B10, B12, B16, B21).
Reading Comprehension Tests
The first simulative examination of English subject was performed on summer vacation when some students didn’t attend the extra supplementary course. The second simulative examination of English subject, held in the beginning week of the semester, was also regarded as the pre-reading comprehension test for the current study (see Appendix E). The examination consisted of 56 items of multiple choices adding up to the score of 72, two items of translation from Chinese to English occupying the score of 8, and an at least 120-word composition sharing the score of 20. The researcher adopted the part of 56-item multiple choices as the assessment of participant students’ reading comprehension. The 56 items contained vocabulary score of 15, comprehensive test score of 15, cloze test score of 10, reading test score of 32. Moreover, the 42 students’
reading comprehension ability was classified into High-, Mid- and Low-levels according to the 56-item reading scores.
The college entrance examination on English subject held around the end of this semester was regarded as the second reading comprehension test for this current study (see Appendix F). The structure of the examination was the same as the previous simulative examinations. In order to get the original score of 56 items of multiple
choices (total score was 72), the researcher checked the Grade Comparison Table on English subject and learned the width of interval (6.28) and the limits of each class interval. The midpoint was calculated by the bottom score in each class interval plus 3.14. Then, the individual score of non-multiple-choice items (total score 28) was subtracted from each midpoint score. Therefore, the original score on the 56-item reading comprehension was figured out individually.
The Reading Material
The 187-word narrative passage was chosen from GEPT-Intermediate level
simulation test in AMC English Digest in June, 2014 (see Appendix G). Because the six participants were respectively from High-, Mid- and Low-groups according to the first reading comprehension test, this prepared passage had a popular topic - an artist of silence, coherent contents, and moderate lexical items. Besides, the students were asked to do a quiz of four multiple-choice questions based on this passage after they
completed the reading task.
Classroom Observation
The main items of the classroom observations (10 times) during five weeks are listed in Table 3.
Table 3
Classroom Observations during English Reading Lessons The main items:
1. Classroom management: proactive approaches to make students self-regulated 2. Classroom organization: teacher-fronted activities with whole-class work 3. Classroom interaction: curricula basically focus on grammar translation
4. The use of resources: textbook, workbook, exam exercise paper, supplementary teaching materials, CD, TV
5. Classroom instruction: conventional instruction without systematic strategy training, vocabulary expansion using definitions, synonyms, antonyms, analysis of word parts
Data Collection and Analyses
According to Merriam and Tisdell (2016), data analyses are the process of constructing meaning out of the collected data including inductive and deductive reasoning. In the quantitative research, the data were collected from the participants’
response to the pre- and post-questionnaires, their scores on the pre- and post-reading comprehension tests, and then the statistical software SPSS 20 was applied to compute and analyze the data. In the qualitative research, the data were collected from the six participants’ think-aloud transcripts, their quiz scores on the reading material, and the researcher’s observation note during their reading task. Then, the researcher integrated the findings for patterns, differences, and unique happenings (Liang, 1999; Merriam &
Tisdell, 2016). The six participants’ background information is explicated in Table 4.
Table 4
Six Participants’ Background Information
Participant Gender Reading Ability GEPT experience James Male High Intermediate initial test, passed
Ann Female High Elementary level
Leo
Tim Male Low Elementary test, failed
Flora Female Low No
Reading Task: Think-aloud
First, the six students were selected randomly from three levels of proficiency of 42 participants, two members each level. Then, they were informed of the purpose of this study in advance. After getting their parents’ agreement, they participated in the think-aloud task. The reading task was held in a spare classroom, one at a time during their free time. At the beginning, the students scanned the passage, “Marcel Marceau”,
consulted the problematic words or phrases with a dictionary, and then wrote down their Chinese meanings. The students’ own cellphone with an on-line dictionary was allowed.
Moreover, the researcher audio-recorded the participants’ think-aloud translation of the passage by a cellphone. They were also asked to describe which reading strategies they adopted for each sentence during the reading task. Simultaneously, the researcher observed each student’s reading process and took notes. After completing the reading task, each participant was asked to do the quiz of four multiple-choice questions.
Reliability and Trustworthiness
To examine the reliability of achievement goals and reading strategy categories, the Cronbach’s alpha values were calculated by SPSS 20 based on both pre- and post-questionnaires. Besides, Pearson correlation coefficients were computed for paired pre- and post-reading scores as well as paired pre- and post-questionnaire scores. Then, correlations between achievement goals and reading strategy categories were computed.
On the other hand, in order to persuade the readers that the findings of a study are valid and reliable, a well-organized study with appropriate techniques and the rigorous criteria in qualitative research are necessary. Internal validity involves the match in research findings and reality. Reliability concerns whether the results are consistent with the data collected. Triangulation is the famous strategy to support the internal validity and ensure consistency. Besides, member-check is an important way to identify whether the researcher’s analyses are biased (Merriam & Tisdell, 2016). Therefore, the researcher employed triangulation for the six volunteers, inquired their GEPT
experience before the aloud task, observed their reading behavior in the think-aloud task process, checked their questionnaires, quizzed their reading comprehension about the reading material, and analyzed their think-aloud transcripts. In addition to triangulation, member-check was utilized by giving the six participants their translated Chinese transcripts for confirmation. Besides, ethical consideration such as the right to privacy and the notion of informed consent was also taken into account in the study.
CHAPTER FOUR