CHAPTER THREE METHODOLOGY
3.1 ParticipantsThe participants were 113 second-year junior high school students, ages between 13 and
14. They all have had at least over four years of formal English instruction in school. Most of
them—over 60 percent—have learned English for nearly six years. Originally there were 129
participants, but a few students’ data were excluded from data analysis because they either did
not take the GEPT pretest or posttest, or missed at least one class during this period of time.
The gender distribution was approximately equal. Of the 113 participants, 55 were male
and 58 were female. These participants were from four intact classes in a junior high school in
Taipei County. Two classes were scheduled to meet for the course in the morning, while the
other two met in the afternoon. One morning class and one afternoon class were chosen as the
experimental group, in which there were 30 males and 28 females, while the other two classes
served as the control group, made up of 25 males and 30 females. Prior to the experiment, all
participants were told that they would be having a specially designed one-month course
intended to help them strengthen their English listening ability, and that the results of this
experiment would be presented as a research project. Their consent to participate in the study
was asked for. All necessary approval of the school administration for data collection was
gained.
Selection of the experimental group was based on consideration about the learning
atmosphere in these four classes and the students’ learning abilities. Of the two morning classes,
one was quiet, but the other was loquacious and active. The former did not like to respond to
the teacher’s questions and was consisted of many passive learners; the latter was talkative,
prone to react quickly and its students were generally fast learners. In terms of English ability,
the latter class was more proficient than the former. In similar fashion, one of the two afternoon
classes was hostile toward strangers and would only speak in class when individually requested
by the teacher, while the other afternoon class was friendly and comprised mostly willing
learners. The former was better at English than the latter. In view of the distribution of different
learning atmospheres and learning abilities of these four classes, the two afternoon classes
were selected as the experimental group.
According to the background questionnaire results (see 3.3 Instruments), most of the
participants, as much as 64 percent, normally did not spend any time practicing English
listening outside of the classroom. For those who claimed to take the time sharpening their
listening skill, the majority—22 percent—did so for an average of less than one hour per week.
The most common ways in which the participants tried to develop their listening included
listening to English songs, watching English TV programs, watching English movies, and
taking English classes outside school. But most of the participants, over 40 percent, professed
that they never tried to bolster their listening skill in any possible way.
Among participants in the control group, 12 once took the official GEPT Elementary test
(see 3.3 Instruments), contrasted with only four in the experimental group. Of all participants,
approximately ten percent have purchased commercially available GEPT practice tests, and
more than 18 percent have taken other types of English proficiency tests. Overall, the majority
of all participants had no experience with any kind of English proficiency tests whatsoever.
When asked to give ratings to their English listening ability and the degree to which they
would like to improve it on a five-point scale, participants in the experimental and control
groups gave similar responses. They generally regarded their listening ability as normal (about
42 percent) or not good (about 34 percent), and indicated that they could do without
improvements (around 38 percent) or expressed a desire to make improvements (around 34
percent). The participants’ responses also indicated that none of them was conversant with
strategy learning.
3.2 Experimental Design
The design of this study is quasi-experimental. There are experimental and control groups
which are discriminated by the treatment they experience. But subjects in this study were not
randomly put in one of these groups.
O1 X O2
O3 C O4
X symbolizes the experimental variable manipulated by the researcher.
C denotes the control variable manipulated by the researcher.
Figure 1. Pretest-Posttest Nonequivalent-Groups Design
As mentioned above, two classes in a junior high school were allocated to the
experimental group, and another two classes were chosen as the control group. Individual
participants from each of these four classes stayed in their original class they formed from the
very beginning instead of being randomly assigned to the experimental and control groups.
Therefore, the two groups were not comparable to start with.
Both the experimental and control groups had to take two pretests and posttests—a
listening comprehension test and a listening strategies questionnaire. The independent variable
in the present study was teaching method, with only the experimental group receiving strategy
training. The dependent variables were listening performance and listening strategy use.
Given that listening proficiency is strongly correlated with listening test scores, the
students’ listening proficiency was considered a covariate in this study, or an extraneous
variable, which is not under investigation but the effect of which might impinge on the
dependent variable(s), in this case listening performances, and affect interval validity. The O1, O3 = pretest scores O2, O4 = posttest scores
impact brought forth by the irrelevant, or extraneous variable was removed by way of
statistical control (see 3.5 Data Analysis below). Along with proficiency, learning motivation
was treated as a covariate, too. As it has been generally accepted that motivation has a direct
bearing on learning consequences, the effect of a strategy instruction course can only be
accurately measured when the influence of motivation is excluded. Hence the impact of
motivation justifiably needed to be minimized. To determine whether the learners made
improvements in how they utilized strategies, their initial strategy use was also used as a
covariate.
3.3 Instruments
There are eight instruments in total, used in different phases of the research. Some were
administered either before or after the experiment. Others were employed both before and after
the research—the pretests and posttest. Still others were used together during the experiment.
3.3.1 Background Questionnaire
One week prior to the experiment, a background questionnaire (see Appendix A) and a
Motivational Intensity Questionnaire (MIQ) (Gardner, 1985) (see Appendix B) were administered to all participants. The background questionnaire required the participants to
provide information regarding their English learning such as the amount of time they spent on
learning English outside of class per week, the ways in which they went about practicing their
English listening ability, the experience they had with any English proficiency tests. Some of
these questions were meant to generate the participants’ awareness of language learning and
get them to self-reflect their ways of learning and self-evaluate their ability.
3.3.2 Motivational Intensity Questionnaire
The Motivational Intensity Questionnaire was used to measure the participants’ learning
motivation. It was one of the subtests in the Attitude/Motivation Test Battery (AMTB)
developed by Gardner (1985), which was originally directed toward the investigation of
English-speaking students learning French. The MIQ was made up of ten multiple-choice items
rated on a three-point scale. In the past, it has been used in considerable research concerning
SLA motivation. Many studies have modified and validated the AMTB to study the learning of
English by ESL/ EFL students to assess the major affective components involved in it. Based
on some previous studies with Taiwanese EFL learners, the MIQ was perceived as a reliable
measure of motivation. Hsiao (1997) used the MIQ and obtained a coefficient of internal
consistency at .75. Also, Liao (2000) used a Chinese version of the MIQ and produced the
internal-consistency reliability of .78 in her pilot study. The Chinese version of the MIQ
devised by Liao was adopted in the present study.
3.3.3 Listening Comprehension Strategy Questionnaire
The Listening Comprehension Strategy Questionnaire (LCSQ) (adapted from Cheng,
2002) (see Appendix C) was given only to participants in the experimental group to plumb
their strategy use before and after the strategy instruction course. This questionnaire inquired
about the participants’ knowledge of the range and frequency of their listening strategy use. It
was composed of 33 close-ended items and one open-ended question. Table 1 is a list of all the
strategies that the close-ended questions in LCSQ are meant to tap into (Chien, 2005; Lin,
2006).
Table 1. Strategies That Underlie the Close-Ended Questions in the LCSQ Categories of Strategies Items Individual Strategies
Items 1-2: Advance organization Item 3: Directed attention Item 4: Selective attention Items 1-6:
Planning strategies
Item 5-6: Self-management Item 7: Keeping up with the speed Item 8: Comprehension monitoring Item 9: Double-check monitoring Items 7-10:
Monitoring strategies
Item 10: Refocusing
Item 11: Problem-identification Item 12: Performance evaluation Metacognitive strategies
Items 11-13:
Self-evaluation
Strategies Item 13: Strategy evaluation
Item 14 Translation
Item 15 Resourcing
Item 16 Repetition
Item 17: Recombination
Item 18: Listening for key words Item 19: Repetition
Items 17-20:
Bottom-up strategies
Item 20: Segmentation Item 21: Listening for
main ideas Item 22: Prediction Item 23: Inferencing
Item 24: Thinking in English Item 25: Imagery
Items 21-26:
Top-down strategies
Item 26: Elaboration
Item 27 Note-taking
Item 28 Grouping
Cognitive strategies
Item 29 Summarization
Item 30: Questioning for clarification Items 30-31:
Social strategies Item 31: Cooperation Item 32: Lowering anxiety Socio-affective strategies
Items 32-33:
Affective strategies Item 33: Self-encouragement
The 33 close-form questions of LCSQ were statements which the participants had to
respond on a five-point Likert scale. Questions 1-13 asked about learners’ use of metacognitive
strategies, and questions 14-29 focused on the use of cognitive strategies. The last three
close-ended questions addressed social-affective strategies, and the only open-ended question
requested learners to specify any strategies not mentioned in the previous questions.
The LCSQ performed duel functions by checking the participants’ use of strategies for the
teacher-researcher to orient teaching practices and serving as a prelude to strategy training.
Devised by Cheng (2002), the questionnaire was intended for evaluating Taiwanese junior high
school students’ use of English listening strategies. Its validity and reliability were already
confirmed in several studies (Chien, 2005; Lin, 2006). For use in the present study, item 18
was rephrased to address the strategy of listening for key words.
3.3.4 General English Proficiency Test
Two General English Proficiency Test (GEPT) Elementary Level listening practice tests
published by the Language Training & Testing Center (LTTC) in Taiwan were utilized as part
of the pretests and posttests. GEPT was a criterion-referenced English proficiency test
developed by LTTC and was sponsored and recognized by the MOE. The GEPT test was
designed with the purpose of promoting English learning in Taiwan; its validity and reliability
have been widely accepted in many educational institutions and corporate businesses. There
were five levels in all—Elementary, Intermediate, High-Intermediate, Advanced, and Superior.
As the GEPT Elementary Level test covered topics of interest and relevance to Taiwanese
junior high school students (appearance, food and drinks, places, numbers, occupations,
recreation, school life, social life, traffic, and climate) and passing (80 out of 120) required
candidates to be able to understand daily conversational language—an English proficiency
level equivalent to that possessed by a junior high school graduate, it was considered
appropriate for use in the present study. The GEPT had four subtests assessing all four skills of
listening, speaking, reading, and writing.
A few sample questions from a GEPT Elementary Level listening practice test are shown
in Appendix D. The GEPT Elementary Level listening test was constituted by three types of
tasks—Picture Identification, Making Responses, and Understanding Dialogues, each
consisting of ten items and takes 20 minutes in total. The strategy instruction implemented in
the present study—teaching six targeted strategies—was intended to get them to be aware of
and to practice the skills which would help them do well on the test.
The first type of task in the GEPT Elementary Level Listening Test—Picture
Identification—entailed the three strategies of advance organization, listening for the general
gist, and listening for key words. The second task, Making Responses, required the use of two
listening strategiesprediction and inferencing. The third task, Understanding Dialogues,
called for the use of almost all six strategies—all but advance organization.
3.3.5 Teaching Materials
The materials used for teaching the six strategies mainly came from two commercially
available books, Tactics for Listening (2nd edition) Book 1 (Richards, 2003) and American
Channel Elementary Level (Mitchell & Scott, 2004). Of the three-level Tactics for Listening
series, Book 1, the first level, was selected for use as it is directed at elementary proficiency
level English learners. It featured both top-down and bottom-up processing, included texts of
different types, and contained exercises practicing advance organization, listening for main
ideas, selective attention, listening for details, and several inference-related skills like listening
for attitudes, listening for topics, listening for opinions, and so forth.
American Channel used an integrated approach to the four skills of listening, speaking,
reading, and writing, encompassing topics which could stimulate teenage and young adult
learners’ interest. Its listening activities boasted a wide variety of text types and task formats
and emphasized the strategies of listening for gist and listening for specific information.
Among the four books in the American Channel series (Beginners, Elementary, Intermediate,
and Pre-Intermediate), Elementary was chosen for use. Selection of the materials in the present
study followed Rubin and Thompson’s (1992) advice that the teaching materials should be
moderately challenging and not overly difficult to learners.
The On Channel TV DVD (Mitchell, 2005) was used as the sole treatment given to
learners in the control group. Besides, part of it was also incorporated in the strategy training
course offered to the experimental group. On Channel TV was a three-level DVD series
designed for teenage and young adult learners of English, which could be used to supplement a
course at the same level or as an independent multimedia based course. The DVD featured two
characters working for a television channel in different situations; it comprised short television
programs of different types such as quiz shows, street surveys, interviews, and documentaries.
Of the three levels ranging from Beginners, Elementary to Pre-intermediate, Beginners and
Elementary were adopted in the present study since they were comparable to the GEPT
Elementary level test. Each DVD was composed of 16 five-minute episodes and so was around
80 minutes in length.
3.3.6 Teaching Plan
A brief teaching plan specifying the synopsis of the one-month course was formulated in
advance (see Appendix E). In the teaching plan, the objectives to be achieved in each class
session were spelled out. The equipments, teaching materials, and teaching procedures were
listed, and so were the tasks to be performed by the learners. Table 2 presents the syllabus of
the explicit listening strategy instruction course schedule.
Table 2. Syllabus of the Explicit Listening Strategy Instruction Course Class
session
Targeted strategies Tasks Sources
1 Advance organization &
listening for the general Gist
Preconceiving words which might be heard in the upcoming listening task; identifying pictures with state- ments; placing objects in the correct location
Richards (2003)
2 Listening for key words
& selective attention
Labeling/noting specific detailed information; pointing out mistakes in a picture; deciding in advance what to listen for in the upcoming listening passage; deciding on key words to listen for by studying a picture or comprehension questions beforehand
Richards (2003);
Mitchell & Scott (2004); Mitchell (2005)
3 Prediction Taking guesses at the lines of a character based on a previous sentence said by another character in spoken conversations
Richards(2003);
Mitchell (2005)
4 Inferencing Infer about rain gear/locations/
attitudes/objects from available information in listening passages
Richards (2003);
Mitchell & Scott (2004)
In the first two weeks, each one class was devoted to training of two strategies. The
combinations were formed in accordance with the suggestions by some researchers (Chamot et
al., 1999). They declared that listening for the general gist is usually linked with advance
organization, and listening for key words can be associated with selective attention, hence the
design of this study (see 3.3 Instruments). In the following two weeks, each session was
centered on only one strategy. As noted above, the six strategies taught were sequenced in
Depending on the difficulty of and the time needed for different activities, each class session
was consisted of a reasonable number of activities the participants could handle profitably.
Following suggestions from some researchers (O’Malley & Chamot, 1990, p.6; Rubin,
1994), both metacognitive and cognitive strategies were included in the strategy instruction
course in the present study. Also, advice from Brown (2001, p.260) was also taken—which
stated that both bottom-up and top-down techniques should be included when teaching
listening. The focus of the instruction was six English listening strategies—advance
organization, listening for the general gist, listening for key words, selective attention,
prediction, and inferencing (Brown, 2001, p.259; O’Malley & Chamot, 1990, pp.198-199).
Among the six target strategies, two are metacognitive strategies—advance organization and
selective attention, and the rest are all cognitive strategies.
The definitions of the strategies were as follows. Based on O’Malley and Chamot (1990,
p.136), advance organization denotes previewing the main ideas and concept of the material to
be learned, often by skimming the text for the organization. Listening for the gist equals in
meaning to Oxford’s (1990, p.80) skimming, which represents searching for, in this case
listening for, the main ideas the speaker intends to get across.
Selective attention (Chamot et al., 1999, pp.21-22; O’Malley & Chamot, 1990, p.198)
signifies choosing in advance to focus on certain key words, phrases, and ideas, or situational
details that will possibly induce comprehension when later performing a task. Listening for key
words, in Brown’s term (2001, p.259), is equivalent to Oxford’s (1990, p.80) scanning, which
means to listen for specific details of interest and relevance to the listener. The two strategies
could go hand in hand. An example would be listening to a weather forecast. Before one gets
ready to listen to a train schedule, he or she might choose to find specific information such as
the evening temperature in a certain place—that is when selective attention is employed. Later,
as that person puts the decision into action, listening only for the evening temperature in that
given place, the strategy of listening for key words is practiced.
Listening for the general gist and listening for key words are comparable to skimming and
scanning in Oxford’s strategy classification system. The two are combined together as “getting
the idea quickly” by O’Malley and Chamot (1990, p.80). Based on Oxford’s (1990, p.80)
advice, charts, lists, and diagrams could be used to accompany the learning of the two
strategies in the present study as they offer clues to learners about what general points or
specific details to listen for.
The fifth strategy taught was prediction. According to Chamot et al. (1999, pp.15, 19),
predicting involves anticipating information that one can expect to encounter in the task.
Oxford (1990, p.92) maintained that in order to make correct predictions, “knowing what has
already been said” is necessary. Only by understanding what has been said can “important
information for anticipating what will later be said” be yielded. That is why teaching of this
strategy was postponed until “listening for the general gist” and “listening for key words” were
taught in the present study. An example of making predictions would be listening to a
conversation between two people, in which one asks the other for a favor; by listening to that
person’s request, we make predictions about the reply of the other person.
Inferencing, the final strategy learned by participants in the experimental group, was the
most difficult to master. It requires making educated, logical guesses (about the meaning of
what is heard) by drawing on contextual clues and previous linguistic as well as worldly
knowledge (O’Malley & Chamot, 1990, p.199). Almost all guessing, or compensation
strategies involve inferencing to some extent. According to the taxonomy of listening
comprehension strategies developed by Vandergrift (2003), inferencing could be divided into
linguistic inferencing, voice inferencing, extralinguistic inferencing, and between-parts
inferencing. Oxford (1990, pp.92-93) also asserted that the bedrock of correct inferencing lies
in all possible clues—including linguistic, nonverbal, perceptual (audible, visual),
textual/structural, situational/contextual, and worldly. As inferencing requires compensating for
incomplete information on the basis of available input, it needs to draw on the help of other
simpler strategies.
The present study adopted one of the approaches recommended by Brown (2001, p.217) to
teach strategies, namely integrating strategy use into classroom techniques. All the exercises
and activities, adapted from Tactics for Listening, American Channel and ON Channel TV), in
each session were prepared around the listening strategies highlighted and practiced that day.
For example, the tasks of matching pictures with statements and placing objects in the correct
location were meant to drill in listening for the general gist, while the tasks which required
identifying statements with speakers or labeling specific detailed information were aimed for
practicing listening for key words. Most tasks were modeled after the three types of tasks in the
GEPT Elementary Level listening test.
The explicit strategy instruction classes followed a sequence of instructional steps
suggested by some researchers (See Literature Review) (O’Malley & Chamot, 1990,
pp.201-204; Oxford, 1990, pp.203-208). They were integrated and adjusted to the context in
the present study: (1) Diagnosis: Ascertain students’ prior knowledge of the topics to be taught
and their level of language proficiency. Identify students’ current learning strategies through
whatever methods available. (2) Instruction: Explain the concept and importance/effectiveness
of learning strategies. Use a variety of techniques such as demonstrations and modeling to
make instruction accessible and comprehensible to students. Develop goals for strategy use for
students. (3) Practice: Get students to use the skills or strategies in activities that involve
collaboration or problem-solving. Provide different practice opportunities with varied learning
tasks. (4) Evaluation: Help students assess their own strategy use and gauge their progress.
Evaluate the whole strategy training and revise the teaching content.
By following the above-mentioned steps, the teaching procedures of the four strategy
instruction sessions were rendered identical. At the very beginning, having the participants
complete a background questionnaire, a Motivational Intensity Questionnaire, and a Listening
Comprehension Strategy Questionnaire and two pretests corresponded to the first
step—Diagnosis. Throughout the training course, the components of “Instruction”, “Practice”,
and “Evaluation” coexisted in each class, which was consonant with Oxford’s (1990, p.203)
claim that instructional steps do not necessarily have to be in a fixed order, but could be
executed at the same time.
Teaching of the strategies naturally involved the step of “Instruction,” in which the
teacher-researcher labeled and introduced the strategy/strategies and explained how it/they
worked to get students acquainted with the target strategy/strategies. After presenting the target
strategy/strategies, the teacher-researcher modeled it/them on a listening task. What followed
was “Practice”—the learners fulfilled a whole host of on-the-spot listening tasks varied in
nature via the target strategy/strategies. Whenever possible, the teacher-researcher made sure
that good performances, or correct answers to the listening tasks as well as detailed and
insightful responses were praised. At the end of each class, the participants’ act of filling out a
reflection questionnaire was considered part of “Evaluation”, which culminated in selective
individual interviews and posttests after the instruction course. The learners’ self-evaluation
carried out through the LCSQ posttest, reflection questionnaires, and interviews helped the
teacher-researcher to assess the effect of the strategy instruction, the lesson planning, the use of
materials, and the teaching techniques (Chamot et al., 1999, p.115).
3.3.7 Reflection Questionnaire
At the end of each strategy training session, students in the experimental group were
asked to fill out a reflection questionnaire (see Appendix F). As application of learning
strategies is not directly observable, reflection questionnaires in the present study served as a
means to tap into the students’ mental processes of using the given strategies. In response to the
questionnaire, the participants had to (1) describe how they made use of the strategy/strategies
taught that day to help them understand what they heard and (2) express whether they enjoyed
each class or not and why. Both the questions were open-ended.
The use of reflection questionnaires in the present study is identical to employing a
learning log. A learning log is an individual student’s summary of what has been learned over a
given period of time (Chamot et al., 1999, p.123). The items can be either close-form or
open-ended. According to Chamot et al. (1999, p.123), examples of learning logs can range
from students’ copying brief summaries dictated or written by the teacher to students’ noting
what they learned that day without any directions from their teacher. In accordance with
Oxford’s advice (1990, p.167), the experimental group participants in the present study were
asked to periodically complete one reflection questionnaire—10 minutes before each class
ended.
3.3.8 Interview Guide
An interview guide (see Appendix G) was developed for conducting interviews with some
participants in the experimental group after the strategy instruction course ended. The
interview guide was constituted by eight questions that were asked during the interview and
was meant to be semi-structured. The purpose of holding interviews was to obtain a fuller
understanding of how the participants liked the strategy training course, whether they found it
useful, which of the strategies they regarded as the most and least helpful, whether they
encountered any difficulty when receiving the instruction, what suggestions they would offer to
improve the strategy training course, and whether they thought their self-confidence and
interest in learning English were boosted or not. According to Chen (2007), learners’
self-evaluation of their improvement in proficiency is essential to determining if they are
satisfied with the strategy training carried out, though it is not always valid due to lack of
statistical verification. The rationale for using interviews as the main research tool to obtain the
experimental group’s perceptions of the strategy training course can be justified by Borg and
Gall’s (1983, p.436) statement that interviews could dig more deeply to provide a true picture
of interviewees’ opinions and feelings. Borg and Gall listed the advantages that interviews have
over questionnaires—immediate feedback, follow-up leading to more data, and greater clarity
through careful motivation and maintenance of rapport with interviewees.
3.4 Data Collection Procedure
The data collection procedure lasted for six weeks in duration. Among them, four weeks
were spent on the strategy training course. Table 3 displays the listening strategy training
course schedule.
Table 3. Schedule of the Research
Week Research activities
1 Administer a background questionnaire, a Motivational Intensity Questionnaire and two pretests (GEPT and a Listening
Comprehension Strategy Questionnaire) 2 Teach the two strategies of advance organization &
listening for the general gist; give a reflection questionnaire 3 Teach the two strategies of listening for key words &
selective attention; give a reflection questionnaire 4 Teach the strategy of prediction; give a reflection questionnaire 5 Teach the strategy of inferencing; give a reflection questionnaire 6 Administer two posttests (GEPT and a Listening Comprehension
Strategy Questionnaire) and conduct interviews
One week before the explicit strategy instruction lessons started, all participants
completed a background questionnaire, a Motivational Intensity Questionnaire, and a Listening
Comprehension Strategy Questionnaire. After that, they took a GEPT Elementary Level
listening practice test. In the following month, the experimental group was provided with
weekly classes in which overt instruction was conducted on the six strategies of advance
organization, listening for the general gist, listening for key words, selective attention,
integrated into the students’ regular English curriculum. Rather, it was delivered in the weekly
counseling class, presented as a special course in English listening strategies.
At the beginning of each class, the strategy/strategies taught that day was/were introduced
and defined in order to raise the learners’ awareness of it/them. Also, the strategies presented in
the previous class sessions were discussed again, with their names and functions mentioned.
The few number of strategies taught and the simplicity of them gave sufficient reason why the
duration of this strategy training course was designed to last only for one month. In total, there
were only six strategies taught and it was assumed beforehand that learners would have no
difficulty internalizing them. As many researchers have indicated, learners should not be
overloaded with a vast array of strategies at a time.
After each class session, every learner in the experimental group filled out a reflection
questionnaire. Overall, they would be completing four questionnaires. The reflection
questionnaire was drawn up by the teacher-researcher and was made up of only two questions.
After each training session, the teacher-researcher read through the students’ responses to
obtain information concerning their learning and to use them as a basis for improving the next
session.
On the other hand, the control group did not receive explicit strategy instruction. Instead,
they simply watched the ON Channel TV DVD (Beginners and Elementary levels) for one solid
month over the same time period. Viewing of the 32 five-minute episodes in the two DVDs
was spread out in the four weekly classes. Correspondingly, participants in the control group
were not required to complete a reflection questionnaire.
One month later, all participants took another GEPT Elementary Level listening practice
test to ascertain if they made any progress in their listening performance. Afterwards, they
filled out a Listening Comprehension Strategy Questionnaire. Data collected from the
experimental and control groups were compared.
When the strategy instruction course came to an end, interviews were conducted with
some participants in the experimental group. In each of the two classes which comprised the
experimental group, three participants from each of the three proficiency levels—high,
intermediate, and low—were randomly selected—high, intermediate, and low. The proficiency
level was determined on the basis of the participants’ GEPT pretest and posttest scores. Those
who scored higher than 80 fell into the category of high proficiency level, and those who got a
score lower than 40 were considered low proficiency learners. The intermediate proficiency
level group was consisted of learners who scored between 50 and 70. Of all interviewees, 11
were male, and 7 were female. Of all males, four were highly proficient, three at the
intermediate level, and four in the group of low proficiency. As the interviews were
semi-structured, additional questions were posed according to the way the interviews
progressed—based on the participants’ responses.
The first step of the interview was to summon the 18 interviewees to a counseling room
one by one for individual meetings with the researcher. When conducting the interviews, the
prepared guide was employed. The entire interview process was audio-recorded, the main
pointes taken note of, and the participants were told that their responses would be anonymous
and reported only in group form. As predetermined categories were already derived before the
interviews, the researcher jotted down responses in relation to each given category and noted
unusual answers and additional questions asked in the interviews.
3.5 Data Analysis
3.5.1 GEPT Pretest and Posttest
To compare the differences between the GEPT listening pretest and posttest scores of the
experimental and control groups, the data were entered into SPSS (Statistical Packages for the
Social Sciences). Since the present study was a quasi-experiment—individual participants were
from already existing classes rather than being reorganized and assigned to different groups
through randomization—the data were computed using an analysis of covariance (ANCOVA)
to control for initial differences between groups. As it was not known beforehand if there had
been a pre-existing difference between the two groups’ listening proficiency levels as measured
by the GEPT pretest, ANCOVA fulfilled the function of minimizing possible interference that
might have stemmed from any covariate, in this case GEPT pretest scores and motivational
intensity, which was assessed by the MIQ. The initial group differences were reduced by
making compensating adjustments to the posttest means of the two groups. The .05
significance level was used as the criterion level for determining significant differences.
3.5.2 LCSQ Pretest and Posttest
There were four steps involved to analyze the LCSQ pretest and posttest results. First,
ANCOVA was computed to determine if there was a difference between listening strategy uses
reported by the experimental and control groups. Next, the paired samples t-test was calculated
to compare the differences between the experimental group’s two sets of responses before and
after the experiment to the 33 close-ended questions concerning their strategy use. In a similar
way, the paired samples t-test was performed on the data of the control group. Third, the paired
samples t-test was computed on the experimental group’s two sets of answers to the seven
questions probing the learners’ use of the six targeted strategies. Then, the paired samples t-test
was performed again, on the control group’s responses to the same seven questions before and
after the experimental period. Finally, all participants’ responses to the only one open-ended
question were described, and categories were established when general patterns were
identified.
3.5.3 Reflection Questionnaires
Concerning the four reflection questionnaires answered by all participants in the
experimental group, only the second question, which inquired them about whether they were
fond of the class that day as well as the reasons for their answers, was included for data
analysis. For one thing, the students’ ratings of each class session were divided into three types,
positive, neutral, and negative. Then, their reasons were recorded under each of the three
groups, which in turn were classed into several categories.
3.5.4 Interviews
The procedure for analyzing the selective individual interviews was as follows. First of all,
the interviews were transcribed. The next step was to create categories—turning the
predetermined questions into categories, so to speak. During the process, general patterns were
looked for, and responses in every given category were classified together. Examples of the
learners’ views and perceptions were recorded under each category, yielding a definitive group
of patterns and reactions. Categories under the same topic are grouped together, forming
separate dimensions. Frequencies of the responses to each category were then counted. The
major findings were described with illustrations from the interviews given, offering a better
insight into the students’ learning process. Eventually, interpretations of all these data were
coupled with information from the participants’ background questionnaire to provide plausible
explanations.
