Chapter 3 Methodology
3.5 Qualitative Analysis and Scoring and Data Analysis
The qualitative analysis in this study included the classroom observation reports
collected from different co-teaching programs throughout Taiwan, which would help to
provide a more in-depth understanding of interactions between the participants and the
NESTs which might help explain the relationship between the listening ability of the
participants and the NESTs. Data were read and re-read so as to get a better
understanding of the content, to discover or label variables and to find the relationship
among variables (Glaser & Strauss, 1967).
37
As for the quantitative analysis of the result of the communicative listening test,
statistical analysis was performed using the Statistical Packages for Social Sciences
(SPSS) software, in which descriptive statistics, t-test and one-way analyses of variance
(ANOVA) were used.
(a) Descriptive statistics: This study adopted mean scores and
standard deviation analysis to investigate participants’ English
listening test performances with NEST instruction of one-year
and three-years.
(b) T-test: T-test was used to compare the standard deviation of
two different groups. In this study, it was used to compare (1)
participants’ listening performance with NEST instruction of
one-year and three-years; (2) listening performances in each
part of the listening test with NEST instruction of one-year and
three-years. With T-test, standard deviation and test statistics
would be taken into consideration so that we could see whether
the difference reaches the significance level.
(c) One-way ANOVA: It was used to compare the standard
deviation of more than two groups of participants. In this study,
one-way ANOVA was adopted to compare participants’
38
English listening performances across different parts of
Taiwan with NEST instruction of one-year and three-years.
39
Chapter 4
Results and Discussion
As mentioned earlier, the present study aims at investigating younger learners’
(age 12-14) listening proficiency with the authentic input of NESTs in a co-teaching
model in a listening test developed based on the communicative testing approach. The
findings and the analysis of the quantitative results and the supports from the qualitative
reports regarding the influence of the NESTs are presented in the following sections.
4.1 Influence of the Assistance from NESTs on Students’ Listening
Comprehension Skills
Question 1: Do students who are assisted by NESTs show a better improvement in
their listening comprehension skills than those who are not assisted by NESTs?
The descriptive statistic of less-than-a-year NESTs’ assisting group and
more-than-three-year NESTs’ assisting group is shown in Table 6. Generally speaking,
participants who received NESTs’ assistance for a longer period of time performed
better than those who received shorter ETA instruction throughout the six areas. The
mean score of the participants who received less than a year of NESTs assistance
(mean=61.13) was strikingly lower than those who received more than three of NESTs
assistance (mean=70.23). Table 7 shows how participants performed remarkably
different in terms of the listening comprehension tests between the less-than-a-year
40
NESTs-assisting group and more-than-three-year-NESTs-assisting group. There are
statistically significant differences in participants’ listening performance between the
groups (t=-5.61, p<.000). Such result is in line with the previously reviewed studies in
that having NESTs and NNESTs to co-teach can make a better learning environment
thus improve students’ English learning (e.g., Barratt & Kontra, 2000; Benke &
Medgyes, 2005; Poon & Higginbottom, 2000).
Table 6. Descriptive Statistic of Less-than-a-year NESTs Assistance Group and More-than-three-year NESTs Assistant Group
Table 7. Difference of Less-than-a-year NESTs Assistance Group and More-than-three-year NESTs Assistant Group in scores
Mean Difference
41
Prior studies have reported that NESTs and NNESTs have their own strengths and
weaknesses and they seemingly make up for each other’s disadvantages (Medgyes,
1992). In this study, we compared groups of participants with less than a year of NESTs’
assistance and with more than three years of NESTs assistance and found that students
outperformed greatly with more years of NESTs’ assistance. The result of the present
study could suggest that NESTs might have actually created a more relaxing and livelier
classroom atmosphere which enhanced students’ learning motivation and thus
influenced their willingness to engage in class (Ryan & Patrick, 2001). The quantitative
analysis of the reports also provides insight to such a finding that classrooms went from
being quiet and teacher-fronted to being interactive and participative in nature. For
example, in the beginning of the school semester, even when the NEST proposed games
or competitions, no obvious excitement was observed on learners. However, at the end
of the term, learners began showing much excitement through clapping and counting
down with the NEST. Furthermore, one of the learners stated that:
“聽不懂就用猜的啊！反正會有人聽得懂，他就會跟我們說老師說什麼。
老師教的課都很有趣，我喜歡上老師的課。我覺得我英文進步了！”
“I’ll guess when I don’t understand! Someone will understand so we just wait for her to tell us what the teacher says. I like attending the teacher’s class, because it is
always very interesting. I think my English improves!”
42
It seems that with better involvement of the students in the English class and the
presence of the NESTs with their friendliness and perfect linguistic models, students
were more likely to achieve higher learning satisfaction and thus have a better learning
outcome (Zandvilet & Fraser, 2005).
Some other previous studies found that students performed better with the use of
modified listening passages when taking the exam, these studies either tended to have
a smaller number of participants or have grown-up participants, who might have better
test-taking skills. In this study, we explored the listening comprehension ability of
younger age students and extended the number of the participants to a larger scale,
including students around Taiwan. We found that students with more years of NESTs’
assistance did perform better in terms of listening comprehension tests. It seems that
exposure to authentic input, namely the NESTs’ oral instructions and teaching, is
beneficial for listening comprehension and preferred by L2 learners as previous studies
have stated (Vandergrift, 2007; Yen, 2016). The findings of the present study also
extended those of Wu (2015) and Lin’s (2017) studies, confirming that NESTs do
influence learners’ listening comprehension. In fact, the longer the exposure to NESTs,
the more students show improvement (Wu, 2015; Lin, 2017). In addition, the
improvements of the students’ listening comprehension noted in the present study were
unrelated to any of the specific co-teaching models, because the participants included
43
in the present study were randomly selected from different programs implemented in
Taiwan which applied a variety of co-teaching models.
When comparing the listening performance of the participants across five areas
(see Table 8), the differences among the areas became much smaller (F=3.42, p<.05)
with the assistance of NESTs for more than three years than with less-than-a-year
NESTs assisting (F=4.65, p<.001). Although the difference is still statistically
significant, the difference was a much smaller value, suggesting that listening
achievement differences could be narrowed by more years of NESTs’ assistance. This
narrowing of the achievement gap in terms of listening is an encouraging finding
because as there is much discussion (e.g., Bensimon, 2005; Flores, 2007; Haycok, 2001)
on solving achievement gaps in education, not many have found a satisfactory answer
to this gap (Billings, 2006). This might indicate that the benefits of having longer
interaction with NESTs may lead to higher listening comprehension. In other words,
the assistance of NESTs may represent a new approach to mitigate the problem of wide
differences between the English abilities of children from affluent families and those
from poor rural areas in Taiwan (Chang, 2007). As a result, after seeing the encouraging
result of narrowing of the achievement gap in different areas in this study, it will be
worthwhile in future studies to investigate more specifically how NESTs reduce the
achievement differences for these EFL learners.
44
Table 8 Participants’ Listening Test Scores among Different Areas with Less-than-a-year NESTs Assistance and More-than-three-Less-than-a-year NESTs Assistance
Area North Middle South East Offshore
When looking at the result from the gender aspect, the listening performance of
both genders improves significantly with the assistance of NESTs for more than three
years (male: t=-4.8, p<.000; female: t=-3.26, p<.01). Table 9 shows how participants
performed remarkably different in terms of the listening comprehension tests between
the less-than-a-year NESTs-assisting group and more-than-three-year-NESTs-assisting
group. As we can see from Table 9, the impact of NESTs on male participants is
particularly worth noticing because the mean score for male participants who received
more than three years of NESTs assistance (mean=68.84) perform significantly better
than those with less than a year of NESTs’ assistance (mean=58.99). This result might
suggest that NESTs might not only had a great effect on raising male learners’ attitude
45
towards English learning, but also had an effect on improving their English listening
performance as previous studies (e.g., Gardner, 2008; Green & Oxford, 1995; Wu, 2005)
had explored.
Table 9 Male and female student’ listening test scores with one-year and/or less NEST instruction and three-years and/or more NEST instruction
Mean Score
Green and Oxford (1995) pointed out that males and females hold different
motivations, attitudes, and learning styles to language learning. Furthermore, female
learners were more often identified as global learners than male learners. That is,
females performed better in terms of language learning than males because females
were more capable of utilizing global strategies, such as identifying the main idea and
understand the author’s perspective, and were better at managing new languages.
However, the finding of the present study shows that with more years of NESTs’
assistance, the improvement of the male learners was significantly different from their
female counterparts in terms of their listening comprehension. This might be the result
of their constant interaction with the NESTs which expands the results of existing
46
studies such as Lin (2017), Yen (2016), and Wu (2015). The qualitative analysis of the
reports provides insight to such a result. For example, learners stated that:
“我覺得有外國老師的英文課比較好玩也比較有趣。”
“I feel that classes with foreign teachers are more interesting and fun.”
“我可以聽得懂外國老師說的話。”
“I can understand the foreign teacher’s words.”
“我可以輕鬆的與老師用英語交談。”
“I can talk to my teacher in English easily.”
These showed that learners had fun and enjoyed the English class more with the
assistance of NESTs in the language learning classroom. In addition, there seemed to
be the tendency that the learners became more accustomed to listening and speaking
English and felt even more comfortable using English to interact with NESTs. With
higher motivation, lower learning anxiety and more confidence and chance in using
English when interacting with NESTs (Ho, 2013; Huang, 2011; Yen, 2016), it is thus
not surprising to find that learners can perform better in their listening comprehension
after more years of NESTs’ assistance, especially male learners, as the result shown in
the present study. Although many other factors should be taken into consideration, such
as the individual differences of the participants including their family social status,
personal characteristics, the large sample size of this study might be sufficient to
provide a general pattern of learners’ listening performance (Faulkner, 2003).
47
The finding of this research question therefore indicates that the benefits gained
from the assistance of NESTs, especially in terms of improving learner’s listening
comprehension skills, may address that such assistance in whichever co-teaching model
is beneficial to students in Taiwan (Barratt & Kontra, 2000; Carless & Walker, 2006;
Chang, 2007; Storey et al., 2001).
4.2 Influence of the Assistance from NESTs on Students’ Different Listening
Comprehension Skills
Question 2: Do the effects of having NESTs as English Teaching Assistants vary by
different listening comprehension skills?
On the whole, participants with more than three years of NESTs’ assistance
perform better throughout the entire test. That is, the average scores participants with
more than three years of NESTs’ assistance received in every part of the test were fairly
higher than those with less than a year NESTs’ assistance. Table 10 illustrates the
findings of the descriptive statistic of less-than-a-year NESTs’ assisting group and
more-than-three-year NESTs’ assisting group in each test part. The statistically
significant differences of the listening comprehension performance in each part
between less than a year of NESTs assistance group and more than three of NESTs
assistance group were reported in Table 11. The differences between the groups in Part
One is slightly smaller than the other parts; however, it has almost reached statistically
48
significant differences (t=-1.95, p<.051).
Table 10. Descriptive Statistic of Less-than-a-year NESTs Assistance Group and More-than-three-year NESTs Assistant Group in Each Test Part
Part Group N Mean S.D. Min. Max.
49
50
Part Six Less-than-a-year NESTs Assistance
278 1.58 1.67 0 5
More-than-three-year NESTs Assistance
304 2.18 1.77 0 5
51
Table 11. Difference of Less-than-a-year NESTs Assistance Group and More-than-three-year NESTs Assistant Group in Each Test Part
Part Mean
Generally speaking, the listening competence that participants should perform
throughout the present communicative test matches the two major communicative
listening sub-skills that were proposed by Weir (1993) in the taxonomy of
communicative listening sub-skills, including (1) direct meaning comprehension, such
as listening for the main idea(s) or important information, being able to distinguish from
supporting detail, or being able to listen for specifics, and being able to include a recall
of important details, and (2) inferred meaning comprehension, such as deducing the
meaning of unfamiliar lexical items from context, recognizing the communicative
function of utterances, relating utterances to their social and situational contexts.
52
In Part One, according to the rubric of the listening test, participants should be able
to deduct the meaning of unfamiliar lexical items from context (Weir, 1993). Once the
students heard the keywords such as lion, monkey, dog, they could draw lines from the
names of the children to the corresponding figures on the picture. Most of the keywords
for the correct answers were already learned by the participants and the directions to
the right answers are rather straightforward. It should, however, be noted that even
though the listening comprehension test used in the present study was adapted from
standardized tests and the vocabulary had been tailored into elementary level following
the elementary vocabulary list proposed by the MOE, some of the keywords to the
correct answers remained unchanged such as “crocodile” and “lizard” in order to reflect
the authenticity of communicating in English. After all, one cannot expect to understand
every single word in an authentic conversation or listening passage (Morrow, 1979).
Participants with the more years of NESTs’ assistance achieved higher mean
scores in Part One than their counterparts. In order to get the correct answer, participants
should be able to perform inferred meaning comprehension, namely deducting the
meaning of unfamiliar lexical items from context (Weir, 1993). For example, they
should be able to know the boy who was drawing a crocodile is next to the girl who is
drawing a monkey. Even though the participants might not know the word “crocodile,”
the participants should be able to understand the context that “the boy is next to the girl.”
53
When analyzing the classroom observation reports, we also found that NESTs would
use English to give instructions such as “Turn in your paper,” and “Time’s up,” or
“Markers down,” and so on. Although students might not understand the exact meaning,
they were able to infer from the context (Buck, 2001). In the class observation report,
students also pointed out that when they did not understand what the NESTs were
talking about, they would just make a guess based on the task they were doing. When
they took the correct action, the NESTs would smile and praise. As a result, NESTs
seemed to be good at getting learners to participate and arouse their willingness to illicit
or guess meaning from the context (Benke & Medgyes, 2005). It seems that NESTs do
facilitate L2 learners to comprehend the target language in real-life situations (Rogers
& Medley, 1988) for they had built up the ability to deduce the meaning judging from
the given context or situation.
In Part Three, participants should also be able to perform inferring meaning
comprehension. To be more specific, Part Three required test-takers to relate utterances
to their social and situational contexts (Weir, 1993). Once the participants heard the
question words such as what, how many, which, they should choose the correct answer
judging from the given situational context in the listening test. The data showed the
participants with more-than-three-year of NESTs’ assistance had done better on this
part than the participants with less-than-a-year of NESTs’ assistance. This might
54
address that participants from more-than-three-year of NESTs’ assistance group had
gradually developed the tendency of listening carefully as soon as the NESTs started
asking questions, for it was important to hear the very first word of a question due to
the longer length of interacting with the NESTs than their counterparts. Some
researchers also found that NESTs focused more on listening and speaking (Lin, 2008;
Medgyes, 1999). Such finding is in consistent with the results of the present study and
classroom observation. As the classroom observation reports stated, NESTs usually
started the class with greetings and daily routine questions. The NEST asked the
students about the day and a student replied but not in the way the NEST wanted. The
teacher corrected the student by restating the correct answer. Upon hearing the NEST’s
answer, the students laughed. Although the laughter was a small gesture, it showed that
the students were becoming aware of the NEST’s expected answers. The NEST did not
correct the student by pointing out the misunderstanding of the question, neither did the
NEST ask the student to refer to the textbook for what they have already learned. The
NEST we observed was being friendly and encouraging to get learners to speak, which
is in line with the finding of the previous study (Benke & Medgyes, 2005). The students
were experiencing the language which is used for real communication by native
speakers (Rogers & Medley, 1988). Therefore, students with more years of NESTs were
better at performing inferred meaning in the listening test as they had more chances to
55
practice with the NESTs class.
In Part Two, Four, Five and Six, although most of the keywords were learned
(foods, color, etc.), participants could not get the correct answer before listening to the
entire conversation which takes time and patience. Participants were expected to: (1)
listen for specifics, including a recall of important details in Part Two (2) listen for the
main idea(s) or important information; and identification of supporting detail or
examples in Part Four and Six (3) determine a speaker’s attitude or intention towards a
listener or a topic in Part Five base on the rubric of the listening test (Weir, 1993). The
listener or a topic in Part Five base on the rubric of the listening test (Weir, 1993). The