國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 4 Results
When all the lessons had been taught and both tests administered, the data was analyzed. Two scores were uncounted. One was uncounted for the reason that he slept through the instruction periods, and then only took the test, because the local teacher forced him to take the test. Playing with scissors and a ball point pen was more interesting than taking the test. The other was uncounted due to the fact that the student did not really understand the instruction for the worksheets and tests.
On the pretest, the experimental group average was 84. The standard deviation was 12. The average on the reduced sound section was 86.5. The standard deviation for this section was 12.6. The standard deviations may seem large, but this is due to the huge difference between the top learners and the bottom learners. If the standard deviation divided by the average is less than one, there is a low variance between scores.
For the control group, the pretest average was 79.9. The standard deviation was 17.3. The average on the reduced sound section was 81.5. The standard deviation was 19. Again, this is because of the huge difference between the top and the bottom.
The post-test average for the experimental group was 88 percent. The standard deviation was 12.6.The average on the reduced sounds section was 87.5. And the standard deviation for this section was 13.5.
‧
The post-test average for the control group was 85.4. The standard deviation was 13.6. The average for the reduced sounds section was 83.2 and the standard deviation was 15.8.
The pretest median for the experimental group was 87 while the reduced sound section of the pretest was 90. The post-test median for the experimental group was 91.5 and the reduced sound section had a median of 90.
The pretest median for the control group was 87 while the reduced section of the pretest was 83. The post-test median for the control group was 90 and the reduced sound section was also 90.
The t-score for the experimental group was - 1.87 and the t-score for the control group was - 0.80. The score for the experimental group was 0.49. The z-score for the control group was also 0.49. The p-value for both groups was 50 percent.
This confirms the null hypothesis. There was no statistical difference between the experimental group and the control group in terms of pretest and post-test
performance. Normal learning would account for the slight increase in test scores.
(See Table 2 for these statistics.)
Table 2 – Statistics Results of the Study Overall Score
‧
I could cherry pick and point to individual scores that improved by leaps and bounds. Some students showed an obvious benefit from the course. But the overall numbers do not bear that out.
However, statistics are not the only way to measure progress. I also had the student evaluation and the interviews with the local teachers who were with me in every class. In both student surveys and teacher interviews, it was felt that progress had been made in listening comprehension
Teacher Interviews
After the lessons had been taught, I interviewed the three local teachers one by one. These local teachers were present in the classroom with me during the
presentations. The interview consisted of five questions: (1) What went right? (2) What went wrong? (3) What could be improved for next time? (4) Did the listening curriculum measure what it was supposed to measure? (5) Do you think this study will have a long term effect on the students’ listening ability?
I interviewed each teacher alone. After I interviewed the three of them, I compared the answers to each of the five questions. The interviews lasted about 5 – 10 minutes. This seems to be a short time. I was limited by the teachers’ busy
‧
schedules. The teachers were free to speak their minds and address any praise or concern they had. After the interviews, I transcribed the interviews and compared the answers given by the respective teachers.
All the interviews were recorded on a Sony IC Recorder MP3 3.1.0. Each teacher mentioned how much they liked the warm ups and wrap ups. But they were also able to see that the student’s listening comprehension had improved. Hwang, C.P.
said, “I think the students benefited from it a lot” (personal communication, December 8, 2017). (All names are pseudonyms.)
When asked about the possibility of lasting impact, all the three local teachers concurred that these lessons will affect the long term English listening comprehension of the students (Hwang, C.P., personal communication, December 8, 2017; Tsai, Y.W., personal communication, December 8, 2017; Chuang, H.J., personal communication, December 15, 2017).
One local teacher suggested that next time there be more passages read with questions at the end as was the case in section F of my test (Hwang C.P., personal communication, December 8, 2017). Another local teacher suggested that next time I could record different voices on the soundtracks other than just my own (Chuang H. J., personal communication, December 15, 2017). That would be fine, but I did not have the resources to ask friends to record the sound tracks with me at this time.
In the interviews, the local teachers were all complimentary in their criticism of the listening curriculum. “All of the class I like it [sic].” (Tsai, Y.W., personal communication, December 8, 2017)
A criticism of the listening worksheet sound tracks was that only one person read both the passages and the questions. “We need more voices in your recordings.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Otherwise they are easily getting confused [sic]. For example, you need someone to speak in English for the introduction part, and then another to be in charge of the recording. I mean one is in charge of asking questions then the other reads the answers (Chuang, H. J., personal communication, December 15, 2017.).
Student Response
At the end of the post-test, there was a survey in Chinese for all the students. I chose to make the survey in Chinese only so that all the students would be able to answer. Some of the upper level students are not still able to express themselves well in English, not to mention the lower level students. The survey was all in Chinese. I translated the survey by myself except for a few words in illegible handwriting. For those I had a colleague tell me the correct translation.
The survey for the students had three questions : (1) What did you learn in class?, (2) What part was easy for you?, and (3) What part was difficult for you?.
Many of the students answered in words or phrases as opposed to complete sentences.
Disappointingly, many of the students only mentioned that they had enjoyed the warm-ups (songs) and the wrap-ups (games).
However, 34 students specifically mentioned that their listening had improved as a result of the classwork for this study. “My listening improved,” (「聽力進
步。」) said David in room 801. Carrie from 801 said, “This let my listening become better. (「讓我聽力測驗更好。」). Oliver from 805 answered in English, and said,
“You improve my listening and writing [sic].” (All names are pseudonyms)
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
The Researcher/Teacher’s Reflection
The lessons went as they were supposed to. There was a smooth transition between all three parts – the phonics, the grammar and the listening or minimal pairs.
The listening tracks went smoothly. The Chinese native speaker’s voice seemed a bit loud in the first few lessons. So I used Audacity (Audacity team, 2016) to turn the volume down on her voice. Overall, I was pleased with the lessons and the tests.
Slight improvement was found in both experimental and control groups. I believe this is due to students having forgotten much English over the summer, and their teachers having pushed them during the fall semester of 2017.
I also believe that the statistical flat line was partly because the pretest was too easy for the students at the top of the class, and partly because the students at the bottom have a defeatist attitude in English class. The worksheets were not challenging enough. There may also have been a placebo effect, as I told each of the six classes that the program would help their listening. A ceiling effect was noted.
Class 801 was in the control group, but their test scores improved significantly.
I am happy for the improvement, but it really did not help my theory. Class 803 and 805’s test scores remained fairly flat, however. These classes went as expected.
During the coursework, some of the students at the top seemed puzzled by the reduced sounds, specifically the weakened vowels. They did not always hear them consistently. Perhaps this is due to having only one person read all the tracks. Perhaps this is because neither the foreign teacher nor local teacher use reduced sounds in the regular classroom for normal classes.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
During the reduced sound lessons, some of the students at the bottom acted as if they had given up in hopeless despair. I do not think it is wise to push them to listen for reduced sounds when they struggle with formal, unreduced English.
In my log, I spent much of the time noting the misspellings and glitches in the worksheets and tests. But I learned a lot from this study. I learned that one can push the students harder and faster than I previously thought. I learned that if one has the choice as I do, it is better to concentrate one’s effort on the upper level students, and not put too much hope in the students at the lower levels. They have already given up on learning English, and it is exceedingly difficult to inspire them to tr