Effectiveness of TOEIC Preparation Courses for Technical College Students

(1)

31 - 40 頁 pp. 31 - 40

Effectiveness of TOEIC Preparation Courses for Technical College Students

Chien-Hui Hung

1,*

Abstract

This paper studies the effectiveness of TOEIC preparation courses for technical college students. The aim of this study is twofold. The first one is to learn if the TOEIC preparation courses help technical college students in the TOEIC exam. The other is to discuss the factors and variables affecting technical college students’ TOEIC scores. A total of 60 technical college students in northern Taiwan were included in this study. Students were divided into two classes and both classes received 10-week, 60-hour TOEIC preparation courses. TOEIC scores before and after this preparation program were compared to study the improvement. A student satisfactory survey was also implemented to gather students’ opinions for this program. Finally, this paper concludes with suggestions and recommendations for the future course design.

Keywords: TOEIC preparation courses, technical college students, course design

I. Introduction

This paper discusses the effectiveness of TOEIC preparation courses by examining the case of a four-year technical college in Northern Taiwan. This technical college currently has 11 departments and the majority of them are Engineering-related majors. This college has roughly 5000 students and male students outnumber female students. The English education of this technical college is provided by its General Education Center. An 8-hour English curriculum is required for all students including 2 freshman English classes and 2 sophomore English classes.1_{In addition, the}

English graduation threshold of the CEFR A2 level for all students has been implemented starting from the class of 2013.2

Most students at this technical college have limited English proficiency. In addition, these low-achievers have low motivations for learning English. For the 2019 class, only 45% of freshmen have the level of CEFR A2 upon entering

1_{Oriental Institute of Technology} *_{Correspondence author: Chien-Hui Hung}

E-mail：fb061@mail.oit.edu.tw

1_{The English curriculum of 8-credit hours is going to change to 6-credit hours starting from the class of 2020. The old curriculum is}

4-credit English courses for freshmen and 4-credit English courses for sophomores. The new curriculum is 4-creidt English courses for freshmen and 2-credit English courses for sophomores.

2_{This college terminated the required threshold of English in the spring semester of 2017. The English graduation threshold of the A2}

(2)

the college. After the completion of Freshman English courses, still less than 70% of students reach the CEFR A2 level according to the school reports.3_{Students did not perform well on TOEIC exams, either. As shown in《Table 1》below,}

the number of students who reach the CEFR B1 level (TOEIC 550) is just very few. For a college of almost 5000 students, this result was not satisfactory.

Table 1.

Academic Year 2012 2013 2014 2015 # of students

CEFR B1 (TOEIC 550)

11 1 5 6

In order to help students meet the high requirement of English proficiency in the job market in the future, TOEIC preparation courses have been implemented in this technical college to help students prepare for the TOEIC Test. The goal of the courses is to help students achieve a score of at least 550, the CEFR B1 level.

Accordingly, the aim of the study is to understand if the TOEIC preparation courses are effective. The effectiveness is the extent to which this TOEIC preparation program helps improve students’ TOEIC scores. The improvement was measured from students’ score increases by comparing their pre-TOEIC and post- TOEIC test scores and was also judged from students’ opinions on this TOEIC preparation program.

Three important questions were discussed in this paper. First, did the TOEIC preparation courses help technical college students do well in the TOEIC exam? Second, what factors affected students’ TOEIC scores? Third, what did technical college students and instructors think of this TOEIC preparation program?

In order to understand the effectiveness of this TOEIC program and to answer the above questions, this study employed three methods. The first method was the test score comparisons. By comparing the pre-test and the post-test scores, students’ score improvements (if there is any) could be identified. Second, this study conducted a student satisfactory survey to collect students’ opinions on this program. Lastly, teachers’ observations both from engaging in class and from interacting with students after class were used as a way to understand students’ problems and difficulties in preparing for the TOEIC test.

II. Literature Reviews

In terms of planning on a course, the first concern is the length of the course. How long should the course be in order to achieve the effectiveness? There are various studies and reports on this issue of study or teaching hours. The

Association of Language Testers of Europe has provided the number of guided teaching hours needed to reach each

CEFF level. According to the “Teacher’s Guide to the Common European Framework”4_{, in order to reach an A2 level}

(TOEIC 225), 150-200 guided teaching hours are required and 350-400 guided teaching hours are required to reach a B1 level (TOEIC 550) (p.7). In other words, for a student with English level of A2, at least 150 hours of guided teaching hours is needed in order to help him or her to reach the B1 level.

(3)

Previous studies have done in assessing the effectiveness of the TOEIC preparation program by comparing score changes. By adapting the data from Saequsa’s research (1985), Trew (2007) pointed out that for a score increase, the average study time was 2.25 hours (from score 450 to score 550). At an over three-year research period, Hisatsune (2007) discovered that after 36 hours of intensive instructions in three weeks, students could have an average of 100-point improvement for those with an average pre-score of 330. Huang et. al (2017) detected an overall score of gain of 56 after 52.5 teaching hours.

There are also studies done on investigating the impact of TOEIC preparation courses on different English skills. Lai (2008) found out that the TOEIC preparation program helped improve students’ reading comprehension scores significantly. However, students’ listening comprehension scores did not increase significantly after the preparation courses. Wu (2013) also detected a significant improvement on both reading and listening scores after the preparation program. However, in Wu’s research, listening improved greater than reading. Wu believed this insignificant improvement might result from students’ slow reading speed.

As mentioned previously, there have been tons of studies in evaluating the effectiveness of TOEIC preparation program and in discussing its impact. However, fewer studies were done to discuss the effectiveness of the TOEIC preparation program for English low-achieving students. Therefore, this paper targets technical college students with limited English proficiency.

III. Methods

This technical college has already had TOEIC preparation courses for a couple of years. However, students always began with high motivations but started to drift off after classes began. Students’ post TOEIC scores did not improve much, either. In order to understand the reason for dropping attendance rates for TOEIC preparation courses and to help students improve TOEIC scores, this technical college made changes to this TOEIC preparation program.

This college re-designed the program and divided the classes by students’ English levels and by instruction type. Two classes (Class A & Class B) were formed with each had 60 hours of instruction time. The entire program lasted for ten weeks and classes met twice a week, three hours per meeting. Reading and listening were taught separately. In other words, students had thirty-hour reading instruction and thirty-hour listening instruction under this ten-week TOEIC preparation program. .

Ideally, Class A was for students with higher TOEIC pre-scores. However, some students with higher TOEIC test scores signed up for Class B due to the schedule conflicts. Class A used two textbooks and taught by two instructors teaching reading and writing independently, while Class B was led by one instructor only, teaching both reading and listening. Class A and class B both required students to take a pre-test (TOEIC mock test) and a post-test (official TOEIC test) before and after the ten-week program. The Official TOEIC test procedures and regulations also applied to the mock test.

(4)

IV. Results

Comparisons of Pretest and Posttest

As shown in《Table 2》below, twenty-three students sign up for Class A while thirty students sign up for Class B. For both classes, male students outnumber female students. In class A, most students are in the senior classes. Class B includes four freshmen, three sophomores, ten juniors and thirteen seniors.

Table 2. TOEIC Class Student Descriptive

TOEIC Class A

TOEIC Class B

# of students

# of Male

# of Female

23

15

8

30

21

9 # of Freshmen

# of Sophomore

# of Junior

# of Senior

0

11

12

4

3

10

13

Although twenty-three students signed up for Class A, fourteen students showed up at the first few weeks, only eight students had full attendance. Those eight students have been considered to be highly motivated compared to other students. But after this ten-week TOEIC preparation program, statically speaking no improvement was seen from the results.5_{The results are shown on《Table 3》below.}

We did see an improvement on Number 2 student because he had lower English ability to begin with. Thus, this program helped him perform greater than other students. In addition, we found that students’ reading performance was worse than their listening performance. For all these 8 students, their listening scores were very close to 275 (listening required score of the B1 level); however, their reading scores were much lower than the score of 275 (reading required score of the B1 level). In other words, those students on average had worse skills in reading than in listening.

In addition, from《Table 3》we could see that students Number 4 and Number 7 have a great improvement in their reading scores. According to the teacher’s observations, these two students were also the ones who worked harder comparing to other classmates. They spent more time in vocabulary and grammar than their other peers, which could explain the increase in their reading scores.

(5)

Table 3. TOEIC Score Comparisons for Class A

Students Pre-Test TOEIC Mock Test

(Listening, Reading, Total Score)

Post-Test Official TOEIC Test (Listening, Reading, Total Score) No.1 280, 150, 430 275, 155, 430 No.2 185, 70, 255 220, 110, 330 No.3 315, 220, 535 270, 165, 435 No.4 295, 120, 415 250, 150, 400 No.5 280, 180, 460 260, 175, 435 No.6 320, 170, 490 250, 145, 395 No.7 310, 240, 550 220, 280, 500 No.8 325, 215, 540 260, 180, 440

In class B, only 16 students kept a full attendance record. Pre-test and post-test comparisons did not suggest an improvement, either. The results are shown in 《Table 4》below.6

Table 4. TOEIC Score Comparisons for Class B

Average Pre-Test

TOEIC Mock Test

Average Post-Test Official TOEIC Test

t test Class B (N=16) 409.69 (106.97) 389.69 (93.781) p=0.5781

Student Satisfactory Survey

In order to understand what the students thought of this TOEIC preparation program, this study also conducted a Student Satisfaction Survey for this preparation program. Eight students in Class A participated in this survey, while 20 students in Class B participated in this survey. Five questions were asked. These questions were as follows:

Q1. I was very satisfied with class content.

Q2. Textbooks and teaching materials were well-chosen. Q3. The instructor was effective.

Q4. This course was very helpful to me.

Q5. Overall, I was very satisfied with this TOEIC preparation program.

The responses from Q1 are listed in《Table 5》. From the table, we could see clearly that all of the students agree with this statement. The majority of the students were very satisfied with the class content.

6_{For both classes, the average TOEIC test scores after the program were lower than the average scores before the program although the}

(6)

Table 5. Student Satisfactory Survey Q1

Q1: I was very satisfied with class content.

Class A Listening Class A Reading Class B Listening Class B Reading Strongly Agree 3 5 12 11 Somewhat Agree 5 3 8 9

Neither Agree or Disagree 0 0 0 0

Somewhat Disagree 0 0 0 0

Strongly Disagree 0 0 0 0

As to the responses for the question number 2 shown in《Table 6》, all of the students agree with the Q2 statement, and most of the students show strong agreement for Q2 in Class A, while several students keep a neutral attitude and just one student express disagreement in Class B.

Q2: Textbooks and teaching materials were well-chosen.

While asked what students thought of the instructor, almost everyone agreed that the instructors were very effective. And most students agreed that this TOEIC preparation program was very helpful. Finally, most students strongly agreed that overall speaking, they were very satisfied with this TOEIC preparation courses. The results mentioned above are shown in《Table 7》,《Table 8》,《Table 9》respectively.

Q3: The instructor was effective.

(7)

Q4: This TOEIC preparation program was very helpful to me.

Q5: Overall, I was very satisfied with this TOEIC preparation program.

Student Feedback

In addition to a student satisfactory survey, we also asked the students to provide their feedback for this preparation program and for the instructors. For the listening courses, students liked the followings including the training in listening for keywords, accent training and the training in listening speed. Students also favored the interactive teaching styles and that the instructors provided various listening websites for students to practice after class. As to the suggestions for the listening course, students expressed their desires to have more classes or increase the class length in the future.

Concerning the reading courses, students liked organized teaching in grammar and various strategies taught in class to expand their vocabulary and improve grammar skills. Students would also like more classes and more frequent vocabulary quizzes in class. As to the instructor style, students liked encouraging teaching style and liked the instructor with a great humor and not being pushy. In their opinion, a positive classroom atmosphere and great interactions and good relationships between teachers and students helped them more.

Teacher Observations

Although most students were quite satisfied with this program, but this study did not detect much improvement in their TOEIC scores after this 10-week preparation program. Therefore, a report of teachers’ observations was attached as well to study the problem. By engaging in the class and by interacting with student after class, three instructors had the following observations.

(8)

First, the students just did not study hard and did not study enough. Although the students were motivated to come to the TOEIC preparation courses, yet their motivations were not strong enough to push them to study by themselves at home. What students desired was to have more TOEIC classes. Unfortunately, it is impossible for the school to offer longer hours or more classes for them due to the budget limit.

Second, students knew too few English words. They were not motivated to learn new words so their vocabulary was very limited. Although students had acquired a lot of vocabulary strategies from class, it’s difficult for them to expand their vocabulary size because they did not apply these strategies and did not study for them.

Third, their reading speed was too slow so most students were not able to finish all the reading questions within the time limit in the TOEIC test. As shown previously from《Table 3》, the students’ reading scores were much lower than their listening scores. According to teachers’ observations, students’ poor reading scores were found to correlate with a limited vocabulary and a slow reading speed.7

V. Discussion and Conclusion

This paper discusses the effectiveness of the TOEIC preparation program in a technical college in northern Taiwan. This TOEIC preparation program lasted for 10 weeks for a total of 60-hour reading and listening instructions. Although the majority of the students in this program were satisfied with the class content, the class materials and the instructors, no significant improvement on the TOEIC test scores after engaging in this preparation program was detected.

According to the instructors’ observations, students’ low motivation was the primary reason for the limited effectiveness of this preparation program. Although students were motivated to come to classes, they simply did not study enough. Students’ limited vocabulary, poor grammar skills and their slow reading speed resulted in a poor TOEIC score. Accordingly, in order to increase the average TOEIC scores for technical college students, it is essential to take motivation factor into consideration. No matter how good the teaching is or how effective the instructors are, students’ scores will not increase if they do not study. Therefore, it is rather important for the instructor to find ways to motivate students to study hard after class.

In addition, the pre-test used in this study was the mock TOEIC test which was considered easier than the official TOEIC test. Some students have even done the same mock test before so that students on average scored higher in the pre-test. In order to have a more accurate comparison, it is suggested that employing the official TOEIC test as the pre-test in the next study.

Finally, this study has several suggestions for future course design. First, since class length and class time is limited, the priority should be the teaching in reading since students on average have lower reading ability. Second, it is important to help students acquire more vocabulary. Knowing more English words will help them score higher in the TOEIC exam. Third, the training in reading speed is necessary. By increasing students’ reading speed, they will be able to finish the reading questions within the time limit and then score higher in the reading comprehension section.

(9)

References

1. Hisatsune, A.(2007). Meet the challenges: Empowering TOEIC students. In K. Bradford-Watts (Ed.), JALT 2006 Conference

Proceedings. Tokyo: JALT.

2. Huang, L. et al. (2017). Assessing the effectiveness of preparation courses for the TOEIC test. Language and International Studies, 17, 29-58.

3. Lai, Yi-hsiu. (2008). A study on the effectiveness of college English-featured courses on TOEIC. 高雄師大學報, 25, 71-91.

4. Saegusa, Yukio. (1985). Prediction of English proficiency progress. Musashino English and American Literature, 18, 165-185. 5. Trew, Grant. (2007). A teacher’s guide to TIEIC listening and reading test preparing your students for success. Oxford: Oxford

University Press.

6. Wu, Hsing-ling. (2013). Implementation of an elite sophomore English class at a university in Taiwan. Hsiuping Journal of Humanities

(10)