The research instruments applied in this study were the speaking proficiency assessment, speaking proficiency assessment raters, speaking proficiency assessment criteria, teacher’s journal and student’s feedback form. They were introduced respectively as follows.
3.3.1 Speaking Proficiency Assessment
In order to measure the effect of Readers Theater on EFL elementary sixth graders’
English speaking proficiency, the assessment for pretest and posttest were necessary utilized on 6-A students. Lee (2008) proposed that her EOPT (English Oral Proficiency Test) showed the qualification for the criterions of a superior test inventory. The reliability and validity were appropriate and useful for examining English oral proficiency of elementary students. Therefore, this study would adapt Lee’s (2008) English Oral Proficiency Test as the pretest and posttest to check if there was any significant improvement on 6-A students’ English speaking proficiency. By utilizing the same Speaking Proficiency Assessment in the pretest/ posttest, the researcher was able to compare the difference about students’ speaking proficiency after the RT implementation. The content of the pretest and posttest in this study would be modified and rearranged based on different topics which were chosen from Here We Go! Book7 and Taipei City Elementary School Listening & Speaking Assessment Handbook.
Especially the handbook was designed by the teachers from English Curriculum &
Instruction Consulting Team, Taipei City; hence, the reliability had been proved.
The Speaking Proficiency Assessment (Appendix B) included four main topics with three components, linguistic competence (7 questions), interactive competence (7 questions) and sociolinguistic competence (2 questions) (Lee, 2008). The first topic, “A Happy Face” was the topic focused on personal feelings, and the context was based on
the conversation with a new student, Carlos. The second topic, “I’m a Helper” was the topic about students’ family and their daily routines. The context asked students share their experience about helping their family members. The third topic was about some actions in “Hey, Stop!”, and the context was based on meeting a friend on street and had a short talk with him/ her. The fourth topic was “Let’s Go Somewhere” that the context was set in a brief discussion of winter vacation and locations with classmates.
The reason why the researcher chose to create new theme name was to reduce students’ anxiety which might related to their former experience about taking the exams;
furthermore, the different theme name could cause the another impression in case students would relate the speak proficiency assessment to the textbook. The distribution of questions in the speaking proficiency assessment was shown in Table 3.2.
Table 3.2
The Distribution Content of Speaking Proficiency Assessment Theme Original
Let’s Go
Somewhere Unit 5 Locations
13 Linguistic Competence 14 Interactive Competence 15 Interactive Competence 16 Linguistic Competence As shown in Table 3.3, the assessment scoring included 4 topics, 16 questions, and each questions equaled 4 points, so the 16 questions totaled 64 points.
Table 3.3
The distribution of the questions and scores
Language Competence Question No. Numbers of
Questions Total Scores Linguistic Competence 1,4, 7, 9, 10, 13, 16 7 28 Interactive Competence 3, 5, 6, 8, 11, 14, 15 7 28
Sociolinguistic Competence 2, 12 2 4
Totals 16 64
In order to increase the validity and reliability, firstly the researcher would examine the content relevance to see whether the content of the assessment are representative of the kinds of language skills which the researcher wanted to assess.
Moreover, as to the reliability, the researcher would try to make the assessment more reliable. First of all, it was necessary to apply over one rater in the scoring. Used the experienced and trained raters, and provided a detailed scoring scale could help the scoring more smooth and objective. According to the person-related reliability, the researcher should ensure the students were prepared before the test and provide clear and explicit instruction during the test. Therefore, the researcher would offer a detailed explanation before the speaking assessment and make a simple introduction before each task. Finally, the instrument-related reliability should keep the assessment conditions constantly. Hence, in the Speaking Proficiency Assessment, the researcher
would play the recording, and students could answer the questions based on the information of the pictures. Moreover, the researcher would ensure the assessment was well laid-out and perfectly legible.
3.3.2 Speaking Proficiency Assessment Raters
For objectivity’s sake, two raters were required to give scores to the pretest and posttest of the experimental group, 6-A. The section presented the teaching experience and academic background of two raters. The first rater was the researcher who graduated from the English Department of National Taichung University, and she had studied in graduate school of education in National Taichung University for a year. The researcher’s English teaching experience had last for over three years. Within her English teaching experience, the researcher had tried a variety of teaching activities in English class. However, the researcher hadn’t utilized Readers Theater in the English teaching; therefore, the researcher was exciting about how the Readers Theater implementation brought the effect of student’s language improvement.
As to the second rater, Mrs. Lee, she was the sub-teacher in Happy Elementary School, New Taipei City. She had been teaching English for over five years, and she had instructed 6-A since they were fifth-grade. Mrs. Lee put much emphasis on their learning effect of English, and she had tried introducing some activities in teaching festivals in order to increase 6-A students’ learning interest. Speaking of the rating, the two raters would accept the speaking criteria during the assessment. The raters would give scores together, and they would score right after the student answered each question. The scores would be recorded by the raters in the scoring form (Appendix D).
3.3.3 Speaking Criteria
The pretest and posttest were graded based on the same speaking criteria and raters.
The raters would give the scores on each student’s answers in the pretest and posttest with the speaking criteria. The researcher would adapt Weir’s (1988) Assessment
Criteria for the Oral Test. Weir (1988) provided a simple grading criteria to make it clear for teachers with less experience in speaking assessments, and the reason why the researcher chose the Assessment Criteria for the Oral Test was that it integrated the speaking fluency and accuracy which were the main aspects the researcher wanted to focus on. However, Weir’s grading criteria was so detailed that it would be a bit of difficult for the raters to give all the scores on every aspect in such a short amount of time. Therefore, after the discussion with the other rater, Mrs. Lee, the researcher decided to merge the scoring description of every aspect in Weir’s grading criteria together into a multiple column. The student could get scores if their performance was corresponding to the description, that is to say, from one to four points, the higher level got higher score for each question.
According to Underhill (1987), whatever categories the teacher chose must be fully understood by assessors, and preferably be developed by assessors or at least with their full involvement. The speaking criteria of the speaking proficiency assessment were introduced in Table 3.4.
Table 3.4
Speaking Proficiency Assessment Criteria
Level Score Description
1 1
Unable to function in the spoken language. Almost all grammatical patterns are inaccurate. Utterance is incomplete, fragmentary and incoherent. Vocabulary is adequate. Severe and constant rhythm, intonation and pronunciation problems cause complete unintelligibility.
2 2
Sentence structure is fragmented and there are frequent grammatical in accuracies. Sentences are mostly disjointed and restricted in length. Vocabulary limited to that necessary to express simple elementary needs. Strong interference from L1 in rhythm, intonation and pronunciation.
3 3
Some grammatical inaccuracies. Utterances may still be hesitant, but are gaining in coherence, speed and length. There are signs of a developing active vocabulary. Rhythm, intonation and pronunciation require concentrated listening, but only occasional misunderstanding is caused or repetition required.
4 4
Almost no grammatical inaccuracies. Utterances are occasionally hesitant. Almost no inadequacies or accuracies in vocabulary for the task. Pronunciation, rhythm and intonation are reasonably comprehensible to native speakers.
3.3.4 Instructor’s Journal
The researcher as an instructor would write the journal as a record which included the classroom interaction and the reflection after each English class. (Appendix E) The researcher would keep the journal after each RT lesson, so the journal would last for over 20 times in the 14-week RT implementation. The journal would focus on both teacher and students. The content was contained with the four orientations such as the description of students’ performance, difficulties and teaching problems, the solutions of today’s problems and the researcher’s reflection during the RT teaching. The researcher should concern how the students’ performance in the trainings was, and what
improve teaching to make students enjoy learning? The Journal not only could help the teacher to improve her teaching but also provided the suggestions on further English RT teaching.
3.3.5 Student’s Feedback Form
In the final step of the Readers Theater implementation, students are asked to complete the feedback form after each RT performance. 28 students were asked to finish their four-time RT Feedback Forms every three weeks. The design of the feedback form (Appendix F) was based on three parts, pre-performance, performance and post-performance. In the pre-performance part, students required to recall and wrote down how their feeling was, what difficulties and problem they had encountered, what assistant they might need and whether they felt any improvement or not. In the second part, the performance, students mainly described their feeling that they might feel the self-confidence, anxiety, nervousness or stage fright while on the stage. In the third part, post-performance, students needed to accomplish their reflection such as the attitude toward Readers Theater, the opinions of other groups’ performance, the enthusiasm or the advices of the next Readers Theater script.