3.3 Instruments
3.3.1 The Pretest and the Posttest
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
30
and materials. First, in order to achieve the purposes of the study, the researcher adopted two tests: a pretest and a posttest to collect quantitative data for exploring the effects of DI on junior high EFL students' grammar learning.
Besides, iEnglish Book 1 by Hanlin (翰林) Publisher was selected as the main teaching material for the subjects during the semester. Due to the aim of exploring the effects of DI on grammar, the treatment was based on one differentiated instructional strategy called tiered tasks to design three differentiated worksheets of every grammar unit. Furthermore, the researcher also applied one DI grouping strategies flexible grouping to provide the best learning experience for different proficiency learners.
In addition to the quantitative data, a list of focus group interview questions was designed to help the researcher probe into learners’ thoughts regarding their DI experiences in a qualitative approach. Each of these instruments had unique study focuses and thus needed further explanation in the following subsections.
3.3.1 The Pretest and the Posttest
The pretest and the posttest were both grammar tests compiled by the researcher.
The content of the two tests was selected from an item pool set up by Minister of Education (Figure 1). All the items were chosen from the item pool based on the grammar focus covered from Starter to Unit 5 in iEnglish, the participants’ textbook.
The item pool was an open resource and the questions there were designed by in-service teachers and the experts of National Academy of Education Research (Figure 1). All the questions were categorized by subjects and volumes. In English, the items were compiled according to the content of language learning structure which was also known as grammar (see Figure 2).
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
31
Figure 1. The user interface of the item pool
Figure 2. The main structure of the English item pool
The pretest (Appendix B) was made up of 30 multiple choice questions and
‧
administered in both classes in order to collect the data of their initial ability on grammar before receiving formal instruction. The participants got 2 points for each correct answer. It meant that the total score ranged from 0 to 60 points. Then the two classes were randomly assigned to be the experimental group and the control group.
The test result was also utilized to divide the participants of each class into three proficiency levels: HPL, IPL and LPL. As Gregory and Chapman (2007) have suggested, more levels of readiness may be identified and adjusted, but when first starting the process of DI, three levels are complicated enough to manage.
The posttest (Appendix C) was administered in the 14th week of the semester.
Unlike the pretest, the posttest comprised three sections: 10 multiple choice questions, 10 fill-in blank questions and 10 sentence making questions which served to test whether DI catered to students with different proficiency levels. Each correct answer was worth 2 points in each section, and every mistake in fill-in blank or sentence making questions was deducted by 1 point with 2-point deduction as the maximum of each item. The result of the posttest was used to compare the performance of two groups on grammar learning after the treatment to answer the research question one.
In the meanwhile, HPL, IPL and LPL in the experimental group compared their scores with their counterparts in the control group to answer the research question two.
In view that the pre- and posttests aimed at testing grammar points taught in the 6 units, each of them contained 30 questions with 5 questions for each unit
respectively. To make sure of the content validity of the pre- and posttests, a two-way specification table of the grammar points was employed (See Table 3.3). Moreover, a pilot study was conducted prior the formal study to test the pretest and the posttest in the early September of 2013. The participants in the pilot study was one class that did not join the main study. There were a total of 27 participants in the pilot study. The
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
33
pretest and the posttest were piloted by their homeroom teacher respectively. The two tests required approximately forty minutes to complete. The test results were
evaluated for effectiveness and for validity by the researcher and two colleagues who were also the English teachers of the seventh graders.
An item analysis was performed on the 30 items of the pretest and the posttest to establish their reliability. Reliability is the consistency with which a measurement instrument yields a certain result when the entity being measured has not changed. It is an estimate of the internal consistency of the instrument (Leedy & Ormrod, 2001).
Because a test-retest reliability was not feasible in this study, Cronbach's Alpha
Reliability Coefficient was used to estimate the internal consistency of the instrument.
An acceptable range for the reliability coefficient is from .70 to .90. The value obtained in the pilot study, based on the 30-item of the pretest and posttest , was a Cronbach Alpha of .749 and .913 respectively. As a result, both the two tests were found to be reliable in the pilot study.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
34
Table 3.3
Two-way Specification Table of Grammar Points from Starter to Unit 5
Pretest 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Posttest 26 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Unit Grammar Points
Starter
Personal pronouns (I, you, he, she, it) and be V
determiners (my, your, his, her, its)
Unit 1
Question form of be verb
Wh-question
(who….)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
35 Unit 2
Wh-question (how
old….?)
Adjectives
Unit 3 Plurals (-s,-es,-ies)
Unit 4
Wh-question
(where…?)
Prepositions
Unit 5 There is/ are…
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
36