國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER THREE METHODOLOGY
This chapter starts with the aims of the study. Then, the focus is on a discussion of the instrument used for collecting data and the procedure of the experiment, followed by the methods of the data analysis.
The purpose of the current study was to investigate the effect of teachers’ indirect feedback (IDF) and direct feedback (DF) on elementary school students’ sentence writing. The study also aimed at examining their attitudes toward the implementation of IDF and DF on their English learning.
Participants
The participants in this study were two classes of sixth graders from a public primary school in Hsin Chu County, and one class was the IDF group and the other was the DF group. The IDF group was the experimental group that comprised 33 students receiving the indirect teacher feedback. Another 33 students served as a control group receiving the traditional teacher feedback which was the direct teacher feedback. English Writing Proficiency Test (EWPT) was conducted to examine
whether the two groups were at the similar language proficiency level at the beginning, and there was no significant difference in the results (t = -6.28).
During the experimental phase in 2011, all the participants had three 40-minute lessons per week, taught by the same English teacher, and were using the same
textbook. It should be noted that all the participants had English classes at school from grade one to grade five, that is, two 40-minute lessons per week and most of them had participated in different private English learning institutions after school.
51
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Instrument
This study employed two types of instrument to achieve the purpose of this study:
an English writing proficiency test (EWPT) and an English learning attitudes
questionnaire (ELAQ), the face validity and reliability of these two instruments were examined respectively as following.
English Writing Proficiency Test (EWPT)
In order to measure the effect of IDF and DF on subjects’ sentence writing accuracy, an English writing proficiency test (EWPT) (see Appendix A) was used as the pre-, post- and retention test in the study. It included 35 questions and was divided into four parts: circle the correct word in sentences, multiple choice questions, revising the error and short questions as discussed below.
The first part in EWPT was to circle the correct word in sentences. The purpose was to know if learners could apply the grammatical knowledge correctly. The
question was presented as a sentence with two optional words, for instance, How are/is Mark? Learners will read the sentence and will circle one word from the two options and the circled word should make the sentence grammatically correct. Regarding the scoring criteria, one point was given if the accurate word was circled. Zero point for inaccurate answer and no answer. The sentences were designed according to five targeted error types, such as, misspelling, preposition, noun, tense, and subject-verb agreement. Two sentences were designed for one error type to confirm if the
participants actually knew the grammatical knowledge, thus, 10 sentences were distributed and the total score was 10 points in this part.
The questions in part two were multiple choice questions. According to William (1998), students should be able to find out where the error was, then, they could try to correct the error. The purpose of this part was to know if learners were able to
52
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
( ) 1. How is the girls?
______ 1. How is the girls?
recognize the targeted error in sentences. The format of question was presented in the sample below.
The participants read each sentence and chose the error from three options. As for the criteria, one point was given for each question when the accurate option was chosen, otherwise, zero point would be provided. The questions were designed in reference to the five targeted errors, two questions for each error type; therefore, 10 questions were conducted and the total score of part two was 10 points.
As for part three, revising the error, the purpose was to assess students’ self-repair ability. This part required students to correct the error with the other word. The sample question was showed as below.
Participants need to think and revise one word from the three options; the revised word should make the sentence grammatically correct. Considering the scoring system, one point was given to each sentence when the revised word was correct. On the other hand, zero point would be given for the wrong answer or no reply, thus, 10 points were the total of this part. What is worth to know was that 10 sentences in this part were the same sentences in part two. Since the only difference between these two parts was the students’ reply. When the participants scored differently in these two parts, the score showed their self-repair abilities in different levels. For example, if student A could answer part two correctly but not in part three, this meant that he or she has the ability of recognizing the errors instead of the ability of correcting them.
(A) (B) (C)
53
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Part four was designed for short questions. Since the study focused on the sentence writing ability, the purpose of this part was to know how they improved on sentence writing. Five questions and pictures were provided as the sample question below.
Q: What does David like to do in the park?
A:________________________________
Participants read and answered with complete sentences. Two points were given on each question if the answered sentence was correct. One point was given if the sentence was partly correct. Zero point was given for three situations, including, no answer, the sentence that could not be understood or the sentence with any errors from the five targeted errors. In this part, two points per question so that the total score was 10 point.
Adding these four parts together, 10 points from each part, the total score of EWPT was 40 points. As for the number of the question, 10 questions from the first three parts and 5 questions from part four, 35 questions were conducted in total. The sample questions, number of question and score of each part were listed as table 1 as below.
54
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(A) (B) (C)
Table 1
Sample Questions of EWPT
Part Question Number of question Score
1 How are/is Mark? 10 10
2 ( ) How is the girls? 10 10
3 _____ How is the girls? 10 10
4 Where are you going? 5 10
Total 35 40
The Face Validity of EWPT. The EWPT was examined by the face validity; the purpose was to revise any questions that were unclear or hard to answer for the participants. The EWPT was assessed through the expert validity. Two English teachers from primary schools and two professors from universities evaluated the instrument; with their advice, the modification of the EWPT was made in week 1.
Regarding the advice from teachers, three advices were proposed below, first, part one, unscramble, might be too difficult to understand, such as, Mark? are is how.
The question format was revised into circling the correct word in sentences, like, How are/is Mark? Second, the sequence of parts should be arranged from easy to hard, the sequence of part one and two should be exchanged. Since part one was already revised into circling the correct word, the original sequence of these two parts was kept. Third, in part four, No. 2, the hospital picture was unclear, and thus, the revision included a red cross symbolizing the hospital as the picture below.
55
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(A)(A) (B) (C) (B) (C)
Where are you going?
_______________________________ (醫院)
In the professors’ perspective, three suggestions were noted as the following. First, in part one, No. 6, two possible answers could be responded for this item, They are to going at the zoo. The possible answers could be they are at the zoo or they are going to the zoo. Thus, the revised question was as follow: They are to/ at the zoo. Second, in part one, No. 9 and No. 10, two questions for spelling, the professor was wondering the objective of them. Since the format of two questions was not in accordance with the format of part one, the professor was wondering if these two had different objectives from the others. To deal with this concern, No. 9 was revised as we are at the beach/ baech (海灘); and No. 10 was fixed into he is in a
restaurant/restuarant (餐廳). Third, in part two, find the error. No. 9, ( )___
Peter is in the resturant. The professor suggested that spelling was scarcely to be seen in this test format, consequently, this item was revised as ( ) 9.選出對的字。
(A)palying (B)playing (C)paly. The suggestions from teachers and professors were summarized in Appendix D.
In a word, the EWPT was modified with the advice of two English teachers and two professors. In addition, the EWPT was examined by the internal consistency reliability as follows.
The Internal Consistency Reliability of EWPT. The internal consistency reliability of the EWPT was tested by Tester 2.0 and Cronbach’s α coefficient respectively. The purpose was to revise if there were any items with low internal consistency reliability.
After conducting the EWPT, the data was collected and tested in week 3 of the first
56
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
semester. The internal consistency reliability of the EWPT from part one to four, were 0.81, 0.78, 0.87 and 0.8 respectively and the internal consistency reliability of the EWPT was 0.82.
English Learning Attitudes Questionnaire (ELAQ)
An English learning attitude questionnaire, adapted from Carreira’s (2006)
Motivation and Attitudes toward Learning English Scale for Children (MALESC), was used to investigate the learners’ attitudes after the experiment. In order to make the questionnaire more suitable for the context of primary schools, the modification had been made with reference to the English Writing Attitude Questionnaire of Haung’s (2009) and the Vocabulary Learning Strategies Questionnaire of Chang’s (2011) for the current study.
The ELAQ (see Appendix B) included 2 parts: background information and the attitude towards English learning. In the first part, there were 2 items used to find out the students’ English learning experience. In the second part, 21 items were included and divided into 7 sections (as shown in Appendix C): interest in foreign countries (3 items), motivation of future school or employment (3 items), parents’ encouragement (3 items), instrumental motivation (3 items), anxiety (3 items), attitudes about the treatment (4 items) and self-evaluation after the treatment (2 items).
According to Dörnyei (2003), an even number of response options were created to avoid the participants from choosing the neutral category without expressing their own opinions. Therefore, this research applied a four-point Likert scale to the response options (i.e. strongly agree = 4, agree = 3, disagree = 2, strongly disagree = 1).
The Face Validity of ELAQ. The ELAQ was also examined by the face validity;
the purpose was to revise any questions that were difficult to understand by the participants. The ELAQ was evaluated by two English teachers from primary schools
57
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
and two professors from universities for the expert validity; the modification of ELAQ had been made with the experts’ opinions (see Appendix E) and the English version of ELAQ was shown as Appendix C.
According to the teachers’ advice, five points were modified. First, in part two, section one, No. 2, I want to know many foreign friends. The description was revised as follow, if it is possible, I would like to write e-mail and contact with foreign friends.
Second, in part two, section one, No. 3, I want to write the complete and correct English sentences since I want to contact foreign friends via written words. The wording was revised as: I want to write the complete and correct English sentences since I want to communicate with foreign friends with written words. Third, in part two, section four, No. 1, English class at school is so fun and it makes me to be interested while doing the work book. Since whether the English class is fun or not, it might not be the cause to affect the attitude of doing work books. The description was revised as: I like doing the work book. Fourth, in part two, section seven, No. 3, the way that teachers correct the work book has an influence on me about the motivation of English learning. The wording was simplified for students to understand as follow:
the way that teachers correct the work book affects my English learning. Finally, all the question numbers were suggested to be changed into 1 to 22 to avoid confusion.
When it came to the suggestions from professors, three concerns in the ELAQ were pointed out, first, from the holistic aspect, the ELAQ emphasized on the
workbook correction instead of the teaching and exercise in class, however, since none of teaching and exercise was special designed for the treatment, the related questions were not included in the ELAQ. Second, in part one, No. 1: Have you ever had any English classes outside of the school? One example was added in the description to make clearer, thus, the revision was as follow: Have you ever had any English classes
58
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
outside of the school (such as in the cram schools)? Secondly, in part two, the original section one was the interest in foreign countries and section three was the extrinsic motivation; both of them were considered as the extrinsic motivation, so that the topic was revised as, section one, the interest in foreign countries. Section two was revised as motivation of future school or employment. Thirdly, in part two, section two, No. 2:
I learn English because it is important while I grow up. The question was suggested to revise as: I learn English because I can use it while traveling abroad.
In short, after the ELAQ was modified, the ELAQ was examined by the internal consistency reliability as below.
The Internal Consistency Reliability of ELAQ. The internal consistency reliability of ELAQ was examined by using Cronbach’s α coefficient and the data was shown from section one to seven as follow: section one =.768; section two =.801; section three= .745; section four =.75; section five =.773; section six =.812; section seven
= .281 and Cronbach’s α coefficient of the total was .892. Since Cronbach’s α coefficient in section seven was lower than the average, one problematic question, question 22, had been deleted. Thus, the revised ELAQ had 21 questions in total.
59
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Procedures
Eight stages were included in the research procedure and each stage was discussed (as shows in Figure 1 below).
Figure 1 Flow Chart of the Study 14 weeks
Pre-test: EWPT
To Test the Face Validity of EWPT & ELAQ
Implementation & revision of EWPT & ELAQ Implementation & revision of Correction Guide, IDF& DF
Retention Test
Data Analysis
Control Group:
Direct Feedback Experimental Group:
Indirect Feedback
ELAQ & Post-test: EWPT
60