CHAPTER 3 METHODOLOGY
3.3 Procedure
3.3.1 Pilot Study
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
3.3.1 Pilot study
A pilot study was carried out from March to June, 2011to three different classes (the 10th, 11st, and 12nd, respectively), not the target classes in the formal study, to ensure whether the pre-test and post-test are suitable for the participants.
Since the researcher was a substitute teacher whose schedule for the formal study was indecisive, he had to squeeze the time from the formal teaching schedule was precious for the pilot study and made the best use of every chance to ensure that the formal study could be put into practice at the same time. In this study, the participants of 40 tenth-graders, 47 eleventh-graders, and 42 twelfth-graders, were invited to complete Questionnaire I－Reading Strategy Questionnaire before &
after RSI and Questionnaire II－Students’ Responses to RSI to check whether the content and the statement need any revision.
In addition, two different versions of reading comprehension tests of 10 items each, are used for both the pre-test and post-test to check the feasibility of the test and appropriateness of the process, and any possible misspellings. Based on the test-takers’ reactions and the results of the pilot study, the researcher made a great deal of changes and completed some necessary adjustments for the main study.
They are as follows:
First of all, the participants were reduced to the twelfth-grade senior high school students only. The reason was that the participants from three grades covered too large population, which complicated the experiment. Simple focus on one grade might simplify the study and made researcher concentrate on less variables. Besides, since the original participants are from different grades, the teaching materials and textbooks are separated in accordance with their individual proficiency levels, which cause more complicated situation. Therefore, the formal experimental
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
participants were confined to the 12th-grade students of the same level.
Second, the entire background information survey was eliminated, since it was irrelevant to our research questions. The statements of the background information survey were too much concerned about the participants’ motivation and learning attitudes, e.g., “Do you have any experience of traveling, studying and living in English-speaking foreign country?” and “Except for school textbooks and magazine subscribed by the school authority, how do you train yourself in reading?”
Third, in order not to be difficult for the participants, the words and phrases in the reading strategy questionnaire (questionnaire I) was made as easy as possible.
Last but not least, the number of the original reading comprehension test items in the pilot study were so small (just 10 items), posing a serious threat to validity and reliability. Thus, they were added to 25 ones for the pre- and post-test in the major study. In addition, the reading comprehension test items in the pre-test were different from those in the post-test in the pilot study. With a view to keeping a consistency in checking out the change of the reading strategy use before & after the instruction, this study adopted just one single set of reading test in both the pre-and post-test.
3.3.2 The Main Study
The main study lasted for 18 weeks from September, 2011 to December, 2011. One class period a week was spent on the instruction. The study falls into three stages. At the first stage, a pre-test, a Group Embedded Figure Test, a GEPT reading comprehension test, and Questionnaire I (Reading Strategy Questionnaire before RSI) were conducted. At the second stage, FD/FI learning style instruction,
‧
reading strategy instruction (RSI), and a statistics of the question types in reading comprehension tests of JCEE, RE, GSAT, and DRT 1996~2011 were executed. At the third stage, Questionnaire I (Reading Strategy Questionnaire after RSI), a post-test, and Questionnaire II on students’ responses to RSI were carried out. In order not to affect the participants’ performance, they were not informed of the research experiment. The pilot study and every stage in the main study were scheduled (see Table 3.10).
Table 3.10 A Schedule for the Pilot Study and the Main Study
Stage Week Content Duration
(min) The Pilot Study
Doing a pilot study
Preparing the reading comprehension test for pre-test and post-test
Designing Questionnaire I before & after RSI
Designing Questionnaire II on participants’
response to RSI
Purchasing GEPT test kit (Intermediate level for senior high school students)
One semester
The Main Study The
First Stage
1 Administering the pre-test (5 passages, 25
items) 40
2
Administering Group Embedded Figure Test-to group the participants into the FD/FI Group
Administering GEPT reading comprehension test to examine the same proficiency level initially
Administering Questionnaire I (Reading Strategy Questionnaire before RSI) (40 items)
Introducing FD/FI learning styles to inform the participants of their learning styles and
knowledge of how to make use of their learning advantages and possible disadvantages
Introducing a statistics of the question types in reading comprehension tests of JCEE, RE, GSAT, and DRT 1996~2011
20
10
20
4~15 Implementing Reading Strategy Instruction (RSI)
16 Administering Questionnaire I (Reading
Strategy Questionnaire after RSI (40 items) 20
17 Administering the post-test 40
18 Administering Questionnaire II on participants’
responses to RSI (14 items) 25
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
To elaborate on the consecutive steps, see the following explications.
Step 1: Pre-test before RSI for both FD and FI Groups
The pre-test was carried out for the two classes the same week. A test of 25 items in the reading comprehension tests was administered. It took 45 minutes.
Step 2: GEFT (Group Embedded Figure Test)
The Group Embedded Figure Test (GEFT) was administered to determine whether the learner’s preferred learning style was field dependent (FD) or field independent (FI). The test spent 12 minutes.
Step 3: GEPT (General English Proficiency Test)
The intermediate-level General English Proficiency Test was administered as a homogeneous test to see whether the FD/ FI groups were at the same proficiency level initially.
Step 4: Questionnaire I－Reading Strategy Questionnaire before RSI
Questionnaire I (Reading Strategy Questionnaire) was conducted to figure out the frequency the FD/FI groups using these reading strategies before our treatment. The Questionnaire took 20 minutes.
Step 5: Learning Style Instruction for FD/FI
The participants were informed of the differences between the FD/FI learning styles and learning suggestions (see Table 2.1 and Part II in Appendix J).
Step 6: Reading Strategy Instruction (RSI)
The Reading Strategy Instruction lasted for four months, one hour each week.
It was composed of two parts: reading strategy instruction and reading materials. In view of the reading strategy instruction, all the question statements in the Questionnaire I (RSQ) were reviewed, attracting students to the strategies of the organization and content in a text (the relations between paragraphs), the strategies
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
of the organization and content in a paragraph (the relations between sentences), and the strategies of the author’s writing crafts. In other words, by taking advantage of these reading strategies, inclusive of global and local strategies, learners could be aware of how to get the whole picture of the passage (one of the characteristics of FD learning styles) and simultaneously, comprehend details (one of the characteristics of the FI learning styles) in their reading texts.
With regard to the reading materials, a detailed statistics of 1996~2011 Reading Comprehension Tests of JCEE, RE, GSAT, and DRT was introduced to arouse learners’ interest and attract their attention, including the number of articles, test items, question types, and word number (see Appendix A). Moreover, the reading materials are direct from the reading comprehension tests from1996 to 2011 with diversified question types to illustrate the reading strategies above-mentioned.
Together with the explicit demonstration of a lot of articles and examples, students were taught how to raise their awareness and familiarize themselves with these reading strategies available for them through sufficient practices and exercises. As a result, they would get and comprehend the gists and essences of these underlying reading comprehension constructs, and further, make most use of these accessible strategies to identify the main idea, title, and source, recognize style/tone, draw implications and inference, find specific details, and determine bold word and reference in the context. Instead of being afraid of and averse to taking reading comprehension tests, learners might be gradually fond of and even enjoy reading.
Step 7: Questionnaire I－Reading Strategy Questionnaire after RSI
After four-month reading strategy instruction, Questionnaire I was conducted again to examine to what extent both FD and FI groups changed their reading strategy use. It also took 20 minutes.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Step 8: Post-test after RSI for the FD & FI Groups
The post-test was administered to the two classes by the researcher, one week after the final instruction in December. The test was intended to measure the effects of the RSI on the reading comprehension. The test also took 45 minutes. The post-test in December, 2011, was administered in the same way as the pre-test, and then, a comparison in performance between the pre-and post-test was made.
Step 9 : Questionnaire II－Student’s Responses to RSI
This questionnaire was aimed to obtain the participants’ responses to the reading strategy instruction. This questionnaire was administered to the FD/FI learners to explore the fact that the participants with different learning styles might manifest different degrees of awareness after the instruction. Students were requested to complete 14 questions in 25 minutes. Moreover, since the participants were from two different classes, the data were collected separately. All the data were gathered within a week. The results of questionnaire II were further analyzed and discussed afterwards.
3.4 Data Analysis
The research results were obtained mainly through the pre-and post-test, Questionnaire I (Reading Strategy Questionnaire) before and after RSI, and Questionnaire II (Students’ Responses to Reading Strategy Instruction and FD/FI Learning Style). The close-ended question data collected from the Questionnaire I before and after RSI were computed with SPSS 12.0 (Statistical Package for the Social Sciences) statistical software in Chinese version. The criterion level of .05 was adopted to determine a significant difference. In addition, the answers to the open-ended questions in the Questionnaire II were summarized to further clarify the
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
authenticity of the close-ended questions.
To answer RQ 1, paired samples- and independent sample t-test were used in the pre-and post-test to evaluate the effectiveness of the reading strategy instruction for the FD and FI groups. That is, the results explain if there was any significant effect after RSI on each group.
To answer RQ2, descriptive statistics, such as means, standard deviations, and t-values was adopted to compute how often different reading strategies were used, and find out the most frequently used global and local reading strategies.
Paired samples t-tests were also used for the Questionnaire I (Reading Strategy Questionnaire) to see if there was any significant difference between the FD and FI groups in the change of global and local reading strategy use before and after RSI.
To answer RQ3, the Excel of the Microsoft was used in the pre-and post-test to compare the percentage of the numbers of the correct answers made by the FD and FI groups in each question type.
To answer RQ4, the Excel of the Microsoft was used to compute the answers to the closed-ended questions in Questionnaire II. Then, the most beneficial and the most difficult global and local reading strategies were listed in this part. Also, the participants’ responses to the open-ended questions on the teaching and reading materials were further discussed.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Chapter 4 Results
In this chapter, the collected data are analyzed and the results are presented in the sequence of the four research questions raised in Chapter Two. Four sections are elaborated on (1) the results in response to Research Question 1; (2) the results in response to Research Question 2; (3) the results in response to Research Question 3, and (4) the results in response to Research Question 4.
4.1 Research Question 1: How do FD/FI Learners perform in English reading comprehension before and after instruction? And which group of learner benefits more from the reading strategy instruction?
4.1.1 How do FD/FI Learners perform in English reading comprehension before and after instruction?
To examine whether the FD group and FI group performed significantly differently in reading comprehension test before & after RSI, a paired samples t-test was conducted. The results of the paired samples t-test are represented in Table 4.1. There were significant differences between the FD group (p = .012＜.05) and the FI group (p = .000＜.05) in the pre-test and post-test.
Table 4.1 Results of the Paired Samples T-test in the Pre-and Post-test with FD and FI Groups
Group Test M SD N t df Sig.
Paired 1 FD pre-test -
FD post-test -6.50000 13.84709 32 -2.655 31 .012*
Paired 2 FI pre-test -
FI post-test -9.48837 13.35303 43 -4.660 42 .000**
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
4.1.2 Which group of learner benefits more from the reading strategy instruction?
To examine which group of learner benefits more from the reading strategy instruction, an independent sample t-test was conducted. Based on the results listed in Table 4.2, there were significant differences between the FD and FI groups in pre-test (p = .000,＜.05) and post-test (p = .000,＜.05). The mean score of the pre-test in the FD group was 61.63 while that of the pre-test in the FI group was 58.42. Furthermore, the mean score of the post-test in the FD group was 68.13 while that of the post-test in the FI group was 67.91. The results showed that both groups (the FD group and the FI group) made much progress in the reading test after the 18-week reading strategy instruction (RSI). In comparison, the FD group performed better than the FI group before and after RSI; however, the FI group (+9.49) improved better than the FD group (+6.50).
Table 4.2 Results of the Independent Sample T-test in the Pre-and Post-test with FD and FI groups
Group Test M SD t df Sig.
FD Pre-test 61.63 17.42 20.016 31 .000**
FI Pre-test 58.42 13.69 27.973 42 .000**
FD Post-test 68.13 15.89 24.258 31 .000**
FI Post-test 67.91 12.27 36.302 42 .000**
Note.
1.FD = Field Dependent group, FI = Field Independent group 2.*Significant at p＜.05 (two-tailed)
A Summary of Responses to Research Question 1
Based on the results in Table 4.1, there were significant differences between the FD group and the FI group in the pre-and post-test. Based on the results in Table 4.2, the mean scores of FD in the pre-test and post-test were 61.63 and 68.13, and those of FI group in the pre-test and post-test were 58.42 and 67.91, respectively. So, learners in the FI group (+ 9.49) benefits more than those in the FD group (+ 6.50) after RSI.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
4.2 Research Question 2: How frequently do FD/FI Learners use different reading strategies before and after instruction?
Table 4.3, Table 4.4, Table 4.5, and Table 4.6 present the results of the FD and FI groups’ global and local reading strategy use in the Reading Strategy Questionnaire (Questionnaire I) before and after RSI.
4.2.1 Frequency of the FD group’s Global Reading Strategy Use before & after RSI To compute the FD group’s responses to the global reading strategies (Q1-Q9, 34-40) before & after RSI, paired samples t-tests were conducted. Based on the results listed in Table 4.3, there were significant differences found in Item1 “the opening paragraph, the main paragraph and the closing paragraph”(p = .011＜.05), Item 4
“every sentence in the first paragraph, especially in the thesis statement” (p = .001
＜.05), Item 5 “every sentence in the last paragraph” (p = .003＜.05), Item 34 “how to arouse the reader’s interest”(p = .013＜.05) and Item 36 “how to conclude the text”(p
= .037＜.05).
Second, the most frequently used global reading strategies (over total mean score =3.19, see Table 4.3) before RSI by the FD group were Item 1” the opening paragraph, the main paragraph and the closing paragraph” (M = 3.47), Item 3 “title, subtitle, figure, and table” (M = 3.47), Item 4“ every sentence in the first paragraph, especially in the thesis statement” (M = 3.34), Item 5 “every sentence in the last paragraph” (M = 3.44), Item 7 “the possible development patterns of the supporting sentences (paragraph development) ” (M = 3.19), Item 8 “the cohesion of the preceding or the next sentences ”(M = 3.53), Item 9 “the closing sentence echoes with the topic sentence” (M = 3.44), Item 37 “get conclusion, inspiration, implication, or moral lesson” (M = 3.38), and Item 39 “style/tone” (M = 3.31).
The frequently used global reading strategies (overall mean score = 3.52, see
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Table 4.3) by the FD group after RSI were Item 1” the opening paragraph, the main paragraph and the closing paragraph” (M = 4.0), Item 3“title, subtitle, figure, and table” (M = 3.69), Item 4 “every sentence in the first paragraph, especially in the thesis statement” (M = 4.34), and Item 5 “every sentence in the last paragraph” (M = 4.25), Item 37 “get conclusion, inspiration, implication, or moral lesson ”(M = 3.59).
Apparently, Items 1, 3, 4, 5, and 37 were the frequently used global reading strategies by the FD group both before & after RSI (see bold figures in Table 4.3).
Table 4.3 Frequency of the FD group’s Global Reading Strategy Use before &
after RSI
No. Global Reading Strategy Items
FD before RSI
FD after RSI
Mean SD Mean SD t df Sig
1 the opening paragraph, the main paragraph and the closing paragraph
3.47 0.98 4.0 0.80 -2.715 31 .011*
2 the coherence of paragraphs 2.88 0.91 3.19 0.82 -1.621 31 .115
3 title, subtitle, figure, and table 3.47 1.02 3.69 0.90 -1.126 31 .269
4 every sentence in the first
paragraph, especially in the thesis statement
3.34 1.31 4.34 0.79 -3.712 31 .001*
5 every sentence in the last paragraph 3.44 1.16 4.25 0.80 -3.227 31 .003*
6 the topic sentences and the closing
sentences in the main paragraph 3.09 1.17 3.34 1.04 -1.245 31 .222
7 the possible development patterns of the supporting sentences (paragraph development)
3.19 1.06 3.31 0.93 -.584 31 .564
8 the cohesion of the preceding or the
next sentences 3.53 1.05 3.41 1.01 .643 31 .525 9 the closing sentence echoes with
the topic sentence 3.44 1.13 3.5 1.16 -.239 31 .813 34 how to arouse reader’s interest 2.72 1.30 3.25 1.22 -2.647 31 .013*
35 the specific details and substantial
evidence 2.94 1.13 3.19 1.12 -1.114 31 .274
36 how to conclude the text 2.72 1.22 3.22 1.24 -2.184 31 .037*
37 get conclusion, inspiration,
implication, or moral lesson 3.38 1.21 3.59 1.19 -.865 31 .394 38 the writing purpose 3.06 1.22 3.44 1.34 -1.482 31 .148
39 style/tone 3.31 1.12 3.5 1.11 -.882 31 .385 40 the possible sources of the text 3.13 0.94 3.16 0.99 -.144 31 .887
Total Mean Score 3.19 3.52
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Note.
1. FD = Field Dependent group 2. N = 32 for the FD group
3. *Significant at p＜.05 (two-tailed)
4.2.2 Frequency of the FD group’s Local Reading Strategy Use before & after RSI To compute the FD group’s responses to the local reading strategies (Q10-33) before & after RSI, paired samples t-tests were conducted. Based on the results listed in Table 4.4, there were significant differences found in Item 17 “comparison -contrast” (p = .020＜.05), Item 19 “the positive degree, the comparative degree, and the superlative degree” (p = .048＜.05), Item 25 “illustrative, distractive, and explicative” (p = .024＜.05), and Item 28 “appositive” (p = .030＜.05).
Second, the most frequently used local reading strategies by the FD group before RSI (overall mean score = 3.42, see Table 4.4) were Item 10“ time order”
(M = 3.84), Item 11“ who, when, where, why, how, which, and what” (M = 3.78), Item 14 “cause & effect” (M = 3.75), Item 15” condition” (M = 3.53), Item 16“ aiming” (M = 3.47), Item 18 “subtractive/ exception, or particularity” (M = 3.50), Item 22 “exemplifying” (M = 4.06), Item 23 “listing order” (M =4.06), Item 27 “number or date in Arabic numeral or in English” (M = 4.09) and Item 29
“stylistic” (M = 3.63) .
In addition, the most frequently used local reading strategies by the FD group after RSI (overall mean score = 3.65, see Table 4.4) were Item 10 “time order” (M
= 4.03), Item 11 “who, when, where, why, how, which, and what” (M = 3.84), Item 14 “cause & effect” (M = 3.92), Item 16 “aiming” (M = 3.72), Item 17
“comparison-contrast” (M = 3.88), Item 18 “subtractive/ exception, or particularity”
(M = 3.91), Item 22 “exemplifying” (M = 4.13), Item 23 “listing order” (M = 4.09), Item 25 “illustrative, distractive, and explicative” (M = 3.72), Item 26 “logical order/ the order of the importance” (M = 4.16), Item 27 “number or date in Arabic
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
numeral or in English” (M = 4.22), and Item 29 “stylistic” (M = 4.00).
Apparently, Items 10, 11, 14, 16, 18, 22, 23, 27 and 29 were the most frequently used local reading strategies by the FD group before and after RSI. (see bold figures in Table 4.4)
Table 4.4 Frequency of the FD group’s Local Reading Strategy Use before &
after RSI
No. Local Reading Strategy Items
FD Before RSI
FD after RSI
Mean SD Mean SD t df Sig.
10 time order 3.84 1.08 4.03 1.03 -.641 31 .526
11 who, when, where, why, how,
which, and what 3.78 0.98 3.84 0.92 -.291 31 .773 12 spatial order 3.03 1.00 3.22 1.04 -.757 31 .455
13 additive 3.28 0.96 3.38 1.01 -.487 31 .629
14 cause & effect 3.75 0.95 3.94 0.98 -.882 31 .385
15 condition 3.53 0.88 3.47 1.02 .360 31 .721
16 aiming 3.47 0.88 3.72 1.25 -1.187 31 .244
17 comparison-contrast 3.41 0.91 3.88 0.83 -2.462 31 .020*
18 subtractive/ exception, or
particularity 3.50 0.92 3.91 0.89 .254 31 .801 19 the positive degree,
the comparative degree, and the superlative
3.22 1.18 3.53 0.98 -2.058 31 .048*
20 concessive 3.25 1.16 3.56 1.13 -1.186 31 .245 21 defining 3.16 1.14 3.59 1.07 -1.951 31 .060
22 exemplifying 4.06 0.88 4.13 0.71 -.329 31 .745
22 exemplifying 4.06 0.88 4.13 0.71 -.329 31 .745