This chapter reports the results of the study in three sections. First, the demographic information of the participants and the descriptive statistics of the variables are shown. Second, the statistical results of the ANOVAs within and between the variables under investigation are presented. Third, the analysis on the written responses concerning how the participants regarded the importance of background knowledge and English proficiency, and how they perceived the contributions of pre-reading materials in different languages are detailed.
Descriptive Data
In this section, the following data collected in the study will be presented: (1) demographic information of the participants; (2) descriptive statistics of the
participants‘ performance on the content knowledge test and the GEPT reading test;
(3) descriptive statistics of participants‘ performances on domain-specific reading comprehension questions; and (4) descriptive statistics based on the summary counts of the participants‘ reported perception toward the importance of background
knowledge and English proficiency, and their preferences over the pre-reading treatments in different languages.
Demographic Information of the Participants
All 319 participants were required to complete three testing measurements followed by one survey questionnaire. Nonetheless, as these instruments were administered in a span of two weeks, a few participants were absent or late for the
assigned class when the instruments were administered, which led to 19 cases of incomplete data. The final sample consisted of data gathered from 300 participants who completed all the tasks required in the study.
The first part of the survey questionnaire was background information, which asked the participants for their gender, age, and year of study (freshman, sophomore, junior or senior) in their major field. As shown in Table 12, the complete data set contained 300 participants, with 55 freshmen (18.3%), 79 sophomores (26.3%), 95 juniors (31.7%), and 71 seniors (23.7%). Among the data collected, 68 of them were males (23.7%) and 232 were females (77.3%). In Taiwan, the population of English majors consists of more females than males. The mean age of the total participants was 20.32.
Table 12. Demographics of the Participants
Year of Study N % Accumulated %
Freshmen 55 18.29 18.29
Sophomores 79 26.31 44.60
Juniors 95 31.67 76.27
Seniors 71 23.73 100.00
Total 300 100.00 100.00
Table 13 reports where the 300 participants came from, either the public (A and B) or private universities (C, D, E, and F) in Taiwan and the participants‘ year of study. The performances based on individual class for English reading proficiency and content knowledge were also calculated and shown below. Generally speaking, juniors and seniors performed better than freshmen and sophomores in terms of English proficiency and content knowledge tests.
Table 13. Performance of English Proficiency (Eng)/Content Knowledge (Cont) Tests Based on Individual Class at Six Universities
Universities Year of Study N M (Eng) M(Cont)
A (Public) Seniors 24 29.32 16.45
Juniors 27 33.42 16.32
B(Public) Sophomores 28 23.50 15.30
C(Private) Freshmen 17 16.09 10.22
Freshmen 20 16.21 10.72
Sophomores 31 22.82 13.17
D (Private) Juniors 23 31.18 14.93
Juniors 27 32.04 14.60
Juniors 18 28.14 16.81
E (Private) Freshmen 18 16.37 12.42
F (Private) Seniors 38 33.22 15.76
Seniors 9 30.13 17.85
Sophomores 20 20.32 15.71
Descriptive Statistics of Participants’ Performance on the Measurement Instruments As indicated in Chapter Three, from the quantitative perspective, the present study employed the analysis of variance (ANOVA) to test whether the performances of the participants were all alike. The independent variables included the different pre-reading treatments, the participants‘ content knowledge levels, and their L2 reading proficiency; the dependent variable was the participants‘ performances on domain-specific academic reading. The variables were measured on numerical scales, and the statistical procedures were conducted to investigate their relationships. Since
ANOVAs were adopted to analyze such data, the scores of the independent variable of the content knowledge and English reading proficiency levels were used to divide participants into higher and the lower performance groups.
These two independent variables were categorized into the dichotomous variables by splitting the scale. In this study, the split was set based on standard deviation. The participants who performed one standard deviation above or below the mean were put into the higher or the lower performance groups respectively. After such split, these two independent variables were treated as the categorical variables and statistical tests were carried out to determine whether there was a significant difference in the mean reading comprehension scores of the academic domain-specific article for the higher/lower group represented by the dichotomized independent
variables.
The researcher used quantitative and qualitative data together to obtain a fuller picture of the sampling population. For research questions 1 to 3, the descriptive statistics of the independent variables were shown. For research question 4, the frequency for data items gathered from the survey were reported in the section of Analysis of the Survey Questionnaire Data before the illustrations of examples on written responses were provided.
Content Knowledge Levels
The content knowledge test was designed to measure the participants‘ levels of knowledge in the domain-specific field. The participants‘ performances in the test suggested their understanding in the subject of Political Science. In Table 13, among the 300 cases, the lower content knowledge level group (N =46, M = 10.07, SD =2.70) performed one standard deviation (SD = 2.86) below the mean of the total participants (M = 14.90), with the bottom performer obtaining 2 points out of the possible 20, and
the top performer(s) obtaining 12 out of 20 (range: 2-12). In contrast, the higher content knowledge level group (N =47, M = 18.67, SD = 0.62) consisted of the participants who performed one standard deviation above the mean, with the top performer(s) obtaining the perfect score, 20 out of 20, and the bottom performer(s) scoring 18 out of 20 (range: 18-20). When the higher-and-lower content knowledge groups were combined, together these two groups represented 30% of the total participants, as represented in Table 14 and Figure 3.
Table 14. Results of the Content Knowledge Test
Note: The maximum possible score for content knowledge test is 20.
Figure 3 demonstrates the mean scores for political content knowledge test of the groups. The figure indicates where the performances of participants in each group fell in the spectrum. It also shows that participants‘ scores scattered across the dimensions.
In this test, even though English and political science seem to be two distinct
disciplines, the test results manifested that some English majors were equipped with the knowledge of political science as they were able to obtain a high score on the test in this domain.
Figure 3. Group differences in political knowledge levels L2 Reading Proficiency
A reading section of GEPT at High-Intermediate Level was adopted in the present study. The participants‘ scores demonstrated their general English reading proficiency. Of all the participants, those who scored one standard deviation (SD = 9.57) below the mean (M = 25.30) were selected and grouped as the lower proficiency group (N =72, M = 12.74, SD =1.63). Among them, the lowest performer(s)‘ score was only 10 points out of 45, while the highest was 15 points out of 45 (range: 10-15).
In contrast, those who scored one standard deviation above the mean were selected and grouped as the higher proficiency group (N =69, M = 37.70, SD =2.26). The highest score in this group was an almost perfect score, 44 points out of the possible 45, while the lowest score was 35 (range: 35-44). When lower and higher performers were combined, they represented 47% of the total participants (See Table 15).
Table 15. Results of the GEPT Reading Test Grouping
Note: The maximum possible score for reading proficiency test is 45
19
Figure 4 highlights the group mean for High/Low English proficiency levels.
This figure indicates a sharp difference between the higher and the lower English reading proficiency groups. All participants were English majors, their English reading ability was at varying levels.
Figure 4. Group differences in English reading proficiency
Performances on Domain-Specific Comprehension Questions
Different types of comprehension questions were developed in the
domain-specific academic reading comprehension test for participants to answer.
These questions can prompt responses ranging from simple information identification to abstract processes of applying, synthesizing, and evaluating information. The questions include the lower-order literal questions, and the higher-order inferential and evaluative questions. Table 16 demonstrates the mean scores of three types of questions among three groups of participants. It was clearly shown that for literal questions, the three groups of participants performed quite similarly. The differences between control and treatment groups were on evaluative and especially, on
referential questions.
Higher English Group Lower English Group All Participants
Mean Highest Score Lowest Score
score
Table 16. Mean Scores of Different Question Types in Three Groups of Participants
Literal Evaluative Inferential
Q1 Q2 Q3 Q4 Q5 Q6
Control Group(N=96) 0.54 0.56 0.37 0.24 0.21 0.12 L1 Support Group(N=101) 0.55 0.78 0.67 0.52 0.51 0.40 L2 Support Group (N=103) 0.51 0.81 0.57 0.48 0.59 0.37 Note: The maximum possible score for each question is 2.
Results of the ANOVAs
To answer the three research questions, three ANOVAs were performed to examine the effect of different factors (i.e., treatments, content knowledge levels, and English reading proficiency) on the scores of domain-specific reading comprehension.
One one-way and two two-way ANOVAs were performed to decompose the variance of the dependent variable (domain-specific scores) that could be explained by the independent variables. For the present study, it is important to determine which independent variable has a significant effect on the domain-specific reading comprehension.
The first research question addressed the effects of pre-reading treatments to explore whether there was a difference between the two language inputs. The descriptive statistics of the participants‘ performances were shown on Table 17. The table seems to indicate that the treatment effects between L1 and L2 is small. The performance of each group and the difference of mean between/among the three groups can be seen in Figure 5.
Table 17. Performance of Domain-specific Reading for Participants in
Figure 5. Differences of means among three pre-reading treatment Groups
A one-way ANOVA was then performed. The between-subject ANOVA was conducted to compare the effect of pre-reading treatments on domain-specific reading comprehension score in three conditions: L1 treatment, L2 treatment, and no
treatment. The results showed that there was a significant effect of treatment on domain specific reading at the p<.05 level for the three conditions, F (2, 297) = 15.62, p = .000, p2
= .095 (See Table 18).
Table 18. Results of the One-Way ANOVA Test (Pre-reading Treatment)
Source df SS MS F Sig. p2
As a significant effect for the overall ANOVA was found, a post hoc test using the Bonferroni procedure was computed to compare each of the conditions to every other condition, i.e., L1 treatment, L2 treatment, and no treatment. The post hoc
comparisons indicated that the mean score for L1 treatment condition (M = 3.49, SD = 2.12) was significantly different from the no treatment condition (M = 2.06, SD = 1.74), which revealed that the mean score for control group was significantly lower than L1 treatment group (p = .000).
The mean score for the L2 treatment (M = 3.37, SD = 2.05) was also
significantly different from the control group (M = 2.06, SD = 1.74). The post-hoc test revealed that the mean score for the control group was significantly lower than L2 treatment group (p = .000). However, L1 treatment (M = 3.49, SD = 2.12) did not significantly differ from L2 treatment (M = 3.37, SD = 2.05). In other words, there was no statistically significant difference between L1 and L2 pre-reading treatment groups, p = .909.
For the second research question, the researcher aimed at assessing the effects of content knowledge and treatments, and whether there was an interaction between content knowledge levels and pre-reading treatments on the performance of domain-specific reading amongst EFL university students. The treatments (control/L1/L2) and content levels (high/low) were independent variables, and comprehension score of domain-specific reading was dependent variable. The descriptive statistics on their domain-specific reading performance were shown on Table 19 and Figure 6. The higher content knowledge group of participants seemed to perform better than lower content knowledge ones.
Table 19. Performance of Domain-specific Reading for H/L Content Knowledge Participants
Treatment
Content
High/Low M S.D N
Control Low 1.92 2.37 13
High 2.69 1.88 16
Total 2.34 2.11 29
L1 Low 2.17 2.37 15
High 4.07 2.18 21
Total 3.28 2.42 36
L2 Low 1.86 1.42 18
High 4.65 1.49 10
Total 2.85 1.97 28
Total Low 1.98 2.01 46
High 3.72 2.06 47
Figure 6. Differences of means between H/L content knowledge groups Figure 6 above indicates that for the higher content knowledge groups, L2
L2 L1
Control
5.00
4.00
3.00
2.00
Estimated Marginal Means
High Low Content H/L
treatment might help participants in this group to perform slightly better. As for the lower content knowledge group, the participants who received L1 treatment
performed slightly better. A possible difference appeared between the participants at different content knowledge levels.
The two-way ANOVA results showed that the effect of content knowledge on domain specific reading was significant, F (1, 87) = 18.21, p = .000p2
= .17. From the result of pair-wise comparison, the domain-specific reading scores were different for High/Low content knowledge participants. Higher content knowledge level participants performed significantly better (M = 3.72, SD = 2.06) than lower content knowledge participants (M = 1.98, SD = 2.01). See results from Table 19 and Table 20.
Table 20. Results of the Two-Way ANOVA Test (Content Levels x Treatments)
Source df SS MS F Sig. p2
The output of the ANOVA also showed that there was no evidence of significant interaction effects, F (2, 87) =1.76, p=0.178, p2
= .04. Therefore, we cannot conclude that there was an interaction between pre-reading treatments and the levels of content knowledge. Finally, the test for the main effect of treatment F (2, 87) =1.886, p=0.518,
p2
= .04, indicated that there was not enough evidence to conclude that there was a significant treatment effect, when only high and low content knowledge participants were considered (93 students in total).
The focus of the third research question was on the effects of English reading proficiency and pre-reading treatments, and whether there was an interaction between English reading proficiency levels and pre-reading treatments on the performance of domain-specific reading comprehension amongst EFL university students. English proficiency (high/low) and treatments (control/L1/L2) were independent variables and comprehension score of domain-specific reading was dependent variable. The
descriptive statistics on Table 21 indicates the domain-specific reading
comprehension performances based on their High/Low English proficiency levels and the treatments that they received. English proficiency appears to influence the
participants‘ domain-specific reading, and probably different treatments, too. Figure 7 shows that higher English ability participants might have a tendency to perform better when they were given the L2 treatment. But the conclusion had to be drawn carefully, as the ANOVA results below show the interaction effects were not significant.
Table 21. Performance of Domain-specific Reading for H/L English Proficiency Participants
Treatment Eng High/Low M S.D N
Control Low 0.68 1.20 28
High 3.15 1.53 24
Total 1.82 1.83 52
L1 Low 1.83 1.10 20
High 4.23 1.56 22
Total 3.08 1.81 42
L2 Low 1.96 1.65 24
High 4.91 1.84 23
Total 3.40 2.28 47
Total Low 1.42 1.45 72
High 4.08 1.79 69
Total 2.72 2.10 141
The two-way ANOVA results showed that there were two main effects: English proficiency and treatment. The effect of English proficiency on domain specific reading was significant, F (1, 135) = 105.51, p = .000, p = .44. (p<.05) (see Table 22). The Bonferroni procedure indicated that the participants‘ domain-specific reading comprehension scores were different at two English proficiency (High/Low) levels (p ＝ 0.000). Higher proficiency participants (M＝4.07, SD=1.79) performed
consistently better on the academic reading than the lower proficiency participants (M
＝1.42, SD=1.45).
As for the main effect of treatment, there was also a significant treatment effect in this ANOVA, F (2, 135) = 13.69, p ＝0.000, p = .17. (p<.05), even only when
L2 L1
Control
Treatment
Figure 7. Differences of means between H/L English proficiency groups
5.00
4.00
3.00
2.00
1.00
0.00
Estimated Marginal Means
Low High English HL
high and low proficiency students were considered (141 students in total). The Bonferroni procedure indicated that the participants‘ domain-specific reading comprehension scores were different at three different treatment conditions. The participants who received L1 treatment (M＝3.08, SD=1.81) performed significantly better than those who received no treatment (M＝1.82, SD=1.83), and the participants who received L2 treatment (M＝3.40, SD=2.28) performed significantly better than those who received no treatment (M＝1.82, SD=1.83). However, there was no difference between the participants who received L1 treatment (M＝3.08, SD=1.81) and L2 treatment(M＝3.40, SD=2.28) (See Table 20).
Table 22. Results of the Two-Way ANOVA Test (English levels/Treatments)
Source df SS MS F Sig. p2
Treatment 2 61.57 30.78 13.686 .000 .169
EngHL 1 237.30 237.30 105.508 .000 .439
EngHL × Treatment (Interaction)
2
2.11 1.06
.469 .626 .007
Error 135 303.63 2.25
Finally, the output of the ANOVA from Table 20 shows that there was no evidence of a significant interaction effect, F (2, 135) = .47, p = .63, p = .008.
Therefore, there was no interaction between pre-reading input treatments and English proficiency levels.
Analysis of the Survey Questionnaire Data
The researcher attempted to juxtapose the results obtained from the fourth research question with the findings obtained from the other three research questions.
A frequency analysis on the information collected from the qualitative design of the
fourth research question with five survey questions was presented before the analysis of written responses were provided.
The first three research questions have focused on the participants‘ actual performance on domain-specific reading comprehension when the factors of
pre-reading treatment, content knowledge, and L2 reading proficiency came into play.
The fourth question shifted the focus to students‘ learning experiences and conceptual understanding of these above-mentioned factors by investigating the participants‘
perceptions toward these factors.
By completing the survey questionnaire, the students provided their insights on the perceived importance of background knowledge and L2 reading proficiency to domain-specific reading. The survey questionnaire explored the overlap between cognitive and non-cognitive factors. As the cognitive factors alone cannot account for all the learning process, the non-cognitive domain such as self consciousness,
intuitions, emotions, feelings, or perceptions could turn out to be of vital importance in language learning.
In the present study, the researcher divided the fourth research question into two parts. The first part explored the participants‘ intuitive recognition on the importance of the above-mentioned two knowledge bases, namely, domain background
knowledge and English proficiency. This questionnaire might offer certain suggestive insights into the issues of domain-specific reading which differ from the results of the statistical analyses. The qualitative data could be interpreted as the extent to which the participants were aware of the role background information and L2 proficiency play, yet with varying depths of understanding and conceptualization of how they should have applied these knowledge bases.
The second part of the fourth research question investigated how students of varying levels of background knowledge and L2 proficiency perceived the
contributions of pre-reading treatment. The emphasis was on how the high/low background knowledge participants found the contribution of pre-reading inputs, and also what language they preferred for the pre-reading input. The analysis starts with the first part of the fourth research question in answering how the participants perceived the importance of background knowledge and L2 reading proficiency in domain-specific reading, and then the researcher moves on to the second part of the fourth research question by analyzing the participants‘ perceptions of pre-reading inputs.
The Importance of Background Knowledge to Academic Reading The answers provided by all three hundred participants were taken into account in the quantifiable results. In the first question of the three-question-set, regarding the perspective on the importance of background knowledge for academic reading, the majority (89%) of the participants agreed disciplinary background knowledge was related to academic reading; a smaller group of respondents (7%) disagreed; and the remaining participants (4%) did not answer this question. Figure 8 illustrates that a large percentage of the participants agreed that disciplinary background knowledge is important to content area reading.
Figure 8. Participants’ responses to the importance of background knowledge in academic reading (N=300)
89%
7% 4%
Yes (89%) No (7%)
Missing data (4%)
As this survey required open-ended answers, an individual respondent might have mentioned two or more ideas for one single question. For the informants who agreed that disciplinary background knowledge was important, several reasons were mentioned. The reasons could be grouped into 7 categories. The responses generated by the participants are described in order of frequency: (1) Background knowledge entailed a deeper understanding (109 times); (2) Disciplinary training helped readers see the connection between what was read and what has been learned (80 times); (3) Background knowledge included disciplinary terminology (or vocabulary), which facilitated reading (35 times); (4) Substantive knowledge on political science helped readers find main ideas (28 times); (5) Without background knowledge, reading speed slowed down (18 times); (6) Background knowledge gave readers confidence in reading (3 times); (7) A reader would be able to focus better on the article when equipped with certain background knowledge (2 times). The top three reasons and their frequencies are counted and illustrated in Figure 9.
Figure 9. Reasons why background knowledge is important for academic reading
For those who claimed that the academic reading was not related to disciplinary background knowledge, two most frequently mentioned reasons were (1) political knowledge itself was not sufficient to tackle academic reading difficulties
109 (36%)
encountered (14 times), and (2) having related background knowledge was not the
encountered (14 times), and (2) having related background knowledge was not the