Cram schooling is prevalent among students in Taiwan. The competitive entrance examination system tied with the hierarchically ranked system of secondary schools
28
and institutions of higher education drive students and their parents to seek
supplementary learning opportunities with private family resources. The belief that cram schooling is helpful for gaining a competitive edge is also a factor that contributes to the growth of Buxiban. This growth continues in Taiwan despite the fact that possible negative impacts of cram schooling are well acknowledged by the public and that the educational reform in the last decade has made the college education much more accessible. Students in Taiwan are always under considerable pressure to achieve academically ever since they enter elementary schools. With this societal background in mind, the purpose of the present research is to assess the causal effect of cram schooling for math in the 9th grade, which is the time when junior high students face their first major entrance examination for senior high schools and make decision about whether to choose academic or vocational track.
An accurate assessment of the causal effect of cram schooling for math, however, needs to take into account seriously all possible baseline differences between the group of students who undertake cramming activities and the group who do not. It is also possible that cram schooling have differential effects for different kinds of students. The understanding of how cram schooling may have a positive effect on academic performance also alerts us to take seriously baseline differences and possible heterogeneous causal effects at both the level of individual student and the level of student’s learning environments. These are questions related to issues of self-selection bias and omitted variables. Conventional regression methods like the OLS regression in general are not the most suitable statistical tools to deal with these issues. The present research uses the method of propensity score matching, which is developed under the framework of counterfactual causal inference, to tackle these issues and attempts to get a more accurate estimation of cram schooling on math performance. What propensity score matching attempts to achieve is to make those
29
who actually experienced the treatment of interest, which in this case is cram schooling for math in the 9th grade, with those who have no such an experience comparable under the assumption that after the matching all differences between treated and untreated are eliminated except their treatment status. If this assumption is valid, then the result obtained by the method of propensity score matching should be close to that obtained by an experimental design with randomization. With proper assumptions, the method can also give us separate estimations of causal effect for the whole population, those who are actually exposed to the treatment, and those who do not.
The present research uses the data set provided by Taiwan Education Panel Survey. TEPS data sets collected in 2001and 2003 give the present research an advantage over previous studies in cram schooling by offering a rich set of variables related to student’s individual characteristics, family backgrounds, and school and class characteristics. By being a panel survey, the research can also explore the advantage of using outcome variables of interest obtained from the previous survey.
In this case of the present research, this outcome variable of interest is the math ability score in the 7th grade. With this outcome variable, the research may be able to control for the impact of omitted variables on the estimation of the causal effect of a
particular cause on the later math performance.
The major findings of the present research are revealing in several ways. First of all, the findings show that after taking into account of baseline differences in the participation in math cramming, the average causal effect of math cramming in the 9th grade is positive but fairly small. Moreover, the effect of math cram differs depending on the tendency of undertaking such an activity, on prior math ability, and on parental education levels. In general, students who are more likely to attend cram schools, who have better prior math ability, and whose parents are highly educated would benefit
30
less from math cramming than their fellow students who do not have these tendencies or advantages. While the present research have not explored further the possible effects of math cramming for those who actually did not attend cram schools, the result of PSM estimation of the effect of math cramming on these students suggests that math cramming would probably even more beneficial to them. Since these students include those who are likely to have disadvantaged backgrounds, this counterfactual finding has the policy implication for government to implement after-school programs for the disadvantaged students. Studies in the U. S. have found positive effects of after-school programs focusing on academic instruction (e.g., Black, Doolittle, Zhu, Unterman, and Grossman 2008; see also Bodilly and Beckett 2005). It should be cautioned, however, that the effect of cram schooling is fairly small. An academically focused after-school program that aims to reduce the inequality between disadvantaged and advantaged students in academic achievement may not be able to change the learning gap. Furthermore, the advantaged students may seek cram schooling or private tutoring with family resources.
Even though the positive effect of cram schooling for math in the 9th grade is found to be fairly small in the present research, I doubt, however, if this result will be able to persuade students or parents who believe in such an effect not to undertake cram schooling. After all, for students who are at the top of the competitive pyramid, one or two points still matter a lot if a small change of scores means the chance of being admitted to a desired top-ranked school, university, or academic program. For those who are not so competitive, cram schooling may have a positive psychological effect not accounted for in the present research. In a society that emphasizes effort not innate ability as the basis of academic achievement, undertaking cram schooling matters a lot since it has its cultural significance (Stevenson and Stigler 1992).
On the methodological front, the difference between the estimated effects of the
31
OLS regression method and the method of PSM is very small. This finding supports the view that if the OLS regression model meets all of the assumptions of regression analysis, the OLS regression method will get estimates close to the method of PSM.
However, if the assumptions are not met, then propensity score matching has the advantages, among others, of being a nonparametric method, more efficient, and able to provide the information about the comparability of treated and untreated cases (Harding 2003). Of course, a researcher normally will not be able to know in advance or fully if regression assumptions are violated. For the present research, I have the advantage of using TEPS data sets, which not only provided a rich set of variables but also an outcome variable of interest of an earlier panel, to examine possible biases caused by omitted variables or inappropriate functional forms.
The method of PSM has its limitations. The results of Table 3 also show that when important variables are not available for matching, then the method of PSM by itself cannot overcome the problem of omitted variables. Furthermore, at present, available PSM matching routines for common statistical packages cannot be employed easily to handle treatment variables other than binary variables. For many-valued treatment, researchers will need to recode each value into a binary variable and perform matching for each pair of binary treatment variables (see
Morgan and Winship 2007: 53-57). The present research has only examined the causal effect of math cramming in the 9th grade which is taken as a binary variable.
Obviously, many important issues about the effects of cram schooling will need to consider cram schooling as many-valued treatments. For instance, the effect of cram schooling may be cumulative, which involves the number of hours, semesters, or academic subjects of cramming activities. These are all important issues about cram schooling that need to deal with many-valued treatments.
Future studies of the effect of cram schooling should also consider international
32
comparisons. Neighboring societies of Taiwan like China, Japan, and South Korea, which have similar institutional arrangements in education and share Confucian belief in meritocracy, also experienced the seemly non-stoppable expansion of cramming industry (Zeng 1999; Bray 2003). Whether or not cram schooling has similarly small effect in these countries should be examined with appropriate methods and data in the future.
33
Table 1 Summary statistics for sample cases included and excluded from the study
Included sample cases (N = 11373) Excluded sample cases Variable
Spending most time in doing
math homework .262 .440 .232 2605 .422
Never let anything distract
doing homework 2.067 .820 2.124 2605 .860 Always review course lessons
after school 2.292 .824 2.337 2605 .883
Always try to solve difficult
problems in learning 1.980 .756 2.038 2605 .813 Math is always a headache 2.163 974 2.068 2605 .952
Can keep up with math teaching 2.275 .747 2.580 2061 .847 Math homework performance 3.489 .726 3.23 2061 .878 Attend special class for gifted
students .077 .267 .070 2605 .255
Make own decision about
undertaking cram schooling .736 .441 .485 2605 .500 Cram schooling experiences
Before 4th or never Parental education level
High school
1. According to the source of data, the excluded sample size is different for each variable.
34
Table 1 (continued)
Included sample cases (N
= 11373) Excluded sample cases Monthly family income
Under NT$20,000 NT$20,000 – Less than NT$50,000 NT$50,000 – Less than NT$100,000 NT$100,000 or above Parental educational expectation
High school diploma Classmates often discuss homework or
study together 2.128 .780 2.170 2605 .831 Intense academic competition among
classmates 2.160 .843 2.188 2605 . 882
Classmates often discuss entrance examination
1. According to the source of data, the excluded sample size is different for each variable.
35
Table 2 Logistic regression of undertaking math cramming in 9th grade (N=11,373)
Model 1 Model 2
Variable
Coefficient
(Odds Ratio) Std. Err. z Coefficient
(Odds Ratio) Std. Err. z
Male -.054(.947) .040 -1.35 -.072(.930) .042 -1.73
Spending most time in doing math homework .165( 1.179)*** .046 3.62 .159(1.172)*** .045 3.55
Never let anything distract doing homework -.043(.958) .029 -1.46
Always review course lessons after school -.095(.901)** .030 -3.14
Always try to solve difficult problems in learning .005(1.005) .031 0.15
Math is always a headache .068(1.071)*** .021 3.20
Can keep up with math teaching -.587(.556)*** .033 -18.01
Math homework performance .110(1.117)*** .032 3.40
Attend special class for gifted students -.035(.966) .074 -0.47
Make own decision about undertaking cram
schooling -.716(.489)*** .046 -15.67
Cram schooling experiences
Before 4th or never Parental education level
College
Professional or clerical Sales or service
.214(1.238)*** Family monthly income
NT$20,000 – Less than NT$50,000 NT$50,000 – Less than NT$100,000 NT$100,000 or above
.467(1.595)***
Parental educational expectation
College degree Attend private school -.510(.600)*** .063 -8.08 The 9th grade class is a high ability class
Poor class climate for learning Class average grade is good
Classmates often discuss homework or study together
Intense academic competition among classmates Classmates often discuss entrance examination School location
Log likelihood -7155.3471 -7288.1712
LR χ2(df) 1408.36 (22) 1142.71 (13)
Pseudo R2 0.0896 0.0727
*P<.05 **P<.01 ***P<.001
36
Table 2 (continued)
Model 3 Model 4
Variable
Coefficient
(Odds Ratio) Std. Err. z Coefficient
(Odds Ratio) Std. Err. z
Male -.086(.918)* .043 -1.99 -.082(.922) .044 -1.87
Spending most time in doing math homework .108(1.114)* .046 2.33 .127(1.135)** .047 2.72
Never let anything distract doing homework -.058(.944) .030 -1.89 -.052(.950) .031 -1.68 Always review course lessons after school -.079(.924)* .031 -2.53 -.072(.930)* .032 -2.26 Always try to solve difficult problems in learning .039(1.040) .033 1.21 .038(1.039) .033 1.15 Math is always a headache .039(1.040) .022 1.76 .034(1.035) .022 1.54 Can keep up with math teaching -.424(.654)*** .034 -12.32 -.432(.649)*** .035 -12.41 Math homework performance .119(1.126) .034 3.54 .114(1.121)*** .034 3.38 Attend special class for gifted students -.110(.896) .076 -1.43 -.047(.955) .079 -0.59 Make own decision about undertaking cram
schooling -.790(.454)*** .048 -16.62 -.803(.448)*** .048 -16.75
Cram schooling experiences
Before 4th or never Parental education level
College
Professional or clerical Sales or service
.207( 1.23)*** Family monthly income
NT$20,000 – Less than NT$50,000 NT$50,000 – Less than NT$100,000 NT$100,000 or above
.472(1.603)***
Parental educational expectation
College degree
Classmates often discuss homework or study
together .064(1.066)* .031 2.08
Intense academic competition among classmates -.106(.900) .027 -3.83 Classmates often discuss entrance examination .003(1.003) *** .028 0.10
School location
Log likelihood -6940.6923 -6859.6238
LR χ2(df) 1837.67 (27) 1999.80 (36)
Pseudo R2 0.1169 0.1272
*P<.05 **P<.01 ***P<.001
37
Table 2 (continued)
Spending most time in doing math homework .123(1.131)** .047 2.64
Never let anything distract doing homework -.057(.945) .031 -1.84 Always review course lessons after school -.083(.920)** .032 -2.61 Always try to solve difficult problems in learning .055(1.057) .033 1.66
Math is always a headache .011(1.011) .023 0.48
Can keep up with math teaching -.362(.697)*** .037 -9.79 Math homework performance .101(1.107)** .034 2.98
Attend special class for gifted students -.080(.923) .079 -1.01 Make own decision about undertaking cram
schooling -.826(.438)*** .048 -17.12
Cram schooling experiences
Before 4th or never Parental education level
College
Professional or clerical Sales or service
.174(1.191)*** Family monthly income
NT$20,000 – Less than NT$50,000 NT$50,000 – Less than NT$100,000 NT$100,000 or above
.464(1.590)***
Parental educational expectation
College degree Classmates often discuss homework or study
together .062(1.064)* .031 2.00
Intense academic competition among classmates -.095(.909)*** .028 -3.44 Classmates often discuss entrance examination -.011(.989) .028 -0.38 School location
Log likelihood -6843.9479
LR χ2(df) 2031.15(37)
Pseudo R2 0.1292
*P<.05 **P<.01 ***P<.001
38
Table 3 Average treatment effect of math cramming: Comparisons between OLS and PSM Model (N=113731)
Variable OLS Model
PSM Model
ATE ATT ATU
Difference between OLS and
PSM (ATE) Model 1：Math cramming in 9th grade 11.328***
(.380) 2 ---3 --- --- ---
Model 2：Math cramming in 9th grade + Variables of Model 1 in Table 2
5.795***
(.355) 5.903 5.360 6.379 - 0.108 Model 3：Math cramming in 9th grade + Variables of
Model 2 in Table 2
6.571***
(.329) 6.751 5.889 7.509 - 0.180 Model 4：Math cramming in 9th grade + Variables of
Model 3 in Table 2
3.720***
(.316) 3.828 2.910 4.645 - 0.108 Model 5：Math cramming in 9th grade + Variables of
Model 4 in Table 2
3.757***
(.313) 3.817 3.044 4.504 - 0.060 Model 6：Math cramming in 9th grade + Variables of
Model 5 in Table 2
2.784***
(.264) 2.922 2.639 3.176 - 0.138 1. The sample used for comparing OLS and PSM estimates is the matched sample with common support.
2. Numbers within the parenthesis are standard errors.
3. Model 1 has no matching variable and hence no estimation of ATE、ATT, or ATU.
Table 4 Average treatment effects for those undertaking math cramming (ATT) stratified by propensity scores PSM Model 1 (includes all matching
variables) PSM Model 2 (matching without math ability score in 7th grade) Propensity scores ATT Matched
sample size Unmatched
sample size ATT Matched
sample size Unmatched sample size
Table 5 Average treatment effects for those undertaking math cramming (ATT) stratified by math ability scores in the 7th grade
PSM Model 3 Math ability scores in the 7th
grade ATT Matched sample size Unmatched sample size
Total sample 2.636 11,373 15
Table 6Average treatment effects for those undertaking math cramming (ATT) stratified by parental educational level
PSM Model 4 Parental educational level
ATT Matched sample size Unmatched sample size
Total sample 2.636 11,373 15
High school 3.347 7,659 32
College 1.789 3,271 25
Graduate school -.720 350 51
College and above 1.577 3,671 26
40
References
Alexander, Karl L., Dorris R. Entwisle, and Linda Steffel Olson. 2007. “Lasting Consequences of Summer Learning Gap.” American Sociological Review 72:
167-180.
Aronson, Julie, Joy Zimmerman and Lisa Carlos. 1999. “Improving Student Achievement by Extending School: Is It Just a Matter of Time?” San Francisco: WestEd. Retrieved June 15, 2008
(http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b /80/15/ea/c4.pdf).
Baker, David P., Motoko Akiba, Gerald K. LeTendre, and Alexander W. Wiseman.
2001. “Worldwide Shadow Education: Outside-School Learning, Institutional Quality of Schooling, and Cross-National Mathematics Achievement.”