A new review of research studies on the relationship between expenditure on schools and education outcomes has challenged the common view that more expenditure does not lead to better school performance. The review published by the UK Office for Standards in Education shows that numerous international studies conducted since the early 2000s show a positive impact of increased expenditure in schools, especially for disadvantaged students. The study adds to the weight of evidence supporting the new Gonski school funding model.
A strong conclusion of the new review is that increases in resourcing are usually more effective in disadvantaged schools and/or on disadvantaged students. The review concludes that this implies that “it is more efficient (as well as equitable) to target resources at these students” [Gibbons & McNally 2013: 27].
The common view that increasing expenditure on schools does not lead to better school outcomes is largely due to an influential review of research studies by the Stanford University economist Eric Hanushek in the late 1980s and updated in the 1990s [Hanushek 1986; see also Hanushek 1989, 1997]. This view has long prevailed even though Hanushek’s main conclusion was refuted by a re-analysis of his review of research studies by academics from the University of Chicago [Greenwald et.al. 1996]. They found that the vast majority of studies with statistically significant effects show a positive relationship between expenditure per student and student achievement. Their conclusion was in sharp contrast with that of Hanushek:
….a broad range of resources were positively related to student outcomes, with ‘effect sizes’ large enough to suggest that moderate increases in spending may be associated with significant increases in achievement” [361].
The Ofsted review shows that many more recent studies have come to a similar conclusion. The study separately reviews the UK and other evidence for primary and secondary schooling. The following is a summary of the review.
Primary schools
UK studies
Two recent UK studies of expenditure in the primary school years are reviewed. One study examined the impact of increased expenditure from the early to late 2000s on student achievement at the end of primary school using data for all schools in the UK [Holmund et.al. 2010]. It found evidence of a consistently positive effect of expenditure across the different subjects (English, Maths and Science). The effect size was small, corresponding to about a 0.03-0.05 standard deviation increase in attainment for an extra £1,000 per student. The effect sizes were 50 to 100 per cent higher for disadvantaged students.
The other study found much larger effects. It found that urban schools on either side of Local Authority boundaries with high proportions of disadvantaged students receive different levels of funding and that this is associated with a sizeable differential in student achievement at the end of primary school [Gibbons et.al. 2011]. For example, an extra £1,000 of spending was equivalent to moving 19 per cent of students currently achieving the expected level (or grade) in Maths (level 4) to the top grade (level 5) and 31 per cent of students currently achieving level 3 to level 4 (the expected grade at this age). The effect size was about 0.25 standard deviations which is a significant effect.
A number of UK studies have also examined the effect of reducing class size (which requires additional expenditure) on student achievement. Some found no effect [for example, Dearden et.al. 2002]. Others have found significant effects; for example, Iacovou [2002] found a strong relationship between class size and reading achievement with an effect size of around 0.29 standard deviations for a reduction in class size of eight students. However, it did not find a significant relationship between class size and maths achievement. Blatchford et.al. [2002] also found strong class size reduction effects. A decrease of class size of 10 to below 25 was associated with a gain of about one year’s achievement for the lowest achieving group and about 5 months for other students. The effects were similar for literacy and maths.
Another study found small positive effects on literacy of very modest increases in expenditure associated with a structured hour of literacy learning [Machin & McNally 2008]. It found a 2-3 percentile (0.06-0.08 standard deviation) improvement in the reading and English skills of primary school children exposed to the Literacy Hour, compared to children in selected comparison schools. The increase in expenditure amounted to only £25 per student per year.
US studies
Many studies have been published on the Tennessee Star Project which was a large scale randomized trial of lower class sizes for students during their first four years in school. Students and teachers were randomly assigned to ‘regular size’ classes of 22-25 students, to another group of regular size including a teaching assistant or to small classes of 13-17 students.
A recent summary of the findings found that students benefited from being allocated to the smaller classes compared to the regular-sized class [Schanzenback 2007]. The effects were small at about 0.15 standard deviations in terms of average maths and reading scores (measured after each grade for the four years). The effects were much larger for black than for white students (0.24 standard deviations compared to 0.12 standard deviations). There was also a smaller differential between disadvantaged students (those eligible to receive a free lunch) and other students. In third grade, students eligible for a free lunch gained about 0.06 standard deviations more than other students.
All students in the project went back to regular sized classes in grade 4. Studies show a continuing positive effect in grades 4 to 8 of being initially assigned to a small class. However the magnitude of the gain reduced to one-third to one half of the initial effect. Again, the impact remained stronger for black and disadvantaged students.
There has been no class size experiment as thoroughly investigated as the STAR experiment. Other studies have used various other strategies used to identify class size effects and do not always come up with results that are consistent with this evidence. One of the strategies used has been to use demographic variation across year groups within a school to identify class size effects. These studies have produced different results. One on primary school students in Connecticut found no significant effects of class size on student attainment [Hoxby 2000], although its results were subsequently criticised [Goldstein 2000; Jepsen & Rivkin 2009]. Moreover, the variation in class size was quite small. Two other similar studies found small positive effects in Texas schools [Rivkin et al. 2005] and in Minnesota schools [Cho et al. 2012].
Several recent studies have looked at the effect of school expenditure (rather than class size) in primary schools. One study of large funding increases for Massachusetts schools found large increases in 4th and 8th grade test scores [Guryan 2001]. Specifically, it estimated that a $1,000 increase in per student spending was associated with a 0.3-0.5 standard deviation increase in test scores which is quite a large effect.
A study of the effect of school finance changes in Michigan on 4th grade students found large effects of expenditure increases on the pass rate in mathematics, with larger effects in initially low performing schools. A more recent study of the Michigan reforms found smaller positive effects of increased expenditure for 4th grade results [Chaudhary 2009].
Other international studies
Several studies have investigated class size reduction effects in primary schools in other countries. An Israel study found significant effects on the educational attainment of students in the fourth and fifth grade comparable to the size of the effects of the Tennessee Star Project [Angrist & Lavy 1999]. A French study of 2nd grade students found that a reduction in class size induces a significant and substantial increase in mathematics and reading scores, and that the effect is larger for low-achieving students [Piketty 2004]. A similar more recent French study found effect sizes close to those found in the Tennessee STAR experiment [Bressoux et al. 2009]. It also found that the effect size was higher for classes with a low initial achievement and also in areas of high socio-economic deprivation.
A Swedish study of reduced class size for 5th grade found positive effects in mathematics, especially for immigrant children [Lindahl 2005]. The effect sizes were similar to those obtained from the Tennessee Star experiment.
A recent Israeli study examined changes in funding rules to the length of the school week and class time spent on different subject areas [Lavy 2012]. It found that increasing instructional time in each subject by one hour per week increases the average test scores in these subjects by 0.053 standard deviations, which is a very small effect. However, the effects on students with parents who have below average education were twice as large in maths, 25 per cent higher in science, but 25 per cent smaller in English compared to those with higher than average education. The author suggested that providing two or three additional hours of maths instruction per week to the low achieving group would go a long way to narrowing the gap between socio-economic groups.
Another study investigated the impact of two specific subsidies that were targeted at primary schools with large proportions of disadvantaged students in The Netherlands [Leuven et al. 2007]. One subsidy provided extra resources to improve teachers’ working conditions and another subsidy provided additional funding for computers and the internet. Neither subsidy was found to have an effect on student results.
Secondary schools
UK studies
The Ofsted review notes that there are few studies in the UK of the effect of school expenditure or class size reductions on student achievement.
Two reports produced for the then Department for Education and Skills looked at effects of additional spending at school-level on student performance at age 14 and age 16 [Jenkins et.al 2005b & 2006a]. Both studies found small positive effects from general spending (or reduced student-teacher ratios) on attainment in science at both ages, effects on maths at age 16, but no effects on English at either age. A recent study found significant but very small impacts from expenditure on progress between ages 11 and 16 [Nicoletti & Rabe 2012].
Other studies have examined the effects of the Excellence in Cities programme which allocated extra resources in the early 2000s to some secondary schools in disadvantaged urban areas in England. One study found evidence of benefits from the programme in mathematics and on attendance at age 14, but not in English, but with variation in the effects across student and school types [Machin et.al. 2010]. The biggest effects were concentrated on medium to high ability pupils in the most disadvantaged schools. The other study also reported beneficial effects from the program [Bradley & Taylor 2010].
International studies
There is an extensive international literature on the effects of increased expenditure and lower class sizes on student achievement across countries and in specific countries.
A common approach is to use variation in school enrolment between cohorts (grades) to estimate the effect of average class size changes on achievement at secondary level. Woessmann and West [2006] find mixed evidence on mathematics and science scores of 13 year-olds in a sample of 11 countries using data from the Trends in International Mathematics and Science Study (TIMSS). They found significant beneficial effects in Greece and Iceland but none or inconclusive results elsewhere. They suggest that the lack of any general effects of class size reductions could be because the magnitude of the effects is dependent on the educational system.
Another related study on a bigger sample of European countries from TIMSS [Woessmann 2005] also found mixed effects, with Iceland again the only country showing clear beneficial impacts from smaller classes, but no evidence across countries in general that resources spent on class size reductions are productive. Similarly mixed findings were found in another study using the TIMSS data to investigate the effects of differences in subject-specific class sizes in a student’s achievement across subjects [Altinok & Kingdon 2012].
Country-specific studies generally show more positive effects of class size reductions in secondary schools on student achievement. The review canvassed the findings of a number of studies conducted in Denmark, France, and Norway [for example Heinesen 2010; Bonesronning 2003; Gary-Bobo, Mahjoub & Badrane 2006]. The studies generally found small positive effects – some very small and some with moderate size effects.
The review found few other international studies of the direct impact of increasing expenditure on the achievement of secondary school students. A Norwegian study found large positive effects with a 30 per cent increase in funding being associated with 0.28 standard deviations higher achievement at age 16 [Haegeland, Raaum & Salvanes 2012].
A recent study of the school finance reform in Michigan noted above which found small positive effects of increased expenditure for 4th grade results did not find a positive effect on 7th grade achievement [Chaudhary 2009].
The Ofsted review also looked at a number of studies in developing countries but stated that it is very difficult to draw conclusions from developing countries with regard to the impact of school resources in developed countries because of major differences in institutional context. In a review of the literature about randomised experiments in developing countries, Kremer and Holla [2009] argued that supplying more of existing inputs, such as teachers or textbooks, often has a limited impact on student achievement because of distortions in developing country education systems. These distortions include elite-orientated curricula and weak teacher incentives, as manifest through a high level of absenteeism.
Conclusion
The Ofsted review notes that research on the effectiveness of additional resources in schools has moved on some way since the early 2000s. There are now a far greater number of high-quality research designs, a better understanding of the challenges to causal estimation, and better data.
It notes that ten years ago, the two prevailing interpretations of the evidence were succinctly articulated by Hanushek [2003] – “resource policies have not led to discernible improvements in student performance” – and Krueger [2003] – “…reanalysis of the literature suggests a positive effect of smaller class sizes on student achievement, although the effect is subtle and easily obscured…”.
The review concludes that the evidence from more recent studies now favours the second interpretation. A far greater number of studies have found evidence of positive resource impacts although even studies with good research designs and high quality data still produce a variety of sometimes conflicting results.
Studies that found evidence of statistically significant impacts generally found effects of a similar order of magnitude, even if they differed in their exact conclusions. The smallest non-zero effects were in the order of 0.02-0.05 of a standard deviation in achievement for a 30 per cent increase in expenditure (equivalent to roughly a six student reduction in class sizes from a mean of 25). The largest impacts were in the order of 0.25-0.30 of a standard deviation for a similar resource impact, an impact that can be characterised as a moderate sized effect.
The review notes that the size of the effects appears small when judged against the overall variation in student achievement. The usual benchmark figure is from the Project STAR programme, where a class size reduction of 8 students (and the resources this entails) was associated with around 0.15-0.20 of one standard deviation improvement in test scores. However, the review further notes that numerous studies have demonstrated that most (more than 90 per cent) of the variation in student test scores is due to family background, parental inputs, natural student abilities and purely random variation. In this context, it says, it is not surprising that the impacts of resource changes are relatively small by comparison.
The review also discounts to some extent the findings of cross-country studies on the effects of increasing expenditure. It says that comparing results across different countries and education systems is problematic because the response to resource changes is likely to depend on the context in each country or system. This is particularly true of studies that look at general funding changes and differences because the effect of more resources will depend crucially on how these resources are used, and there is generally insufficiently detailed data to answer questions about the impact of expenditures on specific items.
On balance more studies appear to find positive resource impacts in primary school than in secondary school. However, the review says this may be in part because there have been more studies focussing on primary schooling and the research designs have typically been better.
The strong conclusion from the review is that whatever the size of the effects across all students, increases in school expenditure have a significant impact on the achievement of disadvantaged students.
Trevor Cobbold
References are available from the original review:
Stephen Gibbons & Sandra McNally 2013, The Effects of Resources Across School Phases: A Summary of Recent Evidence, Research paper for Ofsted’s ‘Access and Achievement in Education 2013 Review’, Office for Standards in Education.
Additional references cited in this summary are:
Goldstein, H. 2000. Class Size and Educational Achievement: New Evidence.
Greenwald, R; L. V. Hedges & R. D. Laine 1996. The Effect of School Resources on Student Achievement, Review of Educational Research, 66 (3): 361-396.
Hanushek, E. A. 1986. Economics of Schooling: Production and Efficiency in Public Schools, Journal of Economic Literature, 24 (3): 1141-1177.
Hanushek, E. A. 1989. The Impact of Differential Expenditures on School Performance, Educational Researcher, 18 (4): 45-62
Hanushek, E. A. 1997. Assessing the Effects of School Resources on Student Performance: An Update, Educational Evaluation and Policy Analysis, 19 (2): 141-164
Jepsen, C. & Rivkin, S. 2009. Class Size Reduction and Student Achievement: The Potential Tradeoff Between Teacher Quality and Class Size. Journal of Human Resources, 44 (1): 223-250.