Money Matters for Student Outcomes

A new comprehensive review of academic studies in the United States has found overwhelming evidence of a strong causal relationship between increased school spending and student outcomes. It concludes that “the question of whether money matters is essentially settled” and that “….any claim that there is little evidence of a statistical link between school spending and student outcomes is demonstrably false”.

The new review examines two categories of studies – the older literature which was largely descriptive and based on observed correlations and a newer literature that employs sophisticated statistical methodologies which provide more credible conclusions about causation. It was published in December by the prestigious US National Bureau of Economic Research.

Prior to 1995, all US-based studies relating student outcomes to measures of per-student spending were correlational in nature. These studies estimated in several ways: the relationship between school spending and student outcomes after accounting for family background; the relationship between changes in school spending over time within a particular geographical area (such as a state or district) and changes in student outcomes after accounting for family background; or some combination of the two.

A highly influential study by Eric Hanushek of Stanford University in 2003 examined the findings of 163 studies relating school expenditure to student achievement that were published prior to 1995. It concluded that there is little association between expenditure and outcomes. This study is still cited by many commentators critical of increased funding for schools.

However, the new review says the Hanushek study employed faulty statistical reasoning in which the results of multiple-state studies were combined with single-state studies and this had the effect of muting positive results. Another study conducted a meta-analysis of the studies used by Hanushek and concluded that, contrary to Hanushek’s finding, they suggest a strong association between school spending and student outcomes.

The review concludes that:

The older literature provides strong support for there being a positive economically important association between increased school spending and improved student outcomes. That is, despite claims to the contrary, the application of reasonable statistical reasoning to the patterns across studies would lead one to conclude that there is a strong statistical link between spending and outcomes.

Nevertheless, the older literature does not provide conclusive evidence for or against a causal relationship between increased school spending and improved student outcomes. A key reason is that these studies compared students from different households across schools, so that the observed relationships are correlational. Simply comparing outcomes among families that attend schools with different levels of spending does not yield a causal relationship because there may be many other differences between these families and schools influencing results. The existence of correlation does not imply causation and lack of correlation does not imply lack of causation.

Modern studies of the relationship overcome the limitations of the older literature by relying on external changes to school spending that are unrelated to other determinants of student outcomes and are not driven by the decisions of the individual families under study. Using independent variation in school spending enables researchers to credibly disentangle school spending from family background and disentangle variation in school spending from other underlying differences.

A major external change to school funding in the US has been court decisions in many states forcing changes in the finance of school education. Historically, school education in the US was financed by local property taxes, so richer areas had better funded schools. However, court decisions required governments in some 30 states to reduce inequality in expenditure between school districts. This led to changes in school spending that were not related to other variables that would otherwise make it hard to draw a causal link between expenditure and school outcomes.

The review canvasses a number of multi-state (or national) studies that examine the effects of these changes. Twelve out of 13 multi-state studies found a positive and statistically significant relationship between school spending and student outcomes. This was the case across studies that used different data-sets, examined different time periods, relied on different sources of variation, and employed different statistical techniques. The review concludes from these studies that “the evidence points to a causal positive impact of increased school spending on outcomes on average”.

The review also considered several single state studies of increases in expenditure that were not restricted in terms of their uses. Eight out of nine studies found a positive and statistically significant relationship between school spending and student achievement. The review concluded that “this suggests that budget increases (that are unrestricted in the use of funds) will tend to improve student outcomes in most contexts”.

In contrast, changes that restrict expenditure increases to particular uses appear to have mixed effects on student outcomes. One study of increased textbook funding found significant positive effects on school-level achievement in elementary schools. Of seven studies of the impact of capital spending in different states, four found positive effects on student outcomes and three had null impacts.

Similarly, studies of what is called Title 1 spending had mixed results. The Title I program provides financial assistance to schools and districts with high numbers or high percentages of children from low-income families. Three studies of increased Title 1 expenditure in New York City found no effect on student achievement, but studies in other states and cities have found positive effects. It may be that the New York City program does not work as well as Title 1 programs in other cities or districts.

The review’s conclusion about the relationship between increased expenditure on school education and student achievement is unequivocal:

The fact that the early (and less credibly causal) multi-state studies indicate a positive relationship between school spending and student outcomes suggest that the positive association on average is real. The fact that this is also true using only the more recent credible design-based studies that rely on different samples, sources of variation and time periods, is compelling evidence that there is a positive causal relationship and that money does matter on average.

Coalition education ministers in Australia regularly claim that increased expenditure on schools has failed to improve outcomes. They ignore the evidence of a multitude of Australian and overseas studies that increased expenditure does improve outcomes, especially those of disadvantaged students.

They also ignore the fact that past expenditure increases in Australia were badly misdirected. Figures published in the 2018 Report on Government Services, adjusted for inflation and for items included for public schools but not for private schools, show that expenditure per student in public schools was cut by 0.7% between 2009-10 and 2015-16 but increased by 13% in private schools. It is little wonder that test results have not improved markedly when over 80% of disadvantaged students attend public schools.

As David Gonski said in response to similar criticisms of his plan by Abbott’s National Commission of Audit:

…the essence of what we contended, and still do, was that the way monies are applied is the important driver. Increasing money where it counts is vital. The monies distributed over the 12-year period to which the commission refers were not applied on a needs based aspirational system.

31 January 2019

Leave a Reply

Your email address will not be published. Required fields are marked *