The Victorian Competition and Efficiency Commission report on school autonomy published at the end of last month is a remarkable document. It finds that the research evidence on school autonomy is inconclusive about its effects on student performance, but then it rejects its own finding and recommends increasing school autonomy. In so doing, it opts for faith over evidence.
The review of research studies in the report clearly demonstrates the absence of compelling evidence that greater school autonomy leads to better student performance. The report is forced to concede that the evidence is mixed and inconclusive.
The Commission’s assessment of the empirical studies looking at the relationship between autonomy and school performance reveals a mixed and inconclusive picture. While some studies report a performance benefit — particularly if autonomy is accompanied by effective accountability mechanisms — others find little impact, or even negative effects. [xxvii]
The empirical evidence on the impacts of autonomy and choice on disadvantaged students is mixed. While there are examples where autonomy appears to have improved outcomes for significant numbers of these students, negative impacts have also been observed. That evidence further suggests that where autonomy is accompanied by significant scope for parents to choose the schools that their children attend — as is the case in Victoria — any negative impacts may be magnified. [xlviii]
Overall, the international evidence on the impacts of school autonomy on student achievement is mixed…. There is some evidence, from country-specific and cross-country studies, suggesting that more autonomous school systems can improve student performance. Equally, there is some contrary evidence — including from country-specific and cross-country studies — indicating no gains or a deterioration in student performance relative to public schools or school systems not subject to those reforms. The mixed nature of the evidence is illustrated by the experience in the United States with charter schools, which have been operating for more than 20 years. [48]
A large number of studies have examined the performance of charter schools. In summary, the outcomes have been mixed. [49]
Similarly, research on academies in England – which have high levels of autonomy than other state schools – paints a mixed picture regarding the impacts on student achievement. [49]
…cross-country analyses have found positive, negative and no associations between different measures of autonomy and student performance. Given the mixed findings, the research is not definitive about which dimensions of school autonomy are most important for student outcomes. [50]
Like the evidence on school autonomy and student performance, there is also conflicting evidence on the links between autonomy and educational equity. Some country-specific studies report a worsening of, or no improvement in, the equality of educational opportunity under more autonomous regimes….
At the same time, recent cross-country empirical analyses have found no indication that autonomy differentially affects students from well-off and disadvantaged backgrounds. [53]
The existing body of research into the impacts of school autonomy on student outcomes is by no means definitive. The international evidence is mixed regarding its impacts on student achievement and educational opportunity. It is also not conclusive about which areas or domains might benefit from autonomy. [54]
Mixed and inconclusive evidence does not provide a sound basis for making public policy recommendations. Nevertheless, the Commission proceeds to discount its own evidence and calls on experience from other service sectors:
Given the uncertainty and caveats around the evidence, the Commission considers that experience from school systems in Australia and overseas is informative but not definitive in reaching conclusions on policy design and implementation. It may, therefore, be useful to also draw on the experience, principles and lessons learned from other parts of the economy, such as health care, that also deliver social services to the Victorian community (chapter 5). [55]
However, when we get to chapter 5 of the report no evidence is presented, only assertion:
…the Commission’s research confirmed that the empirical evidence on the impacts of autonomy for school performance — and student outcomes in particular — is not conclusive. While some studies report a performance benefit from autonomy, others find little impact, or even negative effects.
In the absence of reliable empirical evidence, policy formulation must rely more heavily on other information — including relevant experience in other parts of the economy. [75]
Thus, in shifting its case for more school autonomy to the experience of other sectors, the Commission fails to deliver. Not one shred of evidence is presented in the report to show that market-based mechanisms in other social services have led to better outcomes.
The best the Commission can do is to point the fact that a significant number of parents choose to enrol their children in out-of-area government schools, non-government schools, or select entry government schools. However, it fails to demonstrate that these choices lead to better student performance. Once again, faith in the market rules for the Commission.
The report claims that holding autonomous schools accountable for the outcomes they deliver has the potential (note, only ‘potential’) to render school management and performance more transparent and, in the process, aid the dissemination of successful innovation across schools. This claim ignores the evidence from England, New Zealand and the United States that competition between schools actually hinders the spread of successful innovation because schools are unwilling to share with their competitors.
The Commission’s final resort is to assert that there is no debate about school autonomy:
The upshot is that, notwithstanding the evidential uncertainties, the debate is not in fact about whether there should be devolved decision making. Rather it is about how far it should extend, through what means it should be given effect… [76]
The report is thus intellectually bankrupt: it ignores its own review of the research that there is no compelling evidence to show that school autonomy leads to increased student results; it shifts the foundation for its case to other service sectors without providing any evidence that the market works there either; and, in the end, it resorts to assertion.
This is a stunning example of how advocates of market mechanisms in education opt for faith when confronted by evidence to the contrary. The stark contradiction between the evidence presented in the report and its recommendations indicates a singular lack of intellectual integrity on the part of the Commission. It makes a mockery of the claim in the Commission’s 2013-14 Annual Report that the Commission is committed to “provide practical, evidence-based and challenging advice” [VIII].
Yet, despite this lamentable failure to demonstrate that school autonomy leads increased student outcomes, the Victorian Government announced that it had accepted the recommendations of the Commission to extend power to local school communities. The Victorian Minister for Education, Martin Dixon, said it would improve student learning:
These reforms will ensure students and school communities benefit from the significant and positive impact that good governance has on school performance and student outcomes.
The Minister ignores both the research evidence and Victoria’s own school results over the past decade. Victoria has had the most devolved school system in Australia for nigh on 20 years, yet its results are no better than the most centralised system in Australia – NSW – and on some measures they are worse.
It is hard to say who is the more intellectually bankrupt – the Victorian Competition and Efficiency Commission or the Victorian Education Minister.
Trevor Cobbold
Victorian Competition and Efficiency Commission 2013, Making the Grade: Autonomy and Accountability in Victorian Schools , Final Report, Melbourne.