Asked by Senator Penny Wright at a session of the Senate Standing Committee on Education and Employment whether there was any evidence that Independent Public Schools (IPS) lifted student performance, Mr Tony Cook, Associate Secretary, Early Childhood Education and Care in the federal Department of Education answered that there was “a range of evidence”.
In his formal response to the question he stated that the following studies have drawn links between school autonomy/school leadership and student performance:
The Productivity Commission, 2012, Schools Workforce, Research Report, Canberra
Organisation for Economic Cooperation and Development, 2011, PISA in Focus 9, Paris
The World Bank, 2011, Making Schools Work: New Evidence on Accountability Reforms, Washington DC
McKinsey and Company, 2010, How the World’s Most Improved School Systems Keep Getting Better
The World Bank, 2009, Decentralized Decision-Making in Schools, the Theory and Evidence on School-Based Management, Washington DC
Wößmann, L., 2007, International Evidence on School Competition, Autonomy and Accountability, Peabody Journal of Education, Volume 82, Issue 2-3
These six reports are all that can be cited, not to show evidence that IPS-style autonomy produces improvements in student learning outcomes, but to show that links can be drawn “between school autonomy/school leadership and student performance”.
Those links, when examined, are found wanting in terms of an evidence base for IPS-style autonomy.
1. The Productivity Commission
Chapter 8 (“Leadership and school autonomy”) of this work begins with the key point that “Strengthening school-level leadership could raise student outcomes…” (p. 223). The key word in this key point is could. There is no claim made in the work that there is evidence of the sort asked for by Senator Wright. Instead, there is a review of a spectrum of autonomous models.
The Chapter does make several faith-based predictions, such as “Greater school autonomy should improve student outcomes” (p. 223), but even this has three provisos attached to it.
Not only is there no citation of evidence that autonomy has improved student learning outcomes, there are warnings that autonomy might work against equity within the system. In an earlier section, the work observes that “allowing schools greater autonomy has the potential to exacerbate inequalities unless all school are properly resourced” (p. 44). The sabotage of the Gonski funding model by the Abbott government means that all schools will not be properly resourced, that many will remain below the Australian Schools Resource Standard. Later in the work, it is noted that “increased autonomy could, in several respects, work against the interests of disadvantaged students” (p. 277).
2. The OECD
The work cited is a four-page brochure that summarises more extensive work done by the OECD. The brochure makes two key points:
• In countries where schools have greater autonomy over what is taught and how students are assessed, students tend to perform better.
• In countries where schools account for their results by posting data publicly, schools that enjoy greater autonomy in resource allocation tend to show better student performance than those with less autonomy. In countries where there are no such accountability arrangements, schools with greater autonomy in resource allocation tend to perform worse.
It would seem at first glance that the combination of autonomy over curriculum and assessment (first dot point) and resources (second dot point) provide compelling arguments in favour of school autonomy in general. However a more careful reading indicates that this is not the case.
On page 2, the first dot point is elaborated so that we learn that “At the country level, the greater the number of schools that have the responsibility to define and elaborate their curricula and assessments, the better the performance of the entire school system…”
This not the sort of autonomy embedded in Australian practice where an Australian Curriculum defines content and performance indicators across the systems.
“In contrast,” the document continues, taking up dot point 2, “there is no clear relationship between autonomy in resource allocation and performance at the country level”. This does not contradict the second dot point. That referred to individual schools. Continuing, the document explains “This may be because how resources are distributed tends to benefit individual schools but does not necessarily affect a system’s overall performance”. In other words, there is a recognised problem of equity between schools where autonomy of resource allocation is implemented.
Even the claim that public posting of data enables schools that practice autonomy in resource allocation (hiring and firing teachers, setting teacher salaries and rewards, prioritising resource allocations etc) to achieve better results than other schools is less than convincing. Trevor Cobbold describes the impact as “trivial”: “Students in higher autonomy schools achieve only 2.6 points higher in reading on the PISA scale than those in an average autonomy school. To put this in perspective, increased learning over the school year amounts to an average of about 35-40 points on the PISA scale.”
3. The World Bank 1
The first study cited from the World Bank focuses on accountability mechanisms in schools in the developing world, noting that “empirical evidence supporting those measures was limited – uncomfortably so” (Introduction p. ix). After citing evidence from countries such as Kenya, Pakistan and Nepal, the authors conclude that “The evidence to date is that SBM can improve learning outcomes, though the results are mixed” (p. 219). It is misleading to quote a study of very poor nations, that admits evidence is mixed, to support an IPS-style model of autonomy in a developed country.
4. McKinsey and Co.
This report surveyed 20 systems from around the world, all with improving but differing levels of performance. It identified three types of measures used by systems to improve school performance: 1) changes to structure including establishing new institutions or school types, altering school years and levels, or decentralizing system responsibilities; 2) changes to resources including providing more staff and funds; 3) changes to processes by modifying curriculum and improving the way that teachers instruct and principals lead.
It observed that the public debate “often centres on structure and resource…However, we find that the vast majority of interventions made by the improving systems in our sample are ‘process’ in nature”.
The report calls for “balancing school autonomy with consistent teaching practice” and agrees with the OECD that “increasing the responsibilities and flexibilities of schools and teachers to shape instructional practice” is a characteristic of “systems further along the journey”, that is, of the systems with the highest levels of improvement. It notes that “collaborative practice becomes the main mechanism both for improving teaching practice and making teachers accountable to each other”.
This is not an endorsement of IPS-style school “autonomy” and offers no evidence for success through the single measure of “decentralizing system responsibilities”.
5. The World Bank 2
This study found difficulty in analysing the evidence because of vastly different types of school autonomy (or school-based management):
SBM programs lie along a continuum in terms of the degree to which decision making is devolved to the local level, from limited autonomy at one end, to more ambitious programs that allow schools to hire and fire teachers, to programs that give schools control over substantial resources, to those that promote private and community management of schools, and finally to those that eventually may allow parents to create their own schools. There are both “weak” and “strong” versions of SBM, based on the degree of decision-making power that has been transferred to the school. (p. 99)
It notes, in a section headed “Evidence Base”:
The number of rigorous studies of the impact of SBM is very limited. A few studies, rigorous and well documented, reliably measure the effect of SBM policies, but it is very difficult to standardize the sizes of the outcome variables because of differences in how they were measured in the various studies. (p. 100)
It adds, in relation to developing countries, that “it is not yet clear how school decentralization eventually will affect student performance” (p. 102).
This is hardly the evidence base to provide support for IPSs!
6. Woesmann
This 2007 study uses 2003 PISA data. Trevor Cobbold simply dismisses this as having been “contradicted since by the 2009 and 2012 PISA studies”. In any case, it suffers from the same variability of autonomy types as noted in World Bank 2, with one of its contributing authors admitting that it is extremely difficult to disentangle various national policy, institutional and cultural factors influencing education outcomes from the impact of school autonomy.
Conclusion
There is no evidence base in the six studies cited by Tony Cook to support his contention that “there are significant amounts” of evidence that Independent Public Schools lift student performance.
There are plenty of “mights” and “coulds”, all in the context of calls for caution in interpreting what is seen to be very mixed evidence.
I would suggest that Mr Cook has failed to produce evidence in support of IPS-style autonomy.
It is not the first time he has appeared before such a committee, and his repeated citation of such sources is quite misleading.
Mike Williss
Research Officer
South Australian Branch of the AEU