A new study has found that school autonomy widens the gap between the top and bottom achieving students. It shows that school autonomy has little effect on overall student performance, but has a small positive effect for the top students and no effect on lower achieving students. It adds to the weight of evidence that increasing school autonomy does not work.
The study analysed schools that converted to academy status in England from 2002 to 2009 under the Labour Government. Under this program 130 secondary schools operating in disadvantaged areas were converted to academy status. Under the Coalition Government, the number of academies has increased as other schools can now convert to academy status.
Academy schools enjoy a larger degree of autonomy than any other school type in the state system in England. They are managed by a private, independent sponsor through a largely self-appointed board of governors that has responsibility for hiring the staff, negotiating pay and working conditions, and deciding on matters such as career development, discipline and performance management. Some academies have more autonomy over a part of the curriculum as well.
The study by researchers at the Centre for Economic Performance at the London School of Economics and Political Science concluded that the Labour academies slightly increased overall student performance but not for lower achieving students.
Whilst there is a paucity of robust and coherent evidence to draw upon, it does not seem unreasonable to say that, on balance, the evidence that does exist at best shows only small beneficial effects on overall pupil performance and very little consistent evidence of improvements for tail students. [p.11]
The study built on an earlier study that showed a small positive benefit from increased school autonomy. The average size effect was 0.15 of a standard deviation which is a small effect. It also found that there was a negative, but statistically insignificant, effect for schools that had only recently converted to academy status.
The new study investigated the impact of school autonomy across the distribution of student performance on the results for the General Certificate of Secondary Education (GCSEs) for 16 year-old students. It examined for the effect on students ranked by their test scores within their secondary school and on students ranked in the national distribution.
Overall, it found that schools that converted to academies between 2002 and 2007 raised the attainment of students in the top half of the performance distribution, and, in particular, students in the top 20% tail. Conversely, it found no evidence that academies helped students in the bottom 10% and 20% of the ability distribution.
The study found positive, statistically significant effects in the 50th to 80th and top 20 percentiles of the within-school distribution. The size effect for 50th to 80th percentiles was small at less than 0.2 of a standard deviation and there was a medium size effect for the top 20 percentiles of 0.32 of a standard deviation. There was virtually no effect on percentile groups in the bottom 50 percentiles.
Similarly, there were small positive effects in the 25th to 50th, the 50th to 80th and top 20 percentiles in the national distribution. None of these effects were statistically significant. However, they were clearly much larger than those for students at the bottom of the performance distribution.
The study found little evidence that later converters to academies (2008 and 2009) had any beneficial effects on students across the whole performance distribution. The effects for schools that were later conversions were negative for each of the percentile groups, but the results were not statistically significant.
The study concludes from this that:
…. irrespectively of whether we rank pupils by the school or national ability distribution, the effects of academy conversion are insignificantly different from zero – and possibly negative for later conversions – in the bottom 10% and 20% of the ability distribution, suggesting no beneficial effects on tail students in academies. [p.9]
The paper also reviews the results from other studies of school autonomy in England, the United States and Sweden. Previous studies of Labour Government academies show mixed evidence. Some found small positive effects on student achievement; others found no effect. Some found that academies had reduced their intake of students from the lower end of the performance distribution. For example, one study found that converting academies enrolled approximately 12.5% fewer students who are eligible for free school meals, suggesting that academies had became more ‘exclusive’.
The review of charter schools in the US is highly selective, being confined to charters that used lotteries to allocate places and excluding studies that compare schools with similar demographic characteristics. Studies of charter schools using lotteries tend to find positive effects on student achievement, but problems have been found with some of these. For example, the Boston study cited in the review included only high achieving charter schools, which comprised only 7 out of 29 charter schools in the city. Further, the New York study referred to contained methodological problems which when corrected resulted in much lower gains.
Unfortunately, the paper ignored a large number of studies of charter schools that found little to no impact on student performance. The overall weight of evidence from reviews of studies is that charter schools are no more successful than traditional public schools in terms of student achievement. For example, an overview of a special issue of the journal Economics of Education Review on charter schools concluded:
… the existing literature is inconclusive about the aggregate effect charter schools have on student achievement. Some studies in some locations find charters outperform traditional public schools, some find they are no different than the traditional ones, and some find they perform worse. [Eugenia Toma & Ron Zimmer 2012. Two Decades of Charter Schools: Expectations, Reality and the Future, Economics of Education Review 31 (2): 209-212]
The paper also reviews the very limited evidence on the experience with “free schools” in Sweden. It notes that the major study finds a very small positive impact on student achievement. However, the study also found that free schools tend to ‘siphon away’ from public schools children whose parents have higher education levels and who are not first-generation immigrants.
All in all, the new study provides little comfort for advocates of greater school autonomy. At best, the effect on achievement is small and concentrated in the upper end of the performance distribution of students. There appear to be no gains for students at the bottom of the performance tail from school autonomy. The result is a widening of the achievement gap.
Trevor Cobbold
Stephen Machin & Olmo Silva, School Structure, School Autonomy and the Tail, Paper No. CEPSP29, Centre for Economic Performance, London School of Economics and Political Science, March 2013.
This article has been slightly revised since first posted.