This article is an abridged version of a new research brief published by Save Our Schools which reviews the research evidence on the impact of charter schools in the United States on student achievement.
Charter schools are an experiment in school autonomy. After 20 years we can say that the experiment has not been a marked success. The latest research evidence shows that charter schools do not deliver better results than other public schools.
Charter schools are independent public schools in the United States. They are publicly funded but operate free from many of the laws and regulations that govern traditional public schools. They are not permitted to levy fees but can obtain funding from private sources and many receive funding from philanthropic and other non-profit organizations. The large majority of charter schools are able to hire and dismiss staff, determine staff working conditions, determine their own curriculum and teaching methods, and control their budgets.
There are an estimated 5,600 charter schools serving approximately two million students in over 40 states and Washington DC. They constitute a small part of the US public education system, enrolling about three per cent of all students. However, charter schools are strongly promoted by US education policy.
Charter schools are seen as a way to provide greater educational choice and innovation within the public school system and increase student achievement. There is much controversy over their success. Some studies show that they do no better than traditional public schools in terms of student achievement. Others show better results and still others show worse results. Other studies point to a range of other effects such as increasing social segregation of students.
Part of the controversy arises from different research methods used to measure the effects of charter schools and the range of factors influencing student results which are taken into account. Isolating the impact of charter schools from other factors involves sophisticated research methods and highly technical statistical modelling. Different methods have their own advantages and disadvantages which may lead to different results.
The first place to start in assessing the evidence about charter schools are professional ‘meta-analyses’ and literature reviews of studies employing sophisticated statistical techniques to account for a range of factors which influence student achievement. These reviews exclude studies which do not employ rigorous research methodologies, of which there are many.
Meta-analyses attempt to synthesise the results of several studies. The most recent meta-analysis was published late last year by the US Centre for Reinventing Public Education. It included 25 studies of charter school performance and found “compelling evidence that charters under-perform traditional public schools in some locations, grades, and subjects, and out-perform traditional public schools in other locations, grades, and subjects”.
It concluded that there was no difference between charter and regular public schools in middle school reading and high school reading and mathematics. There were statistically discernible positive impacts of charter schools in elementary school mathematics and reading and in middle school mathematics. However, the effect sizes were very small. The largest effect found would move a student with median test scores — ranking at the 50th percentile — to around the 52nd percentile after one year at the charter school. Other positive effects were even smaller. A number of studies combine elementary and middle schools together and overall they find no significant effect of attending a charter school on reading or mathematics achievement.
The positive results for charter schools largely come from studies involving relatively few students. Three out of ten studies of elementary schools found negative results from charter schools and two of these studies involved large numbers of students. The number of students in these two studies was 1.6 million and 1.7 million, while the average number in the remaining studies was only 22,000. Most of the estimates finding positive results in middle schools are also from studies of relatively few students. The three studies of high schools which found positive results were also the three smallest studies.
The meta-analysis concluded:
The overall tenor of our results is that charter schools are in some cases outperforming traditional public schools in terms of students’ reading and math achievement, and in other cases performing similarly or worse.
An earlier meta-analysis published by the National Education Policy Centre in 2008 synthesized the evidence across 47 studies and found 19 had positive findings, 12 had mixed findings, and 16 had negative findings. The mean impact rating for charter schools was indistinguishable from zero. The overall conclusion of this meta-analysis was that charter schools perform similarly to traditional public schools.
Literature reviews of charter school studies have also concluded that charter schools do not perform any better than traditional public schools. A review of major studies of charter schools published in the Handbook of Research in Education Finance and Policy concluded:
Research to date provides little evidence that the benefits envisioned in the original conception of charter schools – organizational and educational innovation, improved student achievement and efficiency – have materialized…Convincing evaluation of student achievement effects are now in from five different states. In none of these states have charter schools, on average, had large or unequivocally positive effects on student achievement.
A review published by the Federal Reserve Bank of Chicago in 2008 found that the weight of the evidence does not suggest that charter schools are more effective than traditional public schools. A recent review published by the Albert Shanker Institute concluded that “the vast majority of charter schools get no better and no worse test-based results than comparable regular public schools”. Several earlier reviews came to the same conclusion.
The other main source of evidence on charter school outcomes is large sophisticated national studies.
The largest, most rigorous and comprehensive study of student achievement in charter schools in the United States to date was published by Centre for Research on Education Outcomes at Stanford University in 2009. It found that charter school results were worse than or no better than those of traditional public schools. The study analysed the results of charter schools in 15 states and the District of Columbia and compared them with those of demographically matched students in nearby public schools. It found that the gains in maths results for nearly half of all charter schools (46%) were no different from those in comparable traditional public schools while over one third (37%) of charter schools had significantly worse results. Only 17% of charter schools had significantly higher maths results than students in comparable traditional public schools.
The study also analysed the aggregate impact of charter schools on student performance using a nationally pooled data set covering 70% of all charter school students. On average, the learning growth of charter school students was lower than their traditional public school peers, although the absolute differences were quite small. The gains in reading for charter school students were only slightly below that of traditional public school students while the gains in maths were significantly less for charter school students.
The study concluded:
…this study reveals in unmistakable terms that, in the aggregate, charter students are not faring as well as their TPS counterparts. Further, tremendous variation in academic quality among charters is the norm, not the exception. The problem of quality is the most pressing issue that charter schools and their supporters face.
A large study of middle school charters published by the US Department of Education’s Institute for Education Sciences in 2010 also found no difference in student achievement between charter schools and traditional public schools. It compared the outcomes of 2,330 students in 15 states who applied to charter schools and were randomly assigned by lotteries to be admitted or not admitted to the schools. On average, the charter schools were neither more nor less successful than traditional public schools in improving mathematics or reading test scores, attendance, grade promotion, or student conduct within or outside of school.
Similar findings were made in another study of charter schools across eight US states by the RAND Corporation in 2009. It found that student achievement in charter schools was either lower than or does not differ substantially from those of traditional public schools. It further found that competition from charter schools does not increase student achievement in nearby traditional public schools.
Earlier sophisticated national studies came to the same conclusion. A study published by the US Education Department in 2006 covering 150 charter schools and 6,764 public non-charter schools found that average results in reading and mathematics in charter schools were lower than those for traditional public schools. Another study by researchers at the University of Illinois using national test results in mathematics found that traditional public schools achieved significantly higher results in grade 4 while charter schools achieved slightly higher, but statistically insignificant, in grade 8.
While the latest evidence from national studies generally shows that charter schools do not achieve any better results than traditional schools, some recent sophisticated studies of charter schools in Boston and New York City have found gains by charter schools compared to traditional public schools. However, the gains are over-stated in two of these studies. The Boston study only included high achieving charter schools, which comprised 7 out of 29 charter schools in the city. One of the New York studies contained methodological problems which when corrected resulted in much lower gains. The size of the gains by charter schools in the other New York study is relatively small.
Thus, the general weight of evidence is that charter schools are no more successful than traditional public schools in terms of student achievement. The overview of a recent special issue of the journal Economics of Education Review on the charter school experience concluded:
… the existing literature is inconclusive about the aggregate effect charter schools have on student achievement. Some studies in some locations find charters outperform traditional public schools, some find they are no different than the traditional ones, and some find they perform worse.
As another reviewer has said: “There is no test-based evidence for supporting either form of governance solely for its own sake”. The charter school experience suggests that school autonomy is not a magic bullet for increasing student achievement.