In the weeks around the tenth anniversary of Hurricane Katrina at the end August, there was a veritable storm of reports and comments disputing the outcomes of the re-organisation of the New Orleans public school system following the hurricane. The heart of the issue was the effects from turning New Orleans into virtually an all-charter school city.
New Orleans is the biggest charter school experiment in the United States. Its proponents claim it has boosted student results and have put it forward as a model for other jurisdictions to follow. Others vehemently reject the claims.
After Katrina, the state of Louisiana completely overhauled school education in the city. The vast majority of schools were taken over by the Recovery School District (RSD), which is a state-wide authority established to takeover low performing schools across Louisiana from local district school authorities. The main strategy of the RSD to improve results was to turn schools into charter schools, some operating as independent charter schools and some run by charter school management organisations. At present, there are 57 New Orleans charter schools in this district.
A small number of schools are in the Orleans Parish School Board (OPSB), including thirteen charter schools and five public schools run by the Board. The charter schools in this district are selective and priority enrolment schools.
A new study published by the Graduate School of Education at Stanford University has tried to make sense of the conflicting claims. It examines how the new system affects student choices and the experiences of different kinds of students, how effective it is in providing equitable school experiences for students across the city and what are the outcomes in terms of student achievement and attainment.
Among many findings, the study shows that the New Orleans changes have created a set of schools that are highly stratified by race, class, and educational advantage, operating in a hierarchy that provides very different types of schools and to different types of children. The most selective, highest achieving, best-resourced and most sought after schools within this system are out of reach of the large majority of students in the public schools in New Orleans.
It found that within three descending tiers of schools, and several sub-tiers within each of them, there is a near perfect correlation between the tiered structure and average achievement. These are, in turn, associated with highly differentiated populations of students by race, class, and disability status. The most advantaged students attend schools at the top of the hierarchy, while the neediest attend schools at the bottom.
Schools compete to recruit the most able students and many have a variety of strategies to avoid or eliminate the least able. These include not providing programming for special education students, creating applications processes that high-need students or families are unlikely to complete, organizing academic programs that are aimed at academically able students, and creating stringent academic, attendance, and behavioural expectations that can lead to transferring students out when they are not met. As a result:
…the reform movement in New Orleans has reified and intensified the socioeconomic and racial stratification that existed before Hurricane Katrina. Schools are organized to parallel the external social hierarchy. [p.49]
Students’ academic and disciplinary experiences are strongly shaped by where they land in the school hierarchy. Schools at the top of the hierarchy are the most likely to offer a rich curriculum, with opportunities for higher order thinking and exploration. Schools at the bottom of the hierarchy provide a test-based curriculum enacted in a “no excuses” disciplinary setting that creates frequent punishments that often ultimately lead to exclusion.
The New Orleans accountability system rates schools chronically low performing schools are subject to sanctions and the threat of closure. The study found that these policies create incentives to teach to the test and to recruit and retain students who score better while seeking to avoid or unload those who struggle academically. In this situation, students who are the hardest to educate end up being pushed down the stratification “ladder,” landing at the bottom of the schools hierarchy.
As a function of the accountability incentives, and the exclusions and school closings associated with them, we found that the most vulnerable students have the least choice, the greatest uncertainty, and the greatest disruption and displacement in their educational experiences. They are undesirable consumers who have few options, are often pushed out of the schools they finally gain admission to, and are most likely to have their schools closed – some of them on more than one occasion. The result is that those students who would most benefit from the continuity and reliability provided by a consistent school experience are the least likely actually to get one. [pp.51-52]
School closure is the primary tool for addressing school quality under the New Orleans accountability regime. The study says this is problematic, as market-based “close and replace” strategies do not necessarily ensure high-quality school options for the lowest performing and most socioeconomically vulnerable students. It creates considerable disruption for students and lowers the achievement of those displaced, who often experience multiple changes of schools both because of school closures and because they have been pushed out of schools where they struggled.
The impact of the changes on student achievement is incredibly controversial. Studies using different comparison groups and different measures come to different conclusions about whether educational outcomes have improved for students in New Orleans.
The study says that evaluating the effects of the changes is very difficult because the city’s changing demographics have made it difficult to analyse trends with comparable groups over time. Thousands of families did not return to the city after the hurricane. Census data show that not only has the city become smaller since 2000, but it disproportionately lost low-income black families and those who did not own homes. The white population is similar post-Katrina but the Black population, which have much lower average test scores, has declined by about one-third. Thus, questions are raised about whether studies have adequately controlled for the population changes.
The Louisiana Department of Education has also made several changes to the school rating system and performance benchmarks that have inflated student and school performance across the state.
Louisiana has twice changed the way it calculates schools’ A to F grades for school accountability purposes over the last decade, and these have made it easier to get higher ratings. It has also kept a substantial number of schools that are opening, closing, or involved in turnaround activities out of the rating system for periods of time.
The state has also changed the test content and the cut scores on its tests for designating students as having reached the “basic” or “mastery” level. The proportion of correct answers needed to achieve a given cut score was lowered and virtually all schools have shown improvements in the proportion of students reaching those levels. As a result, the study states, gains in the percentage of students achieving at the basic level or above are difficult to interpret as evidence of educational progress without making comparisons to other similar schools.
Use of different definitions and data sources for comparing results across schools and over time is yet another problem in sorting the claims that are put forward by proponents, sceptics and various researchers.
As a result of all these problems, the results of published studies vary enormously. Resolving the conflicting results is further hampered by the refusal of the state department of education to provide data requested by a wide range of independent researchers. The study states that this has made it impossible to adequately test the various claims with the individual-level student data that is needed to address the questions of student inclusion and comparability.
However, one state-wide study using national test data that has strong comparability over time has found that Louisiana had the largest disparity in the nation between the performance of students in charters and traditional public schools. It found that charter school students performed significantly lower than comparable students in traditional public schools.
National test data also show that, despite having far more charter schools than any other state, Louisiana continues to be one of the lowest-performing states in the US. In 2013 (the latest data available), Louisiana ranked 48th out of the 50 states in fourth and eighth grade reading, 49th in eighth grade maths, and was tied for 50th in fourth grade maths.
The study concludes that the New Orleans school system remains highly unequal:
Our data provide evidence of the inequalities in learning opportunities offered across schools, the dramatic differences in how students are treated, and the ongoing exclusion and disruption that characterizes the education careers of many vulnerable students. [p.54]
It says that New Orleans has a long way to go in creating a high quality school system serving all children:
Ultimately, a successful system reform will be designed to promote high-quality school experiences for all students in non-segregated settings that safeguard children’s rights of access to supportive learning opportunities. In the context of a school portfolio, such a successful reform will also support school improvement in ways that ultimately create a set of schools that are worth choosing, in which every child will choose and be chosen by the schools that meet their needs. That system has not yet been created in New Orleans. [p.55]