The UK House of Commons Committee for Children, Schools and Families has made a devastating critique of the impact of league tables on education in England. In a recent report on school accountability the Committee says that school performance tables present a very narrow view of school performance and there are inherent methodological and statistical problems with the way they are constructed.
The report concludes that most of those who use school performance tables, particularly parents, remain unaware of the very serious defects associated with them. It says that they tend to interpret the data presented without taking account of their inherent flaws. It notes evidence presented by Professor Stephen Gorard from Birmingham University that parents were led to judge schools on the basis of evidence which was largely spurious.
The report says that the way parents interpret school performance data can have grave consequences. If a school is unable to boost its standing in the league tables, it may face public censure and risk a declining student population because parents seek to send their children to a different school. A deteriorating reputation can present serious problems for a school in terms of recruiting and retaining the talented staff necessary to turn the school around.
The report says that the pressure on schools to maintain or improve their standing in performance tables can give rise to a variety of practices which distort teachers’ professional practice and which are not necessarily beneficial for students. It says that school performance tables create perverse incentives for schools to teach to the test, narrow the taught curriculum, and focus on candidates on the threshold of target grades. Further, they encourage teaching approaches that stifle creativity so that the chances of engaging children’s interest in learning are low.
The report also presented evidence that the performance tables can give rise to perverse incentives for schools to choose easier qualifications in which it is more likely that their students will achieve higher grades. Schools looking to improve their standing in the tables can migrate towards those qualifications in which students are more likely to get a higher grade and reject qualifications deemed equivalent but which are actually academically more challenging.
The report found that the gap between the proportion of students getting any 5 A*–C grades for the General Certificate of Secondary Education and the proportion getting 5 A*–C including English and mathematics has increased from 10% ten years ago to around 20% in 2009. It says this is likely to be at least partly due to schools choosing easier qualifications for some students.
The report also found that school performance tables favour private and selective schools, which have a lower intake of disadvantaged children or of children with special educational needs. It says that it is unsurprising, therefore, that such schools consistently top the academic league tables.
The report concludes that there is an urgent need for the Government to move away from these damaging school performance tables and towards a system which gives a full and rounded account of a school’s provision.
In this context, the Committee welcomed in principle the proposal of the UK Government to introduce a new school report card system similar which could provide a broader evidence base for assessing school performance. It says it is a step in the right direction. The pilot card introduced last year is similar to that used in New York City.
However, the Committee issues several warnings about school report cards. Many are pertinent to the report card system being used in Australia.
First, it says that while report cards assess a broader range of aspects of school performance than performance tables they cannot be the basis for a definitive judgement of overall performance. In the Committee’s view school report cards will never constitute a definitive view of a school’s performance and the school report card should not purport to give a balanced view of a school’s overall performance. It says that the Government should make clear on the face of the school report card that its contents should only be considered as a partial picture of the work of a school.
Second, it notes concerns about the lack of evidence to support school report cards. Professor Peter Tymms of Durham University told the Committee that it was not possible to know for sure what effect the report card had been in New York because there was no way of making an evidence-based assessment. He said that many reforms and changes had been introduced simultaneously in New York and this made it impossible to draw causal links between a single initiative and an outcome or set of outcomes.
Third, it expresses concern about the complexity of information being presented to parents. It says that during the Committee’s visit to New York, it was told that the report card used there was considered too complex for many parents to understand.
Fourth, it notes widespread concerns about using an overall score or grade for a school’s performance. The report says that the Committee was struck by the weight of evidence against assessing schools by an overall score or grade as is done in New York. It states that a school report card can be a full account of a school’s performance, yet the inclusion of an overall score suggests that it is.
Fifth, it expresses serious concerns about using student well-being indicators such as attendance, exclusions, post-16 progression, the amount of sport provided, and the uptake of school lunches as part of the assessment of school performance in report cards. It questions the extent to which such indicators can really be accurate, based as they are on a limited set of loosely-related quantitative data and problematic survey evidence. It notes that academic research on school effectiveness is lacking in the field of student well-being and wider outcomes beyond assessment results. Consequently, it says that in the absence of robust, independent research evidence, the Government should exercise great caution in widening the accountability system beyond test scores.
Sixth, the Committee warns that there are inherent methodological problems in using survey evidence, such as parent and student satisfaction surveys, in assessments of school performance. It has concerns about the validity of conclusions drawn from unrepresentative samples of parents and students. It says that it is unacceptable that schools with the most challenging intakes might suffer skewed performance scores because of a low response rate to surveys for the purposes of the school report card.
The Committee did not make a final judgment about school report cards because it was in the pilot stage of development. However, it did say that the school report card requires a considerable amount of work before it is suitable for use as a fundamental part of the English school accountability system. It urged the Government to take account of the concerns raised about the proposal.
The most notable omission from the report is a consideration of the potential for school report cards to be used to create school league tables. The report notes how government published school performance tables are turned into league tables by the media and other organisations. It strongly criticises inaccurate, misleading and unfair comparisons of school results. However, it assumes that school report cards will replace league tables. It fails to note that school report cards also publish the same raw school test scores that are used to create league tables. It fails to note that the New York school report cards are used to highlight the top and bottom performing schools in the city and in different regions of the city.
The House of Commons committee only has to look at what has happened in Australia over the last few days if it wants proof that school report cards can be used to create league tables it only. The new My School report cards have been used by the Canberra Times, the Herald-Sun, the Northern Territory News, and the Sydney Morning Herald to produce full or partial league tables.
Trevor Cobbold
The House of Commons Children, Schools and Families Committee Report on School Accountability can be downloaded at http://www.publications.parliament.uk/pa/cm/cmchilsch.htm