Julia Gillard claims that publishing school results on the My School website will create pressures for school improvement; better inform parent choice of school; identify best practice and where additional resources are needed.
None of these arguments withstands scrutiny. The Minister’s claims are faith-based rather than evidence-based.
First, there is no conclusive research evidence that reporting school results improves school performance. Even the head of ACARA, Peter Hill, admits there is little evidence to support his Minister’s claim.
Very few studies have separately assessed the impact of reporting school results and virtually all show no significant effect on student achievement. For example, a Brookings Institution study found “no discernable effect”. A frequently cited study by Professor Eric Hanushek and Margaret Raymond of Stanford University on the positive effects of accountability measures found that reporting school results alone has no impact on student achievement.
Most studies assess reporting school results along with a range of ‘high-stakes accountability’ measures. These include rewards for improved performance and sanctions such as reconstitution of schools, replacing the principal or teachers, permitting students to enrol elsewhere, grade promotion standards and having a high school graduation exam.
A recent sophisticated meta-analysis of 14 major accountability studies in the US found mixed effects on reading and mathematics achievement. Seven studies favoured states with high stakes testing and reporting, six studies had mixed or insignificant findings, and one study favoured states with low-stakes testing.
The effects, positive or negative, were quite small. In many instances, the results were contradictory between Year levels tested, between subjects and between different measures of achievement.
One leading academic in the field says that the bottom line from school accountability studies is clear: “Accountability has not generated the significant gains in student achievement that policy makers intended”.
Second, parents may be misled by using reported school results to inform their choice of school because they are an unreliable measure of school quality.
School results often more reflect the socio-economic background of its students. They are strongly influenced also by other external factors such as student absenteeism, student turnover, parent involvement in learning at home, and the proportion of students receiving private tutoring.
School results may be artificially boosted by being manipulated and rorted under the pressure to maintain rankings. Overseas studies show that schools often resort to poaching high achieving students from other schools; denying entry to, or expelling, low achieving students; suspending low achieving students on test days; using special dispensations for tests; and outright cheating. Some of this is already happening in Australia.
Parents cannot be assured that their child will be in a better class in a higher performing school than in a lower performing school because the variation in test scores is much greater within schools than between schools. A school with higher scores than another will likely have classes that perform worse than the best classes in the lower achieving school.
Differences in school test results may be largely due to chance because of inevitable measurement and sampling error in tests. Several studies, including one Australian study, show that the results of up to 80% of schools are statistically indistinguishable when measurement error is taken into account. Real differences in school results can be only identified for a small minority of schools.
Third, the claim that reporting school results is necessary to identify struggling schools and those in need of intervention programs and additional resources is sheer nonsense. This information is already available to education departments. Governments have failed to provide the necessary resources and support for struggling schools.
The unreliablility of published school results also makes it difficult to identify effective school practices. It can mislead policy-makers and schools in adopting particular educational programs.
Gillard also argues that parents have a right to information on school results. However, this is never an absolute right as recognised in some court hearings, national security issues and Cabinet meeting records. While there should be a presumption in favour of releasing information about public institutions, it should be based on assessing the public benefit versus public harm.
The research shows no significant gains from reporting school results while the harm can be substantial. It harms education by narrowing curriculum and teaching; discouraging collaboration between schools around improved strategies and practices; and promoting greater focus on school image rather than school improvement.
Julia Gillard has failed to substantiate her case with evidence while ignoring evidence of its harm. The introduction of reporting school results represents a triumph of ideology and faith over evidence.
Trevor Cobbold
This article is published in the February issue of Australian Teacher Magazine.