‘Parents beware! These results are highly inaccurate and misleading! Use them at your peril!’ This should be the red letter warning on school test results published on the My School website.
My School will publish literacy and numeracy test results and Year 12 exam results for all government and private schools in Australia. It will enable direct comparisons of the results of any schools, of schools within local regions and of so-called “like schools” with similar student populations.
However, these results are not a reliable measure of school quality. They reflect the influence of many factors outside the control of schools, they are subject to manipulation and rorting, and the “like schools” comparisons do not compare like with like.
Differences in student background is a critical factor. Test results of mediocre schools with privileged intakes can look good compared to schools with high quality teachers serving less privileged communities. Schools with a high proportion of students from high income families generally have higher average results, but this says nothing about school quality.
Well-off families can also better afford private tutoring. A school may achieve higher results than another simply because a larger proportion of its families use private tutoring.
Ethnic and other differences in enrolment profiles also affect school results. For example, schools with a high proportion of Chinese students generally have higher average results than those with more students from other ethnic backgrounds. Schools with higher proportions of students with disabilities participating in tests may have lower results than other schools.
Some schools may have lower results because they have a high proportion of students who often change school. Studies show that these students tend to have lower average results than students who remain at the same school.
Differences in school results may arise from large differences in school expenditure. Total expenditure per student in many elite private schools is up to double or more that of government and Catholic schools.
Test results are also subject to manipulation and rorting by schools to improve their ranking. Overseas studies show that many schools artificially boost their results by selecting high achieving students; denying entry to, or expelling, low achieving students; suspending low achieving students on test days; using special dispensations such as allowing more time for tests; and outright cheating.
Some of this is already happening in Australia. Many private schools make excessive use of special dispensations for Year 12 exams. Last year, one Sydney private school received special exam dispensations for 42 per cent of its Year 12 students compared to an average of 5 per cent in government schools. Dispensations for students in NSW independent private schools was double the rate in government schools
One Sydney private school that has rapidly improved its ranking has been forcing lower achieving students to complete their Year 12 at TAFE. Many Western Australian schools are pushing their Year 12 students to choose easier subjects to lift the school ranking.
Expert statisticians such as Professor Margaret Wu from Melbourne University and others also warn about large measurement errors in school results which make many school test scores indistinguishable from each other. Many reported differences between schools will be non-existent or due to chance. The results of small schools are particularly volatile because they can be significantly affected by the results of only a few students.
None of these problems are rectified by “like school comparisons” on My School. They fail to consistently compare like with like.
My School groups like-schools according to family socio-economic status (SES), percentage of Aboriginal students and remoteness. Other differences in enrolment profiles, funding, student turnover and school size are all ignored. Like school comparisons are also affected by rorting of school results and statistical error.
Even its school SES measure is flawed. It is based on the average SES of geographical areas, which include both higher and low income families. As high income families are more likely to attend private schools, low SES government schools may be wrongly compared to private schools serving higher SES families in these regions.
For all these reasons, parents should not trust test results on My School. Instead, they should take the time to visit schools, talk to principals and teachers and look in on classes to judge how well a school will meet the needs of their children.
Peter Garrigan
President
Australian Council of State School Organisations