The ‘like school’ comparisons on the My School website purport to compare the test results of schools with similar socio-economic student populations. However, like is not consistently compared with like. My School’s measure of the socio-economic status (SES) of schools is systematically biased in favour of private schools when comparing their results with so-called ‘like’ government schools.
The bias works in two separate, but compounding ways. My School under-estimates the SES of private schools that draw enrolments from high SES families living in lower SES areas. It also over-estimates the SES of government schools because high SES families resident in their area tend to choose private schools.
There are two sources of this bias. One is that the Index of Community Socio-Educational Advantage (ICSEA) used to measure the SES of schools is based on the average socio-economic characteristics of the areas in which students live and not on the actual SES of their families. Studies by the Australian Bureau of Statistics show that some high income families live in low SES areas and vice versa, so the actual SES of some students will be above the area-average and others below the area-average.
ICSEA also fails to allow for differences in the proportion of high and low SES families that enrol in private and government schools. On average, 47% of high income families choose private schools compared to 24% of low income families. In the case of secondary schools, 55% of high income families choose private schools compared to 26% of low income families.
The greater leakage of high SES students from each area into private schools causes the ICSEA rating of private schools to under-estimate their actual SES because these students are classified according to their (lower) area SES measure rather than by their (higher) family SES.
On the other hand, the ICSEA rating of government schools over-estimates their actual SES because of the leakage of high SES students to private schools. Government schools take a greater proportion of low SES students, but these students are classified at the (higher) area SES rating rather than by the actual SES of their families. The lower SES students carry the higher area SES score influenced by high SES families in the area whose students do not attend government schools. Thus, the level of disadvantage in government schools is under-estimated by ICSEA.
This systematic bias in the measurement of the SES of government and private schools can be illustrated by an example from My School.
My School classifies the wealthy King’s School in Sydney as having the same SES rating as Gundaroo Public School, a small school in a semi-rural area of NSW near Canberra. The King’s School has excellent test results with many green colour codes for above average results while Gundaroo has many red colour codes for below average results.
However, far from being ‘like schools’, they are very unalike schools.
The SES rating for the King’s School is likely under-estimated because it traditionally draws many students from farming families. About 30% of its enrolments are boarding students and only the wealthiest of rural families can afford tuition and boarding fees of over $36 000 a year for primary students. Yet, because these students are resident in lower SES rural areas they carry a lower SES score than their actual family circumstances. The relatively large proportion of these students attending the King’s School therefore significantly reduces its ICSEA rating.
On the other hand, the ICSEA rating for Gundaroo Public School is likely an over-estimate of its actual SES composition
The Gundaroo area has a large proportion of high income, well educated, highly skilled households, but it also has a significant proportion of lower SES families. Census data shows that about 12% of households in the Gundaroo region are relatively low income. Some 32% of the population over 20 years of age did not finish Year 12, 25% had certificate-based qualifications and 30% are employed in lower skilled occupations.
Only about half of Gundaroo’s primary age children attend Gundaroo Public School. Many high SES families send their children to schools in Canberra leaving mostly lower and middle SES families at the local school. However, its ICSEA rating is based on the average SES characteristics of the area, including high SES families who do not attend the school, and therefore over-estimates its actual SES.
This comparison of unalike schools is not an isolated example. There are numerous others on My School.
Another source of bias occurs because of the exclusion of international students enrolled in many high fee private schools from the ICSEA ratings. They are excluded because it is not possible to geo-code their addresses to a Census collection district.
This also artificially lowers the rating of some high SES schools because it is only wealthy overseas families who can afford the high tuition and boarding fees and associated costs of sending their children to Australia. This bias may not be large because of the relatively small number of international students, but it does add to the inherent bias of ICSEA.
Thus, the ‘like school’ comparisons on My School tend to pit higher SES private schools against lower SES government schools. This shows private schools in a more favourable light because students from higher SES families tend to have higher average results than students from lower SES families.
It should also be noted that ICSEA omits a range of factors that strongly influence school results. These include differences in the student composition of schools by gender, ethnic sub-groups and students with disabilities as well as differences in funding, school size, student mobility between schools, student selection and private tutoring.
Some of these omissions may further disadvantage government schools in comparisons with their “like” private schools. For example, schools with higher proportions of students with disabilities participating in tests may have lower average results than other schools with a similar ISCEA value. Government schools generally have higher proportions of students with disabilities than private schools
My School is a travesty of ‘like school’ comparisons. Its biased comparisons in favour of private schools will unfairly affect the reputations of government schools and the careers of their teachers and principals. It will also mislead parents in choosing schools and mislead policy makers in drawing conclusions about best practice in schools.
Trevor Cobbold
This article was originally published in the April newsletter of the Australia Institute.