The imminent visit to Australia by New York City Schools Chancellor and prospective US Secretary of Education, Joel Klein, puts his education policies in the spotlight. Federal Education Minister, Julia Gillard, has been spruiking them ever since her visit to New York last August.
Gillard says that Klein’s reforms to the New York City public school system are “working” and have produced “remarkable outcomes”. She says that there has been continual improvement in student achievement in New York City under Klein.
However, these assertions are refuted by test results in reading and mathematics. National tests show that average student achievement in New York City schools has stagnated while state tests show a mixture of increases and declines, with no consistent pattern of improvement.
The National Assessment of Education Progress tests conducted by the US Department of Education show no statistically significant change in average student scores for reading in Grades 4 and 8 between 2003 and 2007 in New York City. They show a small improvement in Grade 4 mathematics but no improvement in Grade 8.
They also show that there was no improvement in average reading scores for low income, Black and Hispanic students in either Grade 4 or 8. There were small improvements in average mathematics scores in Grade 4 for low income, Black and Hispanic students. In Grade 8 mathematics there was no improvement for Black and Hispanic students, but a slight improvement for low income students.
Scores for New York City students on the New York State tests are just as unconvincing. Average scores for English across Grades 3-8 show a mixture of increases and declines between 2003 and 2008. For example, average Grade 3 scores increased significantly while average Grade 8 scores declined substantially. Small increases occurred in Grades 4 and 5 while Grade 5 and 6 scores declined slightly.
Large improvements in average scores occurred in mathematics in Grades 3, 4 and 5 between 2003 and 2007 while there was a large decline in Grade 8. There was a small decline in Grade 7 and no change for Grade 6.
There is also little evidence of any improvement compared to the previous four years, as there was a similar pattern of increases and decreases from 1999 to 2003 and from 2003 to 2007/8.
Most experts agree that the state tests are less reliable than the national tests. For example, the state tests do not report measurement errors. This makes it difficult to determine whether results are statistically different over given periods or between different groups of students. Experts have also suggested that the state tests are too easy.
Claims that the achievement gaps between Black and White students, Hispanic and White students and low income and other students have narrowed in New York City since 2003 are also incorrect. They have remained as large as ever under Klein according to both national and state tests.
National tests show no reduction in the difference in average scores between Black and White students, Hispanic and White students and between low income and other students in reading and mathematics in Grades 4 and 8 between 2003 and 2007. There was also no change in these achievement gaps for 2003 Grade 4 students who reached Grade 8 in 2007.
The State tests show a number of small increases and declines in the achievement gaps for reading and mathematics between Black and White students and between Hispanic and White students from 2003 to 2007/8.
Chancellor Klein resorts to several artifices to claim that student achievement has increased and achievement gaps have narrowed.
He often uses the 2002 results as the comparison benchmark instead of 2003. The 2003 tests were conducted 6 months prior to the implementation of his reforms, so this is the appropriate comparison point. Using 2002 exaggerates the impact of the reforms because there were significant increases in student achievement from 2002 to 2003, but this was well before Klein’s changes were made.
Klein refuses to report the margins of statistical error on test results. He dismisses the importance of reporting statistical error in testing as “playing something of a game”.
For example, the New York City Education Department has re-produced the results of the national tests in a special publication to claim significant improvements in student achievement. However, it failed to follow the national practice of reporting the margin of errors and ignored the fact that the results for 2007 are mostly statistically indistinguishable from those of 2003.
The fact is that New York City has very little to offer Australia in the way of education reform. Student achievement in Australia is amongst the highest in the world and far exceeds that of New York.
Australia should be looking to overseas practice to help address its major challenge – reducing the large achievement gap between students from low and high income families. It should look to Finland, which not only has the highest outcomes in the world according to OECD tests but also one of the smallest achievement gaps.
Julia Gillard would have done better to invite Dr. Sakari Karjalainen, Director-General of Finland’s Ministry of Education, to advise on effective polices to improve equity in education rather than someone whose policies have been shown to be ineffective.
Trevor Cobbold