Evidence continues to accumulate that the teacher bonus scheme soon to be introduced in Australian schools is doomed to failure and a waste of taxpayer funds. Another study published this week has shown that paying performance bonuses for teachers does not increase student achievement or change teaching practices.
The latest scheme to fail is the New York City bonus program introduced by Julia Gillard’s own hero and mentor, former schools chancellor Joel Klein who is now in charge of Rupert Murdoch’s push into the education technology market. The Prime Minister has copied yet another failed policy from Klein.
A study of the New York City Schoolwide Performance Bonus Program (SPBP) found that the performance bonuses did not achieve any of their intended effects. They had no positive effects on student achievement at any grade level in primary, middle and high schools. They also had no positive effect on how schools performed on annual school progress reports (graded A-F).
There were also no differences between the reported teaching practices, effort and attitudes of teachers in schools that participated in the bonus program and schools that did not.
The study concluded:
Overall, the evidence from this study does not support continuing SPBP. The program did not have the desired effects on student achievement or the necessary intermediate teacher outcomes. There is no consistent evidence that the program motivated educators to change practices or that continuing the program would improve outcomes. Moreover the program was costly, and costs fluctuated dramatically across years. [p.xxix]
The study was published by the prestigious RAND Corporation and is the most comprehensive study to date of New York’s performance pay program. As a result of the findings, the New York City Education Department announced that the program will be abandoned.
From 2007 to 2010, nearly 200 New York City public schools participated in the bonus program. It provided financial rewards based on school-level performance to teachers in high-needs elementary, middle, and high schools. Participating schools were given annual performance targets based on school Progress Reports, which are the main school accountability measure in New York City.
The theory behind the program was that an incentive pay system would motivate teachers to improve their practices and increase student achievement. It was also assumed that the chance to earn a bonus on the basis of school performance could enhance collaboration and that winning bonuses could boost morale – both of which would lead to better outcomes. Another assumption was that rewarding some teachers on the basis their individual performances would provide extra incentives for change. None of these assumptions proved to be accurate.
Some $56 million in performance bonuses were paid to schools during this three-year period. The bonuses were paid on a school-wide basis, not an individual-teacher level. Schools qualified for bonuses if they exceeded their performance targets. Each school that won created a committee of teachers and administrators to determine how to distribute the money. Most schools decided to distribute the bonuses equally to all staff members, amounting to about $3,000 per teacher. However, some schools provided differential bonuses to teachers.
The study compared the results of students in schools participating in the program with those of a control group of schools that did not participate but were eligible to do so. In 2007-08, 427 high-needs schools were identified as eligible for the program and about half were randomly selected to participate and half were not selected. The latter were used in the study as the control group of schools.
The researchers found that the average mathematics and English test scores of students from elementary, middle, and K–8 schools invited to participate in the program were lower than those of students from control schools during all three years of the experiment. However, the differences were very small and statistically significant only for mathematics in year 3 and were not significant when the researchers controlled for testing effects from multiple years and subjects.
Similarly, the researchers found no overall effects on state Regents Exam scores for high school students. The program’s effects did not differ among schools of different sizes or according to bonus award distribution plan.
The program also did not affect school Progress Report scores. The researchers found no statistically significant differences between scores of participating and control schools for all years and all categories of scores. The lack of effects held true for elementary, middle, K–8, and high schools.
The theory behind paying performance bonuses to teachers is that it will motivate change, but the study did not find any evidence of this. The program did not affect teachers’ reported attitudes, perceptions, and behaviour. No differences were found between the reported practices and opinions of teachers in treatment schools and those of the control group. The survey responses about instructional practices, effort, participation in professional development, mobility and attitudes from the two groups were very similar, with no statistically significant differences. Furthermore, the vast majority of teachers who received bonuses said that the bonus did not affect their performance.
The lead researcher, Dr. Julie Marsh, a visiting professor at the University of Southern California, told The New York Times: “A lot of the principals and teachers saw the bonuses as a recognition and reward, as icing on the cake. But it’s not necessarily something that motivated them to change.” She said that teachers reported that improving as teachers and seeing their students learn were bigger motivators than a bonus.
Only 15% of teachers reported that not receiving a bonus motivated them to improve their practice the subsequent year, and only a very small proportion reported that not receiving the bonus reduced their motivation.
There was also no evidence that paying bonuses on the basis of overall school performance led to more collaboration between teachers. The reported levels of staff collaboration differed very little between the SPBP and control schools, and the differences that did exist tended to show greater collaboration in the control schools.
While the bonuses were mostly awarded to all teachers in a school, about 30% per cent of schools reported using individual performance as at least one of the factors for determining awards. This system of payment had no differential effect on student achievement. Students in these schools did not have higher achievement than those in other schools participating in the program.
However, the study did note some divisive effects of awarding bonuses according to individual teacher performance. It found that unequal disbursement of bonuses at times resulted in resentment within the schools. Some schools which had highly differentiated allocation schemes one year adopted much more egalitarian plans the subsequent year. The study concluded that differentiating pay based on performance challenges deeply ingrained norms of collaboration and egalitarianism in schools.
The researchers found that several aspects of the implementation of the scheme may have weakened the motivational effects of the program. The majority of teachers felt that the bonus criteria relied too heavily on test scores. Although teachers reported being aware of the program and generally supportive of it, more than a third did not understand key elements of the program, including targets, bonus amounts, and how committees decided on the distribution of bonuses.
The existence of other accountability measures that apply to all schools may also have weakened the motivational effects of the scheme. Many teachers acknowledged that other accountability pressures and incentives such as school Progress Report grades and achieving adequate yearly progress targets held the same motivational value as the possibility of receiving a financial bonus. The study commented that they would probably have had similar motivation without the bonuses because of the high level of accountability pressure all schools and their teachers face.
The study made several recommendations to reduce implementation problems but stated that “we do not have evidence that implementing these adjustments will improve the effectiveness of the program significantly” [p.xxix]. It said that the whole theory behind teacher bonuses may need to be re-evaluated.
The RAND study is the second this year to conclude that the New York City bonus scheme had no effect on student achievement. A study published by the National Bureau of Economic Research in March yielded similar results. Its author concluded:
I find no evidence that teacher incentives increase student performance, attendance, or graduation, nor do I find any evidence that the incentives change student or teacher behaviour. If anything, teacher incentives may decrease student achievement, especially in larger schools. [abstract of study]
These studies add to the accumulation of evidence against paying performance bonuses to teachers. Performance pay systems operating in Iowa and Texas and in the Chicago, Denver and Nashville school districts have all been evaluated recently and found not to have improved student results.
The Prime Minister has long professed to be guided by evidence in formulating education policy. There is now overwhelming evidence that teacher performance pay does not increase student achievement. The Education Minister should follow the lead of the New York City Education Department and cancel the proposed bonus scheme.
There is an alternative. It is to create a standards based career structure that rewards good teachers who remain in the classroom and which is based on a broad evaluation and mentoring system supported and respected by teachers.
Julie A. Marsh; Matthew G. Springer; Daniel F. McCaffrey; Kun Yuan; Scott Epstein; Julia Koppich; Nidhi Kalra; Catherine DiMartino; and Art (Xiao) Peng, A Big Apple for Educators: New York City’s Experiment with Schoolwide Performance Bonuses, Final Evaluation Report, RAND Corporation, Santa Monica, CA.