A huge controversy has erupted in the United States about the publication of “value added” ratings of teachers by the Los Angeles Times. Last weekend, the Times published a league table of the top 100 elementary school teachers in the LA school district. It also published “value added” ratings of 6000 teachers and 470 schools which can be accessed through a search facility on the Times website.
The US Secretary of Education, Arne Duncan, endorsed giving the public access to teachers’ value-added scores. He said parents have a right to know if their children’s teachers are effective and how well individual teachers fare at raising their students’ test scores. “What’s there to hide” he was quoted as saying in one interview after the Times published the findings of its initial analysis and announced its intention to publish individual teacher ratings on a database [ Los Angeles Times, 16 August 2010].
This is the first time the Obama administration has expressed support for the publication of information about individual teacher performance.
The California Secretary of Education, Bonnie Reiss, also endorsed the move, saying that the state will encourage districts to develop and publish value-added scores for teachers. “It’s going to create a more market-driven approach to results”, she said [ Los Angeles Times, 16 August 2010].
The Times ratings assess 3rd, 4th and 5th grade teachers who taught at least 60 students from the 2002-03 through 2008-09 academic years. Most of Los Angeles Unified School District’s elementary (primary) schools are included.
A teacher’s value-added rating is based on his or her students’ progress on the California Standards Tests for English and mathematics. The analysis compared each student’s prior performance to project his or her future test scores. The difference between the projection and the student’s actual performance was the “value” the teacher added or subtracted.
For example, if a 3rd grade student ranked in the 60th percentile among all district 3rd graders, she would be expected to rank similarly in fourth grade. If she fell to the 40th percentile, it would suggest that her teacher had not been very effective. If she jumped into the 80th percentile, her teacher would appear to have been highly effective.
The idea of publicly rating teachers by name has generated enormous controversy among educators and experts across the country. The debate has focused on whether the value-added method is sound and the publicity is fair to those with low rankings.
Among other things, some researchers are concerned about the wide variation in value-added results for individual teachers from year to year, the potential for statistical error in the results and the possibility that the results would be skewed by how students were assigned to classrooms.
Research studies show that value-added evaluations are not stable year-to-year for individual teachers, and that different reading and mathematics tests give different value-added scores for the same teacher. In recent studies, 10 to 15% of teachers in the lowest category of effectiveness one year moved to the highest category the following year, while 10 to 15% of teachers in the highest category fell to the lowest tier.
One recent study found that error rates for teachers being judged on three years of student data was around 25% and rose to 35% when using just one year of student data. In other words, even when using three years of data, 1 out of 4 “average” teachers will be falsely identified as either exceptional or terrible.
Other studies show that value-added teacher ratings can be substantially biased by the non-random sorting of students. The allocation of students to different classes and teachers is rarely random. Some teachers may be allocated higher or lower performing students and this will affect their value-added rating. It is not possible to determine whether higher or lower student test scores result from teacher effectiveness or are an artefact of how students are assigned to teachers.
Another problem is that teachers will be encouraged to generate higher scores by teaching “test preparation” strategies for getting higher scores at the expense of deeper learning experiences. Higher test scores can also be generated by testing selectively, making sure that low performing students are absent on test day.
Then, there is the avenue of increasing test scores by direct cheating – alerting students to the questions before tests, helping students with their answers during tests or changing answers after the test.
Concerns such as this prompted the US National Research Council to issue a blunt warning last year about value-added measurement of teacher performance:
Too little research has been done on these methods’ validity to base high-stakes decisions about teachers on them. A student’s scores may be affected by many factors other than a teacher—his or her motivation, for example, or the amount of parental support—and value-added techniques have not yet found a good way to account for these other elements. [National Research Council Media Release, 7 October 2009]
Its report said that value-added methods “should not be used as the sole or primary basis for making operational decisions because the extent to which the measures reflect the contribution of teachers themselves, rather than other factors, is not understood” [p.10]. Even the Wall Street Journal says that the results so far reflect a flawed statistical approach [21 August 2010].
However, under President Obama’s Race to the Top program of federal grants, several states are now proposing to make value-added a significant component of teacher evaluations. To be eligible for a piece of the $4.35 billion in competitive grants, states must prohibit laws barring a link between student scores and teacher evaluation and their bids for grants were scored in part on whether they evaluate teachers using test results.
Already, several states and school districts use value-added scores to determine which teachers should be rewarded. The Washington DC school district has taken an even harder line: in July, schools Chancellor Michelle Rhee fired 26 teachers based in significant part on their low value-added scores. This is the future for the US education system.
One thing that administrators and school principals in Los Angeles can expect in future is to be besieged with demands from parents for their children to be allocated high-scoring teachers. Any idea of allocating the best teachers to the lowest performing classes will be swamped by a scramble from parents trying to get the best advantage for their children. This is not a recipe for school improvement.