How rigid is the IMPACT bell curve?

On Friday, DCPS announced performance ratings for employees under the IMPACT evaluation system, now two years old.

Most of the headlines have surrounded the firings of 413 employees (288 for poor performance), but little has been said about the distribution of those scores.

We compared the ratings distribution of this year to last year, noticing something very interesting.
No, look again, there are TWO lines there. The distribution of 2011 DCPS teacher evaluation scores nearly mirrors last year’s distribution. What does this mean?

One of DCPS’s purposes with IMPACT is to move away from the old system in which a teacher is rated “meets expectation” or “does not meet expectations,” with no way to differentiate performance within those categories. Under the old system, most teachers are effective and meeting expectations—it’s like Lake Wobegon where all children are above average. IMPACT works to pull the really outstanding—or “highly effective”— teachers out of that category for recognition. It also aims to differentiate the minimally effective teachers who, with the proper support, could improve their performance.

The IMPACT scores are designed to, and do, indicate a broader range of teacher performance, with a distribution curve where most teachers are in the middle, with fewer on each tail. But how flexible is the distribution of IMPACT scores on a year to year basis? We only have two years of data, so it’s hard to draw any conclusions on trends, but the distribution overlap between 2010 and 2011 is remarkable.

Teachers’ IMPACT ratings are individual scores composed of scores from classroom observations, school involvement, student achievement growth, and whole school achievement growth. For teachers in grades with standardized tests, the student achievement growth makes up half of their final IMPACT scores. A teacher’s IMPACT score translates into one of the general categories of effectiveness.

Are teachers getting the same scores as last year? Not necessarily. There appears to be some movement within the distribution. Only 290 Washington Teacher’s Union members were rated highly effective two years running (out of 663 rated highly effective each of the two years), and 175 employees earned a minimally effective score both years.

In future years, will we see that bell curve move more to the left, as teachers improve and more are rated highly effective?  One would hope, but it remains to be seen.