21 December 2010

Thoughts on Hanushek's study of teacher value study

Greg Mankiw's Blog: The Value of Good Teachers

From Eric Hanushek:
A teacher one standard deviation above the mean effectiveness annually generates marginal gains of over $400,000 in present value of student future earnings with a class size of 20 and proportionately higher with larger class sizes. Alternatively, replacing the bottom 5-8 percent of teachers with average teachers could move the U.S. near the top of international math and science rankings with a present value of $100 trillion."
There is a lot of dislike in the educational establishment for programs like Teach for America and Alliance for Catholic Education, much of which is justified and some of which I think is bitterness insiders always feel for those who bypass credentialist regimes.

Some of what frustrates me about such discussions is that it is too often framed as "Are these untrained college kids good teachers?" where goodness is measured on some absolute scale and not "How good are they compared to the marginal teacher?" Are they better than the alternatives -- the credentialed person who is displaced or leaving the position vacant?

If the bottom 5-8% of teachers are as bad as Hanushek says then it becomes a little more likely that a slightly above average college graduate without training is better at the margin.

(Although I suppose that is only true if the hiring process of the school for the "regular" teachers is somewhat efficient.  If they hire completely randomly then the TFA or ACE kid is as likely to be displacing one of the over performing teachers as an under performing one, which makes TFA & ACE worse.  But if that's the case, we have a lot bigger problems than TFA and ACE.)

~ ~ ~

Adam Ozmiek is right on:
There are really two important claims here. I think progressives tend to be very pleased with claims like the first one, which is that teachers have a very high value. You can find similar results in the work of Raj Chetty, which suggests that good kindergarten teachers are worth $320,000. If this is true then the marginal benefit of teaching skill -or quality, if you want to think of it that way- is far below the marginal cost, and therefore we should increase wages to draw more talented teachers.

However, the second claim is just as important and is suggested by, although not a necessary condition of, the first: if good teachers are very valuable, then bad teachers are very costly. This means we should be willing to pay more for good teachers, but it also increases the benefit of getting rid of bad teachers and ensuring we have a system that can do that. After all, every dollar spent on a bad teacher has the high opportunity cost of good teachers.
I would tweak the end of each of those paragraphs a bit. Rather than increasing wages to draw more talented teachers, I would shift the focus to increasing pay for talented teachers. How we determine who is talented is of course a thorny issue.

I would also expand on the last sentence of the latter 'graph. The cost of bad teachers is not only the opportunity cost of their salaries, but the opportunity cost of the man-hours students must spend with them. I would put more stress on that than the salaries. Those are hours students do not get back, and since the number of hours you spend in primary and secondary education is fixed, you're directly displacing hours spent with good or even average teachers.

~ ~ ~

I don't feel like searching for the research now, but I recall seeing that once you get class size under a certain threshold (in the upper 20's I believe) the impact of class size is near negligible. I conclude from this that you might not only be better off firing the bottom 5% of teachers and replacing them with a teacher with an average expected value, but you may also be better off firing them and not replacing them at all. 100 teachers -- including 5 from the far left tail of the distribution -- with 20 kids each may very well be worse than 95 teachers with 21 kids each.

It's a hypothesis worth investigating anyway.

~ ~ ~

Contrast Hanushek's observation with two bullet points from my Waiting for Superman review:
• I posted a couple of weeks ago this stat from the movie, which remains shocking:
About one of 57 medical doctors and one of 97 lawyers loses his or her license annually for malpractice. In contrast, only one in 2,500 unionized, public school teachers with tenure gets fired each year.
• A similar stat also stood out. I don't recall the exact figures, but of the several hundred school districts in Illinois, only 60 or so have ever attempted to fire a single tenured teacher, and of those only half have succeeded. That is absurd. There is no organization in the world that can hire tens of thousands of people without making a mistake and ending up with someone who is not right for the job. You can't run an organization if you assume that once someone has made the decision to hire X, that decision can never be revisited or reevaluated.
Those bottom five to eight percent are not being culled despite their destructivity.

~ ~ ~

Hanushek has been interviewed by Russ Roberts for EconTalk several times. He is worth listening to.


  1. This is a thorny issue and I agree with most of what's here. I differ a little on how we should interpret that last quote. I agree that inability to fire ineffective teachers is a problem we need a solution to, but the way to think about the scale of that problem is not to compare teachers to doctors and lawyers that lose their licenses. The problem is that doctors and lawyers face the removal of their licenses due to acute moments of poor service which puts their inadequacies in high relief.

    In contrast the effectiveness of a teacher is harder to measure and is complicated by the notion that a teaching strategy that might be ineffective in aggregate is actually highly effective for certain students. What do we do with that information? Clearly we can't structure the nation's educational system so that just the right students find their way to the teachers most effective for them. But I also don't know if it's the wisest course to label them as teaching incompetents either.

    And the professionalization of teachers is not as complete as it is for doctors and lawyers. Certainly lawyers don't have to go through an apprenticeship in order to practice, but they often go through one of sorts--working as a research grunt inside of a law firm or in a corporation's legal dept before they are actually responsible for someone's case. And death sentence lawyers do go through an apprenticeship. Doctors, of course, go through their residency program. We might be better off demanding that the student teaching experience is longer and more capable of weeding out ineffective teachers earlier on--before they're unionized. But that's a different post.

  2. I agree this is a whole big ball of complication.

    I wouldn't want to treat teachers exactly like lawyers or doctors or any other job. I don't want to have the percentage of defrocked teachers match those of some other profession. But I would feel better if I got the impression there was effort being made (by all parties involved) to remove the below average teachers. I don't see that happening.

  3. PS I don't have any data, but it wouldn't surprise me if the few tenured teachers fired are also dismissed after acute screw-ups rather than a history of continuous sub-par performance.

    Maybe I'm swayed too much by anecdote, but it seems more common for a teacher to be fired after saying or doing something wildly inappropriate rather than for simply failing to improve students understanding of the material consistently.

  4. This comment has been removed by the author.

  5. Yeah but those acute episodes are not normally of teacher-malfeasance specifically but rather some massive extra-teacherly idiocy: taking ones pants off in the classroom, dating a student, throwing a chair at the dean.

    ...not that a lack of morals and/or a failure to comply to a professional ethical code shouldn't be grounds for firing, but they don't necessarily say anything about the teaching skill set specifically.