Waiting for superman

Wow, three in a row and not one Jackass, I must be slipping. Anyway Waiting for Superman is a documentary about the decaying school system in our country, and follows around five students who are trying to get win a lottery to enter high performing charter schools. Let me say upfront that I think the film does a decent job laying out the big issues in education right now, and I think it'll be an important conversation starter. I also generally agree that many of the problems described are valid (tenure starting to early, no way to pay teachers base don merit, union and bureaucratic inflexibility, etc) but there big solution of expanding the charter school model around the country has one big flaw the film doesn't acknowledge. Basically all the schools they hold up as models suffer from self-selection bias. Basically since its so hard to get in all the students there (and more importantly the parents) are motivated to do the work and succeed. Therefore its unsurprising that the test scores are higher since these are precisely the students that were probably going to be higher achievers regardless of situation. Now I don't doubt at all that the innovations at the schools help them channel their efforts more effectively, and could no doubt help other schools. I just think its dangerous to assume something is the magic bullet when it succeeds with a very select sub sample. I know this is somewhat missing the point, and obscures my general agreement with the film's message. But it is not an immaterial concern and one I think the filmmaker should have acknowledged more explicitly.