Freakonomics by Steven Levitt contains some good true-life statistical detective stories. Levitt has tracked down teachers in the Chicago public school system who fake their student’s results on end-of-year achievement tests; he even got some of them fired. He has identified match-fixing practices among the elite in Sumo wrestling. He bravely criticizes the Ku Klux Klan. But when it comes to taking on more powerful foes, he suddenly develops a selective blindness and fails to see anything to question about the economics of compulsory state schools, drug prohibition, gun control, or any of America’s other herds of sacred buffalo.
His most valuable work is his unmasking of cheating teachers. He examined 700,000 sets of test answers, searching for the telltale pattern of lazy cheating: classes where every student miraculously got the same fifteen answers correct in a row. (In one example case, all the students then got the next question wrong… apparently the cheating teacher couldn’t answer the question correctly either). In follow-up checks, he readministered the tests to 120 classes, finding that the scores were significantly lower than when the teachers monitored the tests.
It is useful for parents to know that a "high-scoring" public school may simply be one that cheats more. It would be even more useful in an economics book for the author to question the economics and incentives of compulsory public schools; unfortunately Levitt declined to do so.
I will pass over his excellent analysis of Sumo match-fixing; a world with morally pure Sumo wrestlers might be a better world, but I don’t think I would notice the difference in my daily life.
He starts to veer away from the data and back into conventionality when dealing with the socially pressing issue of racism among users of online dating services. He concludes that white men are racist because "they sent 90 percent of their e-mail queries to white women." Is it possible that more white women than black used Internet dating services at that time? Is it possible that the men were discriminating on economic or cultural grounds? We can’t tell from the data, so apparently we are just supposed to assume that white men are bigoted.
The chapter on crack dealers claims to have detailed information of the account books of a large gang in Chicago. (Hint to grad students: this is a great idea for keeping your advisor from checking your work; he’s not going to go into the projects and look at the books of any drug gangs). While a business case study of an illegal drug gang may have merit, it is useless without comparison to legal drug sales. Nicotine is the most addictive drug, yet there are no shootouts associated with its use. Levitt never deals with the issue of drug prohibition itself, so the chapter is trivial and superficial.
Once he get into gun control, the scientific method goes out the window. He does admit that gun control programs are not associated with lower crime. However, the pervasive assumption throughout the chapter is that "people don’t kill people, guns kill people." To quote: "The typical gun buyback program yields fewer than 1,000 guns — which translates into an expectation of less than one-tenth of one homicide per buyback." He means, of course, a reduction of less than one-tenth of one homicide per buyback, but the actual text is closer to the truth. Buyback users aren’t random! They do not include the set of premeditated murderers! Buybacks, like the US "nuclear nonproliferation" policy, only disarm the peaceful.
The same error continues into the chapter on child safety. He points out that 550 children die in swimming pools every year, while only 175 are shot to death. He then extrapolates that the risk of allowing a child to play where "the parents keep a gun in the house" is 100 times less than the risk of allowing them to go to a house with a pool. This assumes that all children are killed with guns that are owned by their friend’s parents; a highly questionable assumption. Is it not possible that guns in the hands of parents might actually keep the children safer? Levitt doesn’t make the effort to ask.
So, is this a terrible book? No. Within the permissible limits of conventional wisdom, there are some thought-provoking paragraphs. But it is sad that someone can think so clearly and work so tirelessly when weeding out bad public-school teachers and Sumo wrestlers, but find it too frightening to expose the economic damage done by the privileged and powerful cheats of all kinds.
