We finally got around to reading Charles Wheelan’s 250+ page defense of frequentist methods in his 2013 book “Naked Statistics.” (Curiously, his book was published a year after Nate Silver best-selling tome “The Signal and the Noise,” a book that criticizes many of the statistical methods described in “Naked Statistics.”) Although “Naked Statistics” is “sparkling and intensely readable” (to quote from one of the many positive reviews of his book), our overall verdict is go read Nate Silver’s book instead. Consider this clumsy hypothetical on pp. 127-128 of Wheelan’s book, the case of the missing marathon runners (edited by us for clarity):
“Suppose you live in a city that is hosting a marathon [and an International Sausage Festival] * * * [Marathon runners and sausage munchers] are randonmly assigned to buses * * * Unfortunately, one of the buses [full of marathon runners] gets lost * * * As luck would have it, you stumble upon a broken-down bus near your home * * * This must be the missing bus! * * * Except you have one lingering doubt … the passengers on this bus are, well, very large. Based on a quick glance, you reckon that the average weight for this group of passengers has got to be over 220 pounds. There is no way a random group of marathon runners could all be this heavy.”
So, is the broken-down bus full of marathon runners or sausage munchers? Put another way, is the broken-down bus the missing bus?
The problem with this particular “shaggy-dog story” is that it’s not just one isolated or ill-chosen illustration in an otherwise well-written and thoughtful book. It’s Wheelan’s showcase — the silly and unrealistic example he returns to time and time again throughout his book — in defense of traditional frequentist methods — i.e. statistical significance, confidence intervals, p values, etc., etc. — methods that have already been thoroughly discredited by many others.
Worse yet, the missing-bus example is symptomatic of two larger problems with Wheelan’s book. One is that he equates science with long-since discredited frequentist methods. The other is that he keeps talking about probability, and yet, there is not a single reference to Bayesian methods in his entire tome. What’s up with that?
What happened to Rev. Bayes?
Let’s return to Wheelan’s missing-bus hypothetical, shall we? In the real world, if a bus full of marathon runners were to really go missing, and if you were to find a broken-down bus, you would not need to engage in a time-consuming and tedious analysis of sample sizes and standard deviations — unless you had unlimited time and resources … or an NSF grant! Instead, you would immediately look for relevant clues and update your prior beliefs accordingly. The weight and physical condition of the passengers on the missing bus are two such clues, but so are the location of the bus and the attire of the passengers — are they all wearing sneakers or carrying gym bags, for example?
Fundamentally, Wheelan confuses probabilities. The relevant probability is not whether the average weight of all the passengers on the broken-down bus are within one or two standard deviations of some mythical mean. That is a clumsy, indirect, and easily manipulable method of performing the true task at hand, for the relevant probability we are trying to measure is whether the broken-down bus is the missing bus! In brief, where is the missing bus? Bayesian reasoning offers a more direct and reliable way of guessing this probability.
Although Wheelan should know better — after all, he devotes considerable time and space describing all the potential problems and pitfalls with frequentist methods, especially in Chapters 10 and 12 of his book — in the end, his ill-fated faith in frequentism and linear regressions appears to be unshaken. Maybe next time, Professor Wheelan will consider writing another book — “The Naked Reverend” … Rev. Bayes, that is!