Beliefs are hypotheses to be tested, not treasures to be guarded

That is my favorite quote from “Superforecasting: the art and science of prediction” by Philip Tetlock and Dan Gardner, which I finally got around to reading during this Christmas holiday. Along with Sharon McGrayne’s 2012 book “The theory that would not die” and Nate Silver’s 2015 book “The signal and the noise,” Philip Tetlock and Dan Gardner’s excellent Superforecasting book are must reads for anyone interested in how to improve hypothesis-testing in science or how to improve decision-making in such fields as business, politics, and law. (This trifecta of Bayesian tomes is pictured below.) Below are some of the main ideas I have stolen from Tetlock and Gardner’s beautiful book:

Science is not about certainty or facts; science is about probability. Or about “testability,” to be more precise: “… in science, the best evidence that a hypothesis is true is often an experiment designed to prove that the hypothesis is false …. Scientists must be able to answer the question: ‘What would convince me I am wrong?’” (p. 38) By contrast, “Fuzzy thinking can never be proven wrong.” (p. 252)
Beliefs should be testable. In the eloquent words of Tetlock and Gardner, “Beliefs are hypotheses to be tested, not treasures to be guarded.” This is why so many legal and political debates are such a supreme waste of time, or in the words of Tetlock and Gardner (p. 268): “These are accomplished people debating pressing issues, but nobody seems to have learned anything beyond how to defend their original position.” In short, beliefs must be tested because even the firmest belief may be wrong.
Bayesian methods can be used to test beliefs. As Tetlock and Gardner show, so-called “superforecasters” are just Bayesians by another name. They test their beliefs by restating those beliefs in the form of predictions and they attach a numerical value to express their degrees of belief in their predictions. These Bayesian ideas are so important and so useful in so many different domains, I will devote my next few posts expounding on the Bayesian ideas in Tetlock and Gardner’s book.