In our previous post, we summarized some of the main ideas we stole from Tetlock and Gardner’s fascinating book on “superforecasting.” Their book is really useful because it is a kind of “how to” manual for Bayesian thinking. In fact, it is the first book I’ve read that shows you how to “be like Bayes”–how to apply Bayesian methods to real world decision problems. For the benefit of my loyal followers, I will summarize Tetlock and Gardner’s Bayesian methodology below.
Step 1. Formulate a testable problem. The problem must have an answer, and I will henceforth refer to the answer of the problem as “x” because the answer is unknown until the event occurs.
Step 2. Think probabilistically about x, the answer to the problem. Think of x as a prediction or as a “scorable” guess. Your answer may be correct (or incorrect) with some positive probability between 0 and 1. In plain English, “maybe” is usually the most correct answer as to whether x is true or not, but as Tetlock and Gardner note, there are “many grades of maybe” (p. 141) or “units of doubt” (p. 168), so you should not only make a guess about x, you should also observe how strong or weak your belief in x is.
Step 3. Test your guess. How does one go about testing a guess? We will need to subdivide this step into smaller ones as follows:
- Step 3a. Figure out the base rate. The base rate is “how common something is within a broader class.” (Tetlock & Gardner, 2015, pp. 117-118.) So, to think probabilistically about a problem (see step 2 above), ask yourself: “How often do things of this sort happen in situations of this sort?” (Ibid., p. 279.) Congratulations! That is your “prior”!
- Step 3b. Restate the problem in Bayesian terms by asking more questions. In the words of Tetlock & Gardner (p. 111): “What information would allow me to answer the question?” In other words, what evidence do you need to see for x to be true, or how likely is it that x would be true if I saw such-and-such evidence? (I will elaborate on this crucial question-asking procedure in my next post.)
- Step 3c. Make plausible guesses about the correct answers to these other questions in Step 3b and then revise your guesses whenever you obtain new evidence or information that is relevant to any of these questions.
I will elaborate on and express all these steps in more formal Bayesian parlance in my next post.