prior probability

Review of Chapter 5 of “Thinking in Bets” by Annie Duke

In our previous post, we described Annie Duke’s unorthodox suggestion to her readers in Chapter 4 of her book “Thinking in Bets.” To sum it up, if you want to improve your decisions, create your own personal “betting syndicate”; join a group of trusted intellectual friends who are willing to take each other’s bets. Building on the work of the legendary Robert K. Merton (pictured below), the next chapter (Ch. 5) identifies a set of fundamental betting norms–three normative values that you and the members of your betting syndicate should espouse and adhere to. (Duke discusses four values, but I have pared down this list to three.) Ordinarily, however, whenever we think of bets or betting syndicates, moral duties are rarely the first thing that comes to mind. One of Annie Duke’s contributions to the literature on probabilistic thinking is to identify a set of moral values that good bettors should live by.

First and foremost is the value of nuance (what Annie Duke calls “organized skepticism”), or in Duke’s own words (p. 68), “What if, in addition to expressing what we believe, we also rated our level of confidence about the accuracy of our belief on a scale of zero to ten?” By nuance, then, I mean that the fellow members of your personal betting syndicate should embrace uncertainty: they should not only be willing to take your bets; they should also require you to express your level of confidence in your bets. In other words, whether or not you believe that a particular proposition or given allegation is true, you should always strive to express the level of confidence or degree of belief you have in that proposition or allegation. For example, let’s say someone asks you whether the NBA or Major League Baseball will resume play by July 1. How strong or weak is your belief in this possibility? (Note: In addition to pages 169-171 of Chapter 5, pages 68-72 in Chapter 2 are also worth re-reading in this regard.)

Another essential betting norm is what I shall refer to as “the rule of three” or what Duke labels “disinterestedness” (see pp. 164-169). Ideally, your personal betting syndicate should contain at least three members: two to disagree and one to referee. This value is just a friendly reminder that motivated reasoning is not only a danger for individuals; it can also poison groups as well. The rule of three also makes good practical sense: every bet requires at least two parties: A, a bettor, i.e. the person who is placing the bet, and B, a bookie, the person who is taking the bet. In addition to these two main parties, it is also a good idea to appoint a neutral referee (a person with no stake in the outcome of the bet) to help write up the precise terms of the bet and to mediate any possible dispute about the bet that might occur later in time.

Yet another fundamental betting norm is “show your work.” As an aside, Duke refers to this value as “data communism” (see pp. 155-160), but communism has such an ugly connotation that I prefer to call it “show your work” or the “share and share alike” rule. Simply put, every member of your personal betting syndicate must not only disclose any information or evidence relevant to a possible bet; he must also disclose the source of that information or evidence. Put another way (in Kantian terms), you and the fellow members of your betting syndicate have a moral duty to share all relevant information with the entire group. In fact, it is actually in your self-interest to provide full disclosure. Why? Because whenever you make a bet, the more feedback you receive, the better bets you will make. A good bet, like a good decision, is an informed one, i.e. one based on all the available evidence.

This concludes my review of Chapter 5. We will turn to the last chapter (Ch. 6) and bring our review of “Thinking in Bets” to a conclusion in the next day or two …

Review of Chapter 4 of “Thinking in Bets” by Annie Duke

Thus far, we have seen how motivated reasoning impairs our ability to make good decisions (Chapter 2) and how self-serving bias impedes our ability to accurately assess the outcomes of our decisions (Chapter 3). The solution? Just transform your decisions into bets or wagers. Why is this such an ingenious solution? Because a bet doesn’t make motivated reasoning or self-serving bias go away; it makes these mental quirks work to your advantage! After all, when you make a bet, you want to win the bet, or as Annie Duke herself eloquently explains in Chapter 4 of her beautiful book (p. 136), “Evidence that might contradict a belief we hold is no longer viewed as hurtful a frame. Rather, it is view as helpful because it can improve our chances of making a better bet.”

Okay, but recasting a decision into a bet is easier said than done. How are we supposed to go about doing this? Chapter 4 of “Thinking in Bets” shows us how. In brief, in order to convert your day-to-day decisions into bets, you are going to need a bookie or bookmaker, i.e. someone who is willing to take your bets. So why not create your own “betting syndicate”? In other words, why not join a group of trusted intellectual friends who are willing to take each other’s bets?

Wait up! Do you actually know any open-minded and quirky friends who might be willing to take your bets? (And are you yourself willing to lay odds on the personal wagers of your friends?) If not, you may have to make some new intellectual friends! But where could you possibly find such a crazy group of wagering co-conspirators? Building on the work of the venerable Robert Merton, a great 20th-Century scholar whose ideas are still held in high esteem among most intellectual circles, Annie Duke will identify the essential features of a good betting syndicate as well as the essential personal qualities of a potential betting partner. We will turn to the legendary Robert Merton’s guide to intellectual life in my next post …

Who needs the NBA playoffs or Major League Baseball? More details about this new chess tournament are available here (via David Hill). Updated 4/24: see links below. 

1. Day 1 (4.18): Carlsen vs. Nakamura | Firouzja vs. Ding Liren
2. Day 2 (4.19): Caruana vs. Nepomniachtchi | MVL vs. Giri
3. Day 3 (4.20): Carlsen vs. Firouzja | Nakamura vs. Giri
4. Day 4 (4.21): Nepomniachtchi vs. MVL | Ding Liren vs. Caruana
5. Day 5 (4.22): Caruana vs. Carlsen | Firouzja vs. Nakamura
6. Day 6 (4.23): MVL vs. Ding Liren | Giri vs. Nepomniachtchi
7. Day 7 (4.24): Carlsen vs. MVL | Firouzja vs. Caruana

What if Hallmark made fancy sympathy cards that memorialized various types of academic failures? I am interrupting my extended review of “Thinking in Bets” (which will resume on Monday) to share the whimsical academic-failure sympathy cards pictured below. These were drawn by Tom Gauld, a Scottish artist and illustrator. My colleague and friend Richard Epstein, who infamously predicted only 500 coronavirus deaths in the USA, could use one of these!

Hat tip: @pickover

Review of Chapter 3 of “Thinking in Bets” by Annie Duke

As we saw in my previous post, Chapter 2 of “Thinking in Bets” is devoted to the problem of motivated reasoning. Next, Chapter 3 explores another form of mental mischief: the sin of “self-serving bias.” To simplify, whenever we place a bet or make any type of decision at time T1, two outcomes are generally possible at time T2: a good outcome or a bad one, depending on whether we win or lose the bet at time T2, i.e. depending on whether the decision goes our way. What “self-serving bias” does is to distort our ability to evaluate whether these possible outcomes were the result of our hard work and skill or the result of factors beyond our control, of just pure luck. Simply put, in the words of Annie Duke (p. 92), “we take credit when good things happen and deflect blame for bad things.” (For a visualization of self-serving bias, see image below.)

In reality, however, the outcomes of our decisions are rarely 100% luck or 100% skill. Instead, most outcomes are the result of both skill and luck, or to be more precise, the result of various degrees of skill and luck. Alas, given how entrenched self-serving bias is, what is to be done? How can we avoid or at least ameliorate this dangerous temptation, the sin of self-serving bias? The solution, once again, is to think in terms of bets, to reframe your decisions prospectively and retrospectively in probabilistic terms. How do bets neutralize self-serving bias? The same way they counteract motivated reasoning: by inviting us to engage in probabilistic and Bayesian reasoning, i.e. to update our beliefs based on the available evidence.

But this Bayesian or betting response to self-serving bias (and to motivated reasoning, for that matter) poses a new question of a practical or logistical nature–namely, how do we actually go about redefining or reframing our decisions into bets and wagers? We will review Chapter 4 of “Thinking in Bets” and address this key question in our next post on Monday, April 20.

Review of Chapter 2 of “Thinking in Bets” by Annie Duke

Chapter 2 of Annie Duke’s beautiful book contains so many powerful ideas that I cannot do it justice in a short blog post, but I will nevertheless try. Among other things, this chapter explores the ways in which our beliefs can disrupt or mess with our ability to make sound decisions. To summarize, our beliefs, even our most cherished ones, are dangerous in two different ways. One is belief formation: we often acquire our beliefs without testing their accuracy or probing their truth value. Worse yet, the other potential danger is inertia; that is, once we acquire a given belief, it becomes next to impossible to revise or modify. Put another way, beliefs tend to become fossilized or frozen in our minds, and this mental inertia is dangerous because it leads to motivated reasoning, or in the eloquent words of Annie Duke (p. 56): “Instead of altering our beliefs to fit new information, we do the opposite, altering our interpretation of that information to fit our beliefs.”

To sum up, motivated reasoning can impede the search for truth and impair our ability to make sensible decisions, but does this mental mischief have a viable solution? Is it really possible to stop motivated reasoning in its tracks or at least ameliorate this temptation? Here is where “Thinking in Bets” (literally!) comes into play. Simply put, Annie Duke’s brilliant solution to the sin of motivated reasoning is to reframe our decisions in probabilistic terms, i.e. convert our decisions into bets. Why wagers? Because bets or wagers encourage us to engage in Bayesian reasoning, to question or reconsider the accuracy of our beliefs, to calibrate or adjust our beliefs based on the available evidence. Ok, but how does the act of placing a simple bet produce such a skeptical or questioning Bayesian attitude? By forcing one to have “skin in the game,” so to speak. After all, a bet costs money, and no one likes to lose money, or as Annie Duke puts it (p. 43), “when you are betting, you have to back up your belief by putting a price on it.” (Or as my colleague and friend Alex Tabarrok is so fond of saying, “a bet is a tax on bullshit.”)

Alas, motivated reasoning is not the only form of mental mischief that deforms our good judgments and destabilizes our ability to make sound decisions. In addition to the sin of motivated reasoning, we must also beware the temptation of self-serving bias, as we shall see in our next post.

Review of Chapter 1 of “Thinking in Bets” by Annie Duke

Chapter 1 of “Thinking in Bets” concludes with this pithy phrase: “… you’ve been betting all along.” These five words not only capture one of Annie Duke’s main ideas; they also sum up why probabilistic thinking is so essential to every aspect of our lives. Simply put, the world is rarely black and white. Why? Because of the problem of incomplete information, which introduces various levels of uncertainty into almost every decision we make. Unlike a game of complete information like chess, where all the pieces are in plain sight and where the rules are well-defined and apply equally to both sides, real life strategic situations (such as business competition, romance, or career choices) are more like the game of poker, a game of incomplete information. In the popular Texas hold’em version of poker, for example, each player is dealt two private cards, while the distribution of the five community cards (such as those pictured below) is entirely probabilistic. (If you have never played poker before, here is a fun video tutorial via pokernews.com.)

As an important aside, there is another big reason why real life is less like chess and more like poker, i.e. a game involving various levels of uncertainty. Whenever we break a rule or attempt to engage in cheating in real life, there is some positive probability that we won’t get caught, or if we do get caught, that we won’t get punished. Sometimes the rules are unclear, or even if they are clear, they might be applied unevenly or unfairly for reasons beyond our control. The point is that the law itself is often highly probabilistic, so a decision to cheat or break the rules is very much like placing a bet. Although the particulars of each of our bets will depend on various factors, such as our level of uncertainty and the stakes involved, the main point Annie Duke is making here is that life is more like poker than chess: whenever we make a decision (big or small), we are in reality placing a bet on the future. Stay tuned; we will proceed to Chapter 2 in our next post …

Below is a small subset of the best books, essays, and online courses that I have been reading or watching during this quarantine period:

1. “Thinking in Bets” by Annie Duke (Portfolio/Penguin, 2019), available here via Amazon. Suffice it to say, I have now added Annie Duke to my pantheon of intellectual heroes. She was not only a PhD student at Penn; she is also a world-class professional poker player as well, and her beautiful book “Thinking in Bets” is one of the best works I have read thus far about the virtues of probabilistic thinking. This tome is so good that I will be writing up and posting on this blog a chapter-by-chapter review in the days to come. (I discovered this book by accident last month while I was working on another project. Thanks Google!)

2. “The Economics of Maps” by Abhishek Nagaraj & Scott Stern (Journal of Economic Perspectives, Vol. 34 (2020), pp. 196-221), available for free here via the American Economics Association (AEA). How much does it cost to make a map? Who pays for these costs, and how are they recovered? Among other things, this paper identifies five economic and institutional factors shaping the data and design choices made by mapmakers. Shout out to that polymath of polymaths, Tyler Cowen, for bringing this beautiful paper to my attention many months ago.

3. “Math on trial: how numbers get used and abused in the courtroom” (pictured below) by Leila Schneps & Coralie Colmez (Basic Books, 2013), available here via Amazon. This beautiful book–which contains ten chapters, each one devoted to a leading criminal or civil case involving the use (or misuse) of probability theory, both by prosecutors and plaintiffs as well as by defense attorneys–is dedicated “to all those who have suffered miscarriages of justices, and to all victims of crimes whose perpetrators went unpunished, due to the misuse or misunderstanding of [probability theory] in the legal process.” In other words, judges must avoid false positives (punishing the innocent) and false negatives (allowing the guilty to go unpunished). Among those cases dissected in this tome are the infamous Dreyfus Affair (a tragic example of a false positive) and the Amanda Knox trial (a false negative). Shout out to Professor Karl Schmedders for bringing this excellent book to my attention.

In addition to these readings, I have also enrolled in Professor Charles Fried’s online course on “Contract Law: From Trust to Promise to Contract” (via edX) to compare notes, so to speak. Professor Fried, who teaches contracts at Harvard Law School and who I had the pleasure of meeting in March of 2011 at Suffolk Law School, is one of my academic role models and intellectual heroes.