Alternative title: Is the World Truth League a Keynesian Beauty Contest? [Revised July 1, 2024]

Two of my previous posts (see here and here) have presented a tentative sketch of my colleague and friend Steve Kuhn’s original idea for a World Truth League. To recap: (1) The truth league would consist of a select number of forecast/hindcast teams competing against each other–as well as against A.I. agents–over forecasting questions regarding discrete future events and historical hypotheses regarding uncertain events from the past. (2) Teams would assign probability estimates to these future and past events and would be awarded or deducted points using a Bayesian scoring system. (3) Lastly, members of the public could place bets on their favorite questions and on their favorite teams during each truth round.

The main virtue of this blueprint is that it responds to Nick Whitaker and J. Zachary Mazlish’s recent critique of information markets. (See their excellent essay “Why prediction markets aren’t popular”, which was first published on May 17, 2024, and which I reviewed here.) To borrow Whitaker and Mazlish’s useful terminology, “gamblers” will love the fast-paced action of the World Truth League because the truth rounds would be short and all bets would be resolved instantaneously after each round; “savers” may love specific teams (e.g., the Microsoft Research team, the Mercatus Center team, the NY Times team, etc.), depending on their individual preferences and values; and the “sharps” will join once the the gamblers and savers are onboard.

Today, however, I want to consider one remaining objection to the World Truth League, a possible fatal one: the so-called beauty contest problem. (See the video below, for example. Also, shout out to my colleague and friend David Schraub, who first brought a game-theory variant of this problem to my attention after I presented an early draft of my paper on “Retrodiction Markets” at the Loyola Chicago Law School in November of 2022; see here.)

In summary, the great economist John Maynard Keynes introduced this problem in his classic study on macroeconomics, The General Theory of Employment, Interest, and Money (Keynes 2016 [1953]), where he compares  “professional investment … to those newspaper competitions in which the competitors have to pick out the six prettiest faces from a hundred photographs, the prize being awarded to the competitor whose choice most nearly corresponds to the average preferences of the competitors as a whole.” For Keynes, the optimal strategy in this particular newspaper game is not to pick the six faces that you, the contestant, may personally find the most attractive. To win, you must pick the faces that you think other people will find most attractive:

It is not a case of choosing those which, to the best of one’s judgment, are really the prettiest, nor even those which average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practise the fourth, fifth and higher degrees.

Similarly, the simple and elegant resolution mechanism of the truth league would be open to this same criticism. To put the “beauty contest” objection point blank, when truth teams are providing their truth estimates about the likelihood of X (a forecast about a future event) or Y (a hindcast about a past one), what are they really reporting? (A) Their first-order beliefs about the truth of X or Y, i.e. what each team really believes about the substance of the question presented during each truth round, or (B) their second-order beliefs about what they think the other teams believe about X or Y, i.e. their beliefs about the beliefs of the other teams? Unless they are reporting (A), their true beliefs about the substance of X or Y, there is no guarantee that the average of the truth estimates of the teams will converge on the “right” answer–that is, on the true value of X or Y.

Is the beauty contest problem soluble? Two extreme solutions come to mind. One is outright disqualification. The truth league could disqualify teams that consistently engaged in second-order, beauty-contest reasoning. This draconian solution, however, goes against the authors’ “classical liberal” values: as a general rule, teams should be free to engage in whatever strategy they wish to pursue. (As an aside, we prefer the label “classical liberal” over the term “libertarian” to emphasize our admiration and affinity with such first-generation liberals like Adam Smith and David Hume.)

The other solution is to leave well enough alone, i.e. do nothing. (Cf. Coase 1960, p. 18: “There is, of course, a further alternative, which is to do nothing about the problem [of harmful effects] at all.”)The problem will go away with a sufficient number of truth teams. After all, with our proposed scoring system, a team will win points not only when its truth estimate is correct; it will win even more points the more confident it is in its truth estimate. Also, Keynes’s beauty contest was originally meant as a psychological critique of stock markets, but despite Keynes’s critique, stock markets still attract a sufficient number of gamblers, savers, and sharps to work in practice. Why wouldn’t a well-designed World Truth League with an enticing roster of teams be any different?

I will conclude my series on the World Truth League next week with a few additional observations and a call to action …