prior probability

Are you tired of being lied to by op-ed pundits and TV talking heads? Would you like to see their opinions put to the test? Then join the World Truth League! You can join a team and make forecasts or hindcasts, which will then be aggregated over multiple rounds until a consensus emerges. More details are available in our short (16pp.) white paper/prospectus; in the meantime, join the world truth league waitlist here: https://worldtruthleague.com

A bunch of these silly GPT-5 memes popped into my Twitter feed last month, so I thought I would pass one along. Also, happy 11th anniversary to my blog!

🇺🇸Happy 4th of July🇺🇸 Following up on my previous post, Steve Kuhn, myself, and several others have been brainstorming a new type of second-generation information market: the World Truth League. (Here is a link to our 16-page white paper, which contains a tentative sketch of our idea, including a one-page prospectus/executive summary.) So, how would our proposed truth league keep score? In brief, we propose a simple Bayesian scoring system in which teams would have an opportunity to update their beliefs. In brief, our scoring system consists of three discrete steps, which are further described below the fold, and can be summed up in three words: wash, rinse, and repeat.

Check out this enchanting piano version of the classic late 70’s song “September” from the 2023 animated Spanish film Robot Dreams. My daughter Adys Ann and I were finally able to catch this little gem of a movie on the silver screen the other day, and among other things, both of us loved this piece. We even patiently but willingly sat through the closing credits just to hear it again and see who composed it: Catalan singer-songwriter Alfonso de Vilallonga. Bonus links: here is his homepage, and here is his Instagram.

Via one of my favorite specialty blogs, “Remember Singapore“: https://remembersingapore.org

Originally called the Hill Street Police Station and Barracks, the majestic six-storey Neoclassical-style colonial building was Singapore’s largest …

Old Hill Street Police Station, the Iconic Colourful Landmark by the Singapore River

Alternative title: Is the World Truth League a Keynesian Beauty Contest? [Revised July 1, 2024]

Two of my previous posts (see here and here) have presented a tentative sketch of my colleague and friend Steve Kuhn’s original idea for a World Truth League. To recap: (1) The truth league would consist of a select number of forecast/hindcast teams competing against each other–as well as against A.I. agents–over forecasting questions regarding discrete future events and historical hypotheses regarding uncertain events from the past. (2) Teams would assign probability estimates to these future and past events and would be awarded or deducted points using a Bayesian scoring system. (3) Lastly, members of the public could place bets on their favorite questions and on their favorite teams during each truth round.

The main virtue of this blueprint is that it responds to Nick Whitaker and J. Zachary Mazlish’s recent critique of information markets. (See their excellent essay “Why prediction markets aren’t popular”, which was first published on May 17, 2024, and which I reviewed here.) To borrow Whitaker and Mazlish’s useful terminology, “gamblers” will love the fast-paced action of the World Truth League because the truth rounds would be short and all bets would be resolved instantaneously after each round; “savers” may love specific teams (e.g., the Microsoft Research team, the Mercatus Center team, the NY Times team, etc.), depending on their individual preferences and values; and the “sharps” will join once the the gamblers and savers are onboard.

Today, however, I want to consider one remaining objection to the World Truth League, a possible fatal one: the so-called beauty contest problem. (See the video below, for example. Also, shout out to my colleague and friend David Schraub, who first brought a game-theory variant of this problem to my attention after I presented an early draft of my paper on “Retrodiction Markets” at the Loyola Chicago Law School in November of 2022; see here.)

In summary, the great economist John Maynard Keynes introduced this problem in his classic study on macroeconomics, The General Theory of Employment, Interest, and Money (Keynes 2016 [1953]), where he compares  “professional investment … to those newspaper competitions in which the competitors have to pick out the six prettiest faces from a hundred photographs, the prize being awarded to the competitor whose choice most nearly corresponds to the average preferences of the competitors as a whole.” For Keynes, the optimal strategy in this particular newspaper game is not to pick the six faces that you, the contestant, may personally find the most attractive. To win, you must pick the faces that you think other people will find most attractive:

It is not a case of choosing those which, to the best of one’s judgment, are really the prettiest, nor even those which average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practise the fourth, fifth and higher degrees.

Similarly, the simple and elegant resolution mechanism of the truth league would be open to this same criticism. To put the “beauty contest” objection point blank, when truth teams are providing their truth estimates about the likelihood of X (a forecast about a future event) or Y (a hindcast about a past one), what are they really reporting? (A) Their first-order beliefs about the truth of X or Y, i.e. what each team really believes about the substance of the question presented during each truth round, or (B) their second-order beliefs about what they think the other teams believe about X or Y, i.e. their beliefs about the beliefs of the other teams? Unless they are reporting (A), their true beliefs about the substance of X or Y, there is no guarantee that the average of the truth estimates of the teams will converge on the “right” answer–that is, on the true value of X or Y.

Is the beauty contest problem soluble? Two extreme solutions come to mind. One is outright disqualification. The truth league could disqualify teams that consistently engaged in second-order, beauty-contest reasoning. This draconian solution, however, goes against the authors’ “classical liberal” values: as a general rule, teams should be free to engage in whatever strategy they wish to pursue. (As an aside, we prefer the label “classical liberal” over the term “libertarian” to emphasize our admiration and affinity with such first-generation liberals like Adam Smith and David Hume.)

The other solution is to leave well enough alone, i.e. do nothing. (Cf. Coase 1960, p. 18: “There is, of course, a further alternative, which is to do nothing about the problem [of harmful effects] at all.”)The problem will go away with a sufficient number of truth teams. After all, with our proposed scoring system, a team will win points not only when its truth estimate is correct; it will win even more points the more confident it is in its truth estimate. Also, Keynes’s beauty contest was originally meant as a psychological critique of stock markets, but despite Keynes’s critique, stock markets still attract a sufficient number of gamblers, savers, and sharps to work in practice. Why wouldn’t a well-designed World Truth League with an enticing roster of teams be any different?

I will conclude my series on the World Truth League next week with a few additional observations and a call to action …

You’re welcome!