prior probability

I have been meaning to reblog this post from my fellow francophile and friend Sheree, so here it is!

Happy Pi Day 3.14! I am happy to report that I was invited to attend a research colloquium on “Data Law & AI Ethics” at the University of Pennsylvania (Wharton School) this weekend, where I will be presenting my work-in-progress on “Self-Regulation, ‘Constitutional A.I.,’ and Gödel’s Loophole.” (FYI: A draft of my paper is available here. I will post an updated version after I digest the feedback I receive at this weekend’s colloquium.) Below the fold is the full colloquium agenda:

I will conclude my survey of Part 1 of David Hume’s famous essay “Of Miracles” by proposing a better approach to the problem of miracles: that of Frank Ramsey and Bruno de Finetti.

To the point, Ramsey and de Finetti, building on the ideas of Thomas Bayes and Richard Price, developed a subjective theory of probability (see here and here). Broadly speaking, subjective probability represents an individual’s personal assessment of, or “degree of belief” in, the likelihood of an event happening or the truth of a proposition or hypothesis, e.g. that a given miracle really happened. In other words, whenever we estimate the probability of some event or proposition, our estimate is almost always based on our personal perceptions and experience, i.e. on gut feelings instead of formal calculations. So why not extend the concept of subjective probability as well as Bayesian reasoning more generally to reports of miracles?

Before proceeding any further, two digressions are in order. One is the relationship between subjective probability and political philosophy, for what I find most compelling about the idea of subjective probability is that it is consistent with the classical liberal ideal in favor of natural liberty: people have different beliefs about the world, and those beliefs should be tolerated — nay, celebrated! — so long as no one is harmed. My other digression is that one does not have to express one’s subjective probability or degree of belief in a miracle using numerical values. (See here, for example. See also John Maynard Keynes, Treatise on probability, Macmillan (1921), pp. 20-22.) It is sufficient to use such qualitative formulations as “highly likely, almost certain, or virtually impossible” to describe the likelihood of some event or proposition, given a body of evidence. (See James Franklin, “The objective Bayesian conceptualisation of proof and reference class problems,” Sydney Law Review, Vol. 33 (2011), pp. 545-561, especially pp. 547-548.)

In any case, once we accept the subjective nature of probability, we can easily apply Bayesian reasoning to miracles as follows: given a report of miracle M, you should assign a subjective probability value to the likelihood, however remote, that M really took place, and then you should incrementally update your subjective prior up or down as new evidence about M becomes available. (Moreover, my Bayesian approach to miracles is especially apt given that Bayes’s famous theorem may have originated in response to Hume’s argument against miracles!) For a step-by-step explanation of Bayesian reasoning, see my 2013 paper “Visualizing probabilistic proof” or my 2011 paper “A Bayesian model of the litigation game“. For now, however, I will conclude with one final observation: Bayes > Hume.

Thus far, I have surveyed “the good” and “the bad” sides of David Hume’s famous argument against miracles (see here and here). That leaves “the ugly”: Hume’s circular definition of what a miracle is.

To the point, for Hume “[a] miracle is a violation of the laws of nature” (Hume, Of Miracles, para. 12; cf. Voltaire 1764/1901, p. 272). Alas, the problem with Hume’s definition is that it is circular! Why is this definition a circular one? Because unlike human laws, which are defied and disregarded all the time, “laws of nature” consist of patterns or regularities that can never be set aside or suspended. In other words, a violation of a law of nature has, by Hume’s own definition, a probability of zero!

To see why, ask yourself two deeper questions: (1) what is a “law of nature” (see here, for example), and (2) what does it mean to “violate” or disobey such a thing? Although Hume himself uses the term “laws of nature” in several different senses (see here), at a minimum a law of nature in the traditional Newtonian sense (see, e.g., “Newton’s Three Laws of Motion“) generally refers to some uniform or regular pattern of behavior. On this view of “natural law”, a miracle must consist of an unforeseen departure from or an unexpected interruption of such a pattern.

By way of illustration, consider Newton’s First Law: “Corpus omne perseverare in statu suo quiescendi vel movendi uniformiter in directum, nisi quatenus a viribus impressis cogitur statum illum mutare.”(Translation: “Every object perseveres in its state of rest, or of uniform motion in a right line, unless it is compelled to change that state by forces impressed thereon.”) In plain English, this law tells us that objects tend to “keep on doing what they’re doing” until they are acted upon by an outside force. The dirty clothes in your hamper, for example, will remain there until someone grabs them and throws them in the washing machine.

Maybe you agree with Hume’s circular definition of miracles. After all, what if your clothes did wash themselves? That would be a true miracle! But from a purely logical perspective, the problem with Hume’s definition is that he’s rigged the game ex ante. To see why, recall Hume’s two-part probability test for miracles: 1st, given some evidence that a miracle took place, Hume would have us assign two separate probability values to the evidence — the probability p₁ that the miracle really happened and the probability p₂ that the evidence is either mistaken or fraudulent or otherwise defective — and 2nd, Hume would have us compare p₁ and p₂: you should believe in the miracle only if p₁ > p₂.

The problem, however, is this: under Hume’s natural-law-violation definition of miracles, p₁ = 0, no amount of evidence of whatever kind would ever be sufficient to establish the occurrence of a miracle, a truly dogmatic and closed-minded position to take, especially when we have so many reports of miracles from so many different sources! Nevertheless, that said, it takes a theory to beat a theory, so in my next post, I will present an alternative method for evaluating incredible claims of miracles.

***

As I mentioned at the end of my previous post, David Hume’s argument against miracles has two big blind spots. One is the unknown probability problem; the other is the reference class problem. To appreciate the significance of these logical troubles, recall Hume’s two-part probability test for miracles: 1st, given some evidence that a miracle took place, Hume would have us assign two separate probability values to the evidence — the probability p₁ that the miracle really happened and the probability p₂ that the evidence is either mistaken or fraudulent or otherwise defective — and 2nd, Hume would have us compare p₁ and p₂: you should believe in the miracle only if p₁ > p₂.

But what if the probability values for p₁ and p₂ don’t add up to 1? By way of illustration, what if p₁ = 0.01 and p₂ = 0.1? (Or in plain English, what if you believe there is only a 1% chance that the miracle in question really took place, and at the same time, you also believe there is only a 10% chance the evidence of the miracle is reliable?) In this case, what do we do with the missing 89% of the probability space in this example, i.e. the “unknown probability”?! Alas, Hume does not say, and Hume’s silence, in a nutshell, is what I mean by the unknown probability problem.

The reference class problem, however, is an even bigger problem for Hume, for how do we figure out what numerical values to assign to p₁ and p₂ in the first place? Stated formally, in deciding whether a particular miracle X is real or not, what is the reference class of X? After all, X could be a member of a wide variety of classes or groups, and the probability of X will thus differ depending on how the class or group of X‘s is defined. (See, e.g., James Franklin (2011), The objective Bayesian conceptualisation of proof and reference class problems, Sydney Law Review, Vol. 33, pp. 545–561.)

Airplane crashes, for example, have been in the news lately, especially since the fatal midair collision at Reagan National Airport on January 30th of this year, but at the same time, we are also being told that aviation safety is at an all-time high. This disparity between the number of news reports and the true level of aviation safety provides a textbook example of the reference class problem, for to estimate the probability of an aviation disaster, one might use the frequency of crashes of all aircraft (civilian and military), the frequency of crashes of a particular model or type of aircraft (e.g. helicopters), or the frequency of airplane accidents in the last five, ten, or y years.

To recap, Hume’s argument against miracles is caught between the Scylla of the unknown probability problem — i.e. the possibility that p₁ and p₂ do not add up to 1 — and the Charybdis of the reference class problem — i.e. the difficulty of deciding what class or group of events to use when calculating p₁ and p₂. Can either of these Humean probability monsters be tamed? Stay tuned, for before I try to rescue Hume from this metaphorical Strait of Messina, I want to take the Scottish essayist to task for his circular definition of “miracles”.

As I mentioned in my previous post, this week I will offer my own lawyerly take on David Hume’s influential argument against the possibility of miracles: the good, the bad, and the ugly. Let’s start out on a positive note, shall we, with “the good”? (Rest assured, I will consider “the bad” and “the ugly” sides of Hume’s anti-miracle essay in subsequent posts.)

Simply put, what I like the most about Hume’s argument is his lawyerlike and probabilistic approach to the problem of miracles. For Hume, whenever you read or hear a report of a miracle or some other remarkable event — whether it be a UFO sighting, a levitating saint, or Lazarus’ resurrection from the dead — you should weigh the evidence based on your own experience and common sense. Specifically, Hume proposes a two-part probabilistic test for weighing the evidence.

In summary, the first part of Hume’s probabilistic test requires us to assign two separate probability values to the evidence as follows: the probability p₁ that the miracle really happened and the probability p₂ that the evidence is either mistaken or fraudulent or otherwise defective. The second part of Hume’s test is to compare p₁ and p₂. To the point, you should believe in the miracle only if p₁ > p₂.

Moreover, Hume’s exhortation to weigh the evidence is exactly what juries do when deciding law cases. In fact, Hume’s simple probabilistic test reminds me a lot of my colleagues Ron Allen and Mike Pardo’s “storytelling” account of legal proof. (See Ronald J. Allen and Michael S. Pardo (2008), Juridical proof and the best explanation, Law and Philosophy, Vol. 27, pp. 223–268, which is available here. See also Allen & Pardo (2019), Relative plausibility and its critics, International Journal of Evidence & Proof, Vol. 23, pp. 5–59, available here.)

Among other things, Allen and Pardo’s account of legal proof not only makes intuitive sense; their storytelling approach also parallels Hume’s probabilistic two-part test for miracles:

“The proof process involves two stages: (1) the generation of potential explanations of the evidence and events, and (2) a comparisons of the [plaintiff’s and defendant’s competing] explanations in light of the applicable standard of proof. In general, the process depends on the parties to obtain evidence and to offer what they consider to be the best explanation (or explanations) that support their respective cases.” (Allen & Pardo 2019)

In other words, when jurors are deciding cases, what they are really doing is comparing the competing stories of the plaintiff and the defendant, and it is the party who offers the best explanation of the evidence — i.e. the party who tells the best story — who wins. Sound familiar?

This storytelling account of legal proof — like Hume’s probabilistic test for miracles — also has two major virtues. One is that it makes the most intuitive and descriptive sense: jurors do tend to weigh the evidence presented at a trial holistically, and they tend to favor the side whose story is more persuasive, or to borrow Allen and Pardo’s terminology, juries usually prefer the side whose story offers the best explanation of the evidence. The other virtue of the storytelling approach is an aesthetic one: its elegant simplicity. Why is simplicity a virtue? Because more simple or parsimonious explanations are almost always better than complex or convoluted ones, a fundamental principle called “Occam’s Razor”.

Nevertheless, Hume’s probabilistic test, along with Allen and Pardo’s storytelling approach to legal proof, has two big blind spots: (1) the reference class problem, and (2) the unknown probability problem. Stay tuned: I will explain both of these fatal flaws and present an alternative approach to the problem of miracles in my next few posts.

Last week, we surveyed David Hume’s influential argument against the possibility of miracles. (See here, here, and here.) This week, I want to offer my own take on Hume’s argument — the good, the bad, and the ugly — but before I do, I want to begin by making a full disclosure: I will for the most part avoid consulting or mentioning (except in today’s introductory post) any of secondary and even tertiary literature that already exists on this topic.

For starters, the scholarly literature is massive. A quick search for “Hume on miracles” in Google Scholar, for example, generates over 67,000 discrete entries! (Here, for example, is an early response to Hume by one William Adams.) So, what can I contribute to the problem of miracles? In a word, I will use my background knowledge and experience as a law professor — one with a deep interest in economics, history, and probability theory — to offer my own “legal proof” perspective on Hume’s argument.

In the meantime, below are some fairly recent scholarly works that students of Hume and the problem of miracles may wish to consult for future reference:

• ANTI-HUME: John Earman, Hume’s Abject Failure: The Argument Against Miracles, Oxford University Press (2000).
• PRO-HUME: Robert J. Fogelin, A Defense of Hume on Miracles, Princeton University Press (2003).
• FURTHER READING (HONORABLE MENTIONS): William L. Vanderburgh, David Hume on Miracles, Evidence, and Probability, Lexington Books (2019) and Peter Millican, Twenty Questions about Hume’s “Of Miracles”, Royal Institute of Philosophy Supplements, Vol. 68 (2011), pp. 151-192.

The artist Roberta Flack, whose music made an indelible mark on my childhood, died last month at the age of 88; here is her New York Times obituary.

Having defined what a miracle is (see here) and having established the relationship between evidence and probability (here), Hume is now ready to finally unveil his novel argument against miracles. To the point, for Hume “no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavours to establish …” (Hume, Of Miracles, Para. 13). In plain English, what Hume is saying here is that even when we have direct evidence of a miracle, such as eyewitness testimony, our inquiry is not over. We still have to weigh the evidence. Specifically, we must consider not only the probability that the evidence is reliable or true but also the probability that it is defective or false, or in the immortal words of David Hume himself:

“When any one tells me, that he saw a dead man restored to life, I immediately consider with myself, whether it be more probable, that this person should either deceive or be deceived, or that the fact, which he relates, should really have happened. I weigh the one miracle against the other; and according to the superiority, which I discover, I pronounce my decision, and always reject the greater miracle.” (Hume, Of Miracles, Para. 13, emphasis added)

Hume thus proposes a simple two-part probabilistic test for evaluating reports of miracles. The first part works as follows: if someone, for example, tells you X — that they saw a UFO or were abducted by aliens — you need to consider two separate probabilities: A and B, where A is the probability that X, the remarkable or unusual event in question, took place — i.e., how likely is it, given your own experience and common sense, that the UFO sighting or alien abduction really occurred? — and where B is the probability that the report is either mistaken or fraudulent or otherwise defective — or in Hume’s words, the probability “that its falsehood would be more miraculous, than the fact, which it endeavours to establish” (ibid.).

Next, after assigning probability values to both of these logical possibilities (A and B), the second and last part of Hume’s test is to compare both probabilities. According to Hume’s probabilistic logic, you should prefer the possibility whose probability value is greater. (Or, put another way, only if the probability of B is somehow smaller than that of A should you believe in X.)

This post concludes my review of Part 1 of Hume’s essay “Of Miracles” (paragraphs 1 to 13). Starting next week, I will present a “Bayesian” critique of Hume’s solution to the problem of miracles and then turn my attention to Part 2 of his essay (paragraphs 14 to 41). In the meantime, I would like you to think about the following question: is there any way a report of a miracle or other remarkable event could ever pass Hume’s probabilistic test?