Note: This post is part 7 of our extended review of Allen & Pardo’s new paper on juridical proof “Relative plausibility and its critics.”
We replied to the pesky conjunction problem in our previous post. Today, we will discuss the irony of Allen and Pardo’s two most serious objections to the Bayesian or subjective view of juridical proof. In short, Allen and Pardo assert that subjective probability is subjective, and that being purely subjective, this account bears “no relationship to advancing accurate outcomes” in legal trials (p. 10). Say what? By definition, it’s true that subjective probability is subjective. In fact, all methods of decision-making under uncertainty are subjective! As such, the irony of Allen and Pardo’s bald assertions does not escape us, since the same could be said of their pet theory of relative plausibility! Think about it: just because explanation A is more plausible or persuasive than a competing explanation (explanation B) does not, by itself, make explanation A true, especially if the individual probabilities of A and B are each below 0.5 or don’t add up to 1. The question, then, is not whether judgments about evidence are subjective or not–all are–; the question is which view of proof generates less inaccuracy.
On this score, Allen and Pardo claim about accuracy is doubly ironic. Why? Because the Bayesian or subjective view of proof actually provides an excellent way of aggregating individual judgments and of producing accurate outcomes overall–in particular, outcomes that are less vulnerable to strategic voting by jurors (such as holdouts) or to juror ignorance (jurors with ill-informed or misconceived priors about the world) than outcomes generated by the relative plausibility theory. (We explain why in more detail in our paper “Why don’t juries try range voting?,” published in The Criminal Law Bulletin, Vol. 51 (2015), pp. pp. 682-692.) Here, we wish to focus on the accuracy question by explaining why a subjective view of proof is more likely to produce accurate outcomes than a relative plausibility view of proof.
The main reason is this: under the relative plausibility view of proof, the fact-finder must choose in all-or-nothing fashion or either-or terms the most plausible or most persuasive explanation of the evidence presented at a trial. Under the subjective probability view, by contrast, the fact-finder assigns a degree of belief to each one of these competing explanations before deciding which explanation is the most compelling or persuasive one. [Note: contra Allen and Pardo, it is not necessary to express one’s subjective degree of belief using numerical values. (See Keynes, Treatise of Probability (1921), pp. 20-22.) It is sufficient to use such formulations as “highly likely, almost certain, or virtually impossible” to describe the likelihood of some event or proposition, given a body of evidence. (See James Franklin, 2011, p. 548, in “The objective Bayesian conceptualisation of proof and reference class problems,” Sydney Law Review, Vol. 33, pp. 545-561.)] But either way, the key point that Allen and Pardo neglect is that fact-finders in legal trials often consist of collective bodies, i.e. juries.
Moreover, it is because of the collective nature of most legal fact-finding that degrees of belief (whether qualitative or quantitative) generate more accuracy than mere relative plausibility. To see why, consider the recent decision by Netflix to replace its five-star cardinal voting system with a binary thumbs-up/thumbs-down system of voting. Assume we want to measure the quality of movies. Of course, movie quality (like juridical proof!) is highly subjective. Nevertheless, which of these two voting procedures–i.e. the five-star system or the binary system–will generate more accurate measures of film quality overall? The star system permits subjective assessments of a movie’s quality–with each star reflecting the viewer’s subjective degree of belief in a movie’s quality. The binary system, by contrast, is akin to Allen and Pardo’s relative plausibility approach in which the viewer must decide in all-or-nothing fashion whether to like or dislike the movie.
Now, consider what happens when two people watch the same movie, but one person likes the movie, while the other does not. Under the five-star system, the viewers could compromise by assigning an average score–three stars out of five, say. Under Netflix’s binary system, by contrast, there is no fair way of aggregating the preferences of two people who disagree about the quality of a movie. To the extent most people don’t watch movies alone, our objection should worry Netflix. Furthermore, even when one is watching a movie on Netflix alone, the binary or all-or-nothing system results in a loss of information, since it does not allow one to express the intensity of one’s like or dislike of a particular movie. [Note: Although YouTube and Rotten Tomatoes also employ simple binary votings systems, those websites at least disclose the ratio of likes to dislikes as well as (in the case of YouTube) the total number of views, which allows us to deduce the number of abstentions. In this respect, Facebook’s system of unitary voting (you can’t vote to dislike something) provides even less useful information than Netflix’s binary voting system.]
To sum up, you can file this post under the “annals of irony.” Although Allen and Pardo assert that subjective probability “has no relationship to advancing accurate outcomes,” neither does their theory of relative plausibility!!! Accordingly, the real questions are these: which of the two approaches to juridical proof–i.e. subjective probability or relative plausibility–provides a better explanation of the standards of proof, a better description of how juries actually decide cases, and is more likely to produce accurate outcomes? It turns out that the Bayesian view of proof outperforms Allen and Pardo’s relative plausibility theory on every one of these dimensions. Why? Because the Bayesian approach allows us to aggregate or combine competing views of the evidence when jurors disagree about the relative plausibility of various explanations of the evidence presented at trial. In addition, compared to the relative plausibility account, the subjective view also provides a more accurate way (although not a perfect one) of aggregating collective decision-making under conditions of uncertainty. Stay tuned, we will defend this normative thesis in greater detail in our next blog post …