prior probability

As a follow-up to one of my previous posts (see here), below is my contribution to the most recent episode of the ten-part “Books that Shaped America” series that aired on C-SPAN earlier this week. In brief, I explain why Oliver Wendell Holmes’s 1881 lectures on the common law are still worth reading. (This week’s episode, which is also available in full here, was devoted to Holmes’s classic work The Common Law.)

My previous post highlighted this 2023 paper by my colleague and new friend Ian D. Gow, who conjectures that much of the published research in his field (academic accounting) is the product of p-hacking or data dredging. Professor Gow’s recent paper caught my attention because I suspect that these data manipulation practices are not only common in many other fields beyond academic accounting; they may also be symptoms of deeper problems in scientific research more generally. So, what is to be done? Here, as a public service, I will survey three possible solutions and explain why #3 is my preferred option for now:

1. Prohibition, i.e. banish p-values altogether. Perhaps the time has come for accounting journals to follow the lead of the editors of Basic and Applied Social Psychology, which took this drastic step in 2015 in response to the replication crisis in their field. After all, if the practice of p-hacking is really as rampant as Professor Gow says it is, then his proposed solutions, e.g. paltry reforms like “more descriptive research” and improved research training (see Gow 2023, pp. 8-12), are mere band-aids that are unlikely to address the perverse “publish or perish” incentive structure among academic accountants.
2. Pre-registration, i.e. reduce researcher degrees of freedom. If prohibition of null hypothesis significance testing is too radical a solution, perhaps accounting journals should require all author teams to submit detailed research plans (e.g. hypotheses, design, and analysis) before they begin collecting any data. On page 3 of his paper, for example, Professor Gow compares and contrasts “traditional editorial processes” or TEP, where researchers have so many degrees of freedom that research misconduct is all but inevitable, with “registration-based editorial processes” or REP, which has been tested by some accounting journals, but for some reason Gow himself does not pursue this solution to its logical conclusion. (PS: To learn more about the costs and benefits of TEP versus REP, check out this 2018 paper by Bloomfeld et al.)
3. The invisible hand, i.e. journals should do nothing. What if Professor Gow is wrong? What if accounting research isn’t so bad, after all? For my part, given my Burkean priors, this is my preferred solution, at least until Professor Gow’s claims themselves have been replicated. I would therefore recommend “do nothing” for now, but in the meantime I would like to see Gow’s claims tested by other accounting researchers.

That is the title of this new paper by Ian D. Gow, a professor of accounting at the University of Melbourne, who concludes that most of the findings published in the top journals in his field are of dubious value, if not worthless. Why? Because many researchers in academic accounting engage in rampant data manipulation or p-hacking, if Gow is to be believed! As it happens, I have blogged about p-hacking, “data dredging”, and other forms of research misconduct before (see here and here, for example), but I have always assumed that no self-respecting social science researcher would allow himself to fall into those traps (see, for example, this 2022 paper by Saltiero et al.), but if Gow’s conjecture is correct — if these shoddy methods are the “dominant mode of research in academic accounting” — then it would mean that most research in academic accounting is, to quote Gow, “a largely pointless exercise” or of “limited value” at best.

By way of comparison, I recently mentioned (see here) this scathing critique by Adam Mastroianni of the “cognitive bias craze” in experimental psychology. Among other things, Mastroianni’s critique not only highlights the now-infamous 2015 study in which 60% of psychology studies failed to replicate; he further notes how resistant the field’s dominant paradigms has been to this replication failure. Now, let’s return to academic accounting. According to my reading of Professor Gow’s damning paper, accounting is in even worse scholarly shape than experimental psychology is! At least psych researchers have attempted to replicate their studies. So, what is to be done? I will consider some possible methods for taming this massive elephant in a future post.

That is the title of this new TV series on C-SPAN, my favorite channel on cable television. In summary, this new series features ten great North American books that have provoked thought among leading public intellectuals, that have led to significant public policy changes in the real world, and that are still talked about today. As it happens, I was invited to contribute to the episode on Oliver Wendell Holmes’s classic tome “The Common Law“. (This episode is scheduled to air for the first time on C-SPAN on Monday night, 16 October, at 9 PM EDT; in the meantime, below is a recording of my contribution.)

Taylor Swift is a national treasure! Not only does she merit her own subreddit (r/TaylorSwift); her concert-movie is the best thing I have seen in theatres in a long time!

Dr Luis Martínez-Fernández, author, historian, and Pegasus Professor at the University of Central Florida (UCF’s highest honor), will present an illustrated history of our beloved lost Caribbean isle. The intriguing title of his talk is “Cuba’s History in 25 Artifacts“, and it is scheduled to take place at the main branch of the Orlando Public Library in Downtown Orlando on Saturday, 14 October, at four o’clock in the afternoon. (Details here.) In the meantime, I don’t know whether my colleague and friend will include a vintage Cuban tourism poster in his collection of 25 artifacts, so I am posting a collage of such classic affiches below (photo credit: Al Hurley), just in case!

What is the most beautiful, haunting, and literary law review article of all time? In preparation for my upcoming talk on the life and legacy of Oliver Wendell Holmes (see my previous post), I want to say a few words about Holmes’s classic essay “The Path of the Law”, the first page of which is pictured below. To the point, my thesis is that Holmes’s work should be read as a short story or novella, for it is nothing less than a timeless masterpiece, along with the works of other great writers like Jorge Luis Borges and Ernest Hemingway.

As it happens, I explored Holmes’s seminal essay from a literary perspective in my article “Coase’s Parable“, which was published in a special symposium issue of the Mercer Law Review in May of 2023. (“The Problem of Social Cost” by Ronald H. Coase and Holmes’s “Path of the Law” are two of the most controversial, most cited, and most influential law review articles of all time.) Below the fold is an extended excerpt from my 2023 paper:

Two weeks from today (Friday, 27 October), three of my colleagues — Seema Mohapatra, Michael Morley, and Dan O’Gorman — and I will get together to speak about the life and legacy of the great North American jurist Oliver Wendell Holmes. The student chapter of the American Constitution Society (ACS) will be hosting us at the Dwayne O. Andreas Law School in Orlando, Florida. Details below:

That is the title of this new book about the assassination of President John F. Kennedy. As it happens, I already blogged about this new work, which finally hit bookstores earlier this week (see here or below), so I just want to pose two pithy but poignant questions for now:

1. Why did the author, retired Secret Service agent Paul Landis, wait 60 years to tell his story, and 2. Is his story a credible one? (Alas, I hate to be that guy, but Mr Landis was one of several Secret Service agents who stayed up late, partying and drinking in Fort Worth until the wee hours of the morning, just a few hours before the Kennedy assassination. See here and here, for example.)