My previous post highlighted this 2023 paper by my colleague and new friend Ian D. Gow, who conjectures that much of the published research in his field (academic accounting) is the product of p-hacking or data dredging. Professor Gow’s recent paper caught my attention because I suspect that these data manipulation practices are not only common in many other fields beyond academic accounting; they may also be symptoms of deeper problems in scientific research more generally. So, what is to be done? Here, as a public service, I will survey three possible solutions and explain why #3 is my preferred option for now:

1. Prohibition, i.e. banish p-values altogether. Perhaps the time has come for accounting journals to follow the lead of the editors of Basic and Applied Social Psychology, which took this drastic step in 2015 in response to the replication crisis in their field. After all, if the practice of p-hacking is really as rampant as Professor Gow says it is, then his proposed solutions, e.g. paltry reforms like “more descriptive research” and improved research training (see Gow 2023, pp. 8-12), are mere band-aids that are unlikely to address the perverse “publish or perish” incentive structure among academic accountants.
2. Pre-registration, i.e. reduce researcher degrees of freedom. If prohibition of null hypothesis significance testing is too radical a solution, perhaps accounting journals should require all author teams to submit detailed research plans (e.g. hypotheses, design, and analysis) before they begin collecting any data. On page 3 of his paper, for example, Professor Gow compares and contrasts “traditional editorial processes” or TEP, where researchers have so many degrees of freedom that research misconduct is all but inevitable, with “registration-based editorial processes” or REP, which has been tested by some accounting journals, but for some reason Gow himself does not pursue this solution to its logical conclusion. (PS: To learn more about the costs and benefits of TEP versus REP, check out this 2018 paper by Bloomfeld et al.)
3. The invisible hand, i.e. journals should do nothing. What if Professor Gow is wrong? What if accounting research isn’t so bad, after all? For my part, given my Burkean priors, this is my preferred solution, at least until Professor Gow’s claims themselves have been replicated. I would therefore recommend “do nothing” for now, but in the meantime I would like to see Gow’s claims tested by other accounting researchers.