Deborah Mayo recently reblogged and commented on Nathan Schachtman’s blog post titled “Courts Can and Must Acknowledge Multiple Comparisons in Statistical Analyses.” (Mr Schachtman is not only a lawyer; he is also a lecturer at Columbia Law School and an expert on scientific evidence.) Moreover, because Mr Schachtman’s blog post and Dr Mayo’s comments on Schachtman’s post touch on an area we care about — the role of probability theory in law — we are re-reblogging both items below and shall comment on them in a future post.
Update (22 October 2014): We read Schachtman’s post on the multiple testing problem in law (see below), and we also read Dr Mayo’s comments to his post (ditto), and we were left scratching our heads. We’re not sure where they disagree. After all, isn’t it true that “data trolling” is bad science? And, if so, shouldn’t trial judges retain the discretion to exclude expert testimony based on multiple comparisons?
.
“Courts Can and Must Acknowledge Multiple Comparisons in Statistical Analyses”
Nathan Schachtman, Esq., PC * October 14th, 2014
In excluding the proffered testimony of Dr. Anick Bérard, a Canadian perinatal epidemiologist in the Université de Montréal, the Zoloft MDL trial court discussed several methodological shortcomings and failures, including Bérard’s reliance upon claims of statistical significance from studies that conducted dozens and hundreds of multiple comparisons.[i] The Zoloft MDL court was not the first court to recognize the problem of over-interpreting the putative statistical significance of results that were one among many statistical tests in a single study. The court was, however, among a fairly small group of judges who have shown the needed statistical acumen in looking beyond the reported p-value or…
View original post 2,543 more words
prior probability 