Jan 2, 2021The value of pWhat we lose if we abandon p values — I spent a fair amount of time thinking and writing on both sides of the frequentist/Bayesian divide. In recent years, thanks to reading (and, importantly, re-reading) the persectives of Mayo, Cox, Spanos, Wasserman, I’ve become convinced of the importance of a frequentist perspective on statistical evidence, but I’ve not articulated…Statistics7 min read
Jun 12, 2020Power and precisionWhy the push for replacing “power” with “precision” is misguided — [Code for all the plots in this post is available in this gist.] One of the common claims of anti-significance-testing reformers is that power analysis is flawed, and that we should be planning for study “precision” instead. I think this is wrong for several reasons that I will outline here…Statistics19 min read
Apr 4, 2020To you, on the occasion of your replication crisis“I am very sorry, Pyrophilus…” — The seventeenth century was an exciting time in Europe. A new empirical philosophy — what we call science — was emerging. Oldenburg founded the Philosophical Transactions of the Royal Society, now the oldest scientific journal still being published, to communicate peoples’ observations and ideas. …4 min read
Aug 3, 2019New pre-print: Use of significance test logic by scientists in a novel reasoning taskHow scientists successfully reason about sampling distributions — Rink Hoekstra (@rinkhoekstra) and I are pleased to release a new pre-print, “Use of significance test logic by scientists in a novel reasoning task”. We test the ability of a large, diverse group of researchers to reason from sampling distributions, a core aspect of significance logic. Although it is common…Data Science5 min read
Published in Towards Data Science·Jan 28, 2019Why you shouldn’t say “this study is underpowered”Ensuring our critiques of power are relevant, clear, and based on good statistics — On Twitter recently, I criticised the common practice (some might say “shorthand”) of claiming that an experiment or study is “underpowered”. …Data Science8 min read
Dec 16, 2018Use your statistical intuition to save Christmas!A Christmas-themed statistical game and experiment — (The experiment is over, but you can see a version of it on github) It’s nearly Christmas, but Santa has a problem: his elves have stopped working! The elves have have divided into two gangs, the Jinglies and the Sparklies, each believing that their gang can make toys faster. Santa…Christmas2 min read
Oct 21, 2018Severity and statistical evidenceNotes on Tour 1 of “Statistical Inference as Severe Testing” — Anyone who’s had any contact with statistical methods recently knows that there’s a battle being fought over the future of statistical methods. Actually, more than one; the big ones are significance testing vs confidence intervals and Bayes vs frequentism. The so-called “replication crisis” in the various sciences has provided an…Statistics4 min read
Jul 25, 2018An Open Letter in Support of Markey and ElsonScientists defend two researchers trying to clean up the scientific literature — [Update: the paper in question was officially retracted while this letter was being circulated; but OSU’s complaint is still outstanding. The good news is that both Markey and Elson’s Universities have declined to act on OSU’s complaint. To date, however, they have not made any public statement in support of…Science8 min read
Jan 1, 2018Redefining statistical significance: the statistical argumentsPart two of a three part series — [Part one of this series is here] At the heart of the RSS paper are a number of statistical arguments. There are three, and I will address each of them in this (rather long) post. They are 1) two-sided p values around .05 are evidentially weak, 2) using a lower…Data Science22 min read
Nov 26, 2017When the statistical tail wags the scientific dogShould we “redefine” statistical significance? — [Part two of this series is here] Recently, a zillion-author manuscript came down the pipe suggesting that a change to the common practice of using p<.05 as a criterion for statistical significance. …Science4 min read
