Researchers overturn landmark study on the replicability of psychological science

http://www.eurekalert.org/pub_releases/2016-03/hu-rol022916.php

Public Release: 3-Mar-2016
Researchers overturn landmark study on the replicability of psychological science
Harvard University

According to two Harvard professors and their collaborators, a 2015 landmark study showing that more than half of all psychology studies cannot be replicated is actually wrong.

•••••

But an in-depth examination of the data by Daniel Gilbert (Edgar Pierce Professor of Psychology at Harvard University), Gary King (Albert J. Weatherhead III University Professor at Harvard University), Stephen Pettigrew (doctoral student in the Department of Government at Harvard University), and Timothy Wilson (Sherrell J. Aston Professor of Psychology at the University of Virginia) has revealed that the OSC made some serious mistakes that make this pessimistic conclusion completely unwarranted:

The methods of many of the replication studies turn out to be remarkably different from the originals and, according to Gilbert, King, Pettigrew, and Wilson, these "infidelities" had two important consequences.

First, they introduced statistical error into the data which led the OSC to significantly underestimate how many of their replications should have failed by chance alone. When this error is taken into account, the number of failures in their data is no greater than one would expect if all 100 of the original findings had been true.

Second, Gilbert, King, Pettigrew, and Wilson discovered that the low-fidelity studies were four times more likely to fail than were the high-fidelity studies, suggesting that when replicators strayed from the original methods, they caused their own studies to fail.

Finally, the OSC used a "low powered" design. When Gilbert, King, Pettigrew, and Wilson applied this design to a published data set that was known to have a high replication rate, it too showed a low replication rate, suggesting that the OSC's design was destined from the start to underestimate the replicability of psychological science.

•••••