(Predicting) replications

Résumé

Which results can we “trust”? What share of results replicate in different literatures in the experimental social sciences? I will discuss several recent large replication projects in mainly psychology and economics, where my coauthors and I have redone experiments published in high impact journals with new and larger samples to see whether the main result replicates. I will also discuss our studies on “wisdom-of-crowds” mechanisms like prediction markets and forecasting surveys where researchers attempt to predict these replication outcomes as well as new outcomes. While the replications are mainly on experiments, there are reasons to believe that the problems are worse in non-experimental work. I will also discuss some proposed solutions.