Grad Student Talk | Chris Martin | No Support for Declining Effect Sizes Over Time: Evidence from Three Meta-Meta-Analyses.

In psychology (e.g., Schooler, 2011) and other fields (e.g., Jennions & Møller, 2001), there are reported cases of effect sizes declining over time. Later studies of a given phenomenon report smaller effect sizes than earlier studies. This decline suggests a publication bias toward large effects and regression to the mean. In the current study, we examine whether evidence exists for such a decline effect. In Study 1, we analyzed 3,488 effect sizes across 70 meta-analytic tables, which were drawn from 33 Psychological Bulletin articles (1980–2010). A multilevel analysis revealed no evidence of a linear or quadratic decline effect over time (indexed by publication year). In Studies 2 and 3, we examined 50 meta-analyses each from social psychology and clinical psychology. In both studies, the modal meta-analysis showed no correlation between effect size and publication year. The decline effect in psychology appears to be less prevalent than earlier anecdotal reports suggest. For replications, this finding suggests that expectations that replications will have lower effect sizes than the original may be inaccurate and unfounded. (September 8, 2015)