Health Psychol. 2025 Jul 17. doi: 10.1037/hea0001534. Online ahead of print.
ABSTRACT
OBJECTIVE: To use a recently developed methodology, z-curve analysis, to estimate the likelihood of replication success for recently published studies in three leading health psychology journals with high impact factors that involved some form of a moderation analysis.
METHOD: Utilizing a z-curve analysis, we estimated the replicability, false positives, publication bias, and “file drawer ratio” of 124 independent tests of moderation with significant results published in recent issues of three leading journals in the field of health psychology. z-curve analyses were conducted for all the journals combined and each journal separately.
RESULTS: The distribution of z scores derived from all 124 studies indicated that the estimation of the expected replication rate and false positive ratio were 46.0% and 8.3%, respectively. The estimated file drawer ratio was 1.6, indicating that for every statistically significant interaction reported, nearly two nonsignificant interactions go unreported. In comparing the three journals, Health Psychology had the best overall results (expected replication rate = 52.3%, Soric false discovery rate = 4.8%, file drawer ration = 0.9). Of the 124 studies examined, 23 conducted power analyses to determine sample size, seven preregistered hypotheses, and three conducted a replication analysis.
CONCLUSION: Results suggest a need for change regarding both the methodological practices used and the publication processes in place to improve the validity and efficacy of research regarding moderation effects in behavioral medicine-this includes preregistering hypotheses, using formalized methods to determine sample size, and utilizing attention checks. Journals can encourage or require these practices and foster acceptance of nonsignificant results to limit publication biases. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
PMID:40673996 | DOI:10.1037/hea0001534
