Contemp Clin Trials. 2022 May 4:106782. doi: 10.1016/j.cct.2022.106782. Online ahead of print.
ABSTRACT
It is not uncommon that clinical studies of the same intervention contradicted with each other, e.g., one study produced positive results, while the other produced negative results. Ioanndis (2005a) found that among 49 highly-cited original clinical research studies, published in New England Journal of Medicine, Journal of the American Medical Association, Lancet or in a high-impact medical specialty journal, 32% of them were contradicted in subsequent large-scale studies, or were shown to have potentially overestimated the efficacy of the experimental intervention. This finding is disturbing and of serious concern given the widespread impact of these highly-cited studies and the rigorous standards used to design and conduct the studies. We perform Bayesian analysis of these highly-cited clinical studies based on Bayesian factor. We identified one cause of the issue: p values strongly overstated the experimental evidence. For the highly-cited studies, when the p value was 0.05, there was a 74.4% percentage chance that the null hypothesis was true. The use of a p value of 0.05 as the criterion for significance caused many researchers to mistakenly draw conclusions of positive findings, which were then contradicted by subsequent large-scale studies.
PMID:35525531 | DOI:10.1016/j.cct.2022.106782
