Cureus. 2024 Dec 30;16(12):e76607. doi: 10.7759/cureus.76607. eCollection 2024 Dec.
ABSTRACT
AIM: The aim of the study is to test the null hypothesis that the specificities and sensitivities of the p-value-based significance test for differences between baseline variables and the I2 test for single trials do not significantly differ in detecting selection bias in randomised controlled trials (RCTs).
METHODS: In MS Excel (Microsoft Corp., Redmond, WA, US), 100 trials were simulated, each consisting of two treatment groups (A and B), with 100 subjects in each group. Fifty trials were biased, while 50 remained non-biased. Both tests were applied to all trials, yielding true positive, false positive, false negative, and true negative per test. Subsequently, sensitivities and specificities with a 95% confidence interval (CI) were calculated and statistically compared using the z-test.
RESULTS: No false positive results were observed, and subsequently, the specificities of both tests were identical (100.00%; 95% CI: 92.89%-100.00%). The sensitivity for the significance test and I2 test was 24.00% (95% CI: 13.06%-38.17%) and 76.00% (95% CI: 61.83%-86.94%), respectively. A statistical comparison of the test sensitivities yielded a significant result in favour of the I2 test (z = 5.2; p < 0.0001). Consequently, the null hypothesis for the tests’ sensitivities was rejected.
CONCLUSION: The I2 test appears to be a more effective method than the p-value-based significance test for detecting selection bias in RCTs.
PMID:39886704 | PMC:PMC11779566 | DOI:10.7759/cureus.76607
