Int J Med Inform. 2023 Aug 11;178:105177. doi: 10.1016/j.ijmedinf.2023.105177. Online ahead of print.
ABSTRACT
OBJECTIVE: To develop a machine-learning (ML) model using administrative data to estimate risk of adverse outcomes within 30-days of a benzodiazepine (BZRA) dispensation in older adults for use by health departments/regulators.
DESIGN, SETTING AND PARTICIPANTS: This study was conducted in Alberta, Canada during 2018-2019 in Albertans 65 years of age and older. Those with any history of malignancy or palliative care were excluded.
EXPOSURE: Each BZRA dispensation from a community pharmacy served as the unit of analysis.
MAIN OUTCOMES AND MEASURES: ML algorithms were developed on 2018 administrative data to predict risk of any-cause hospitalization, emergency department visit or death within 30-days of a BZRA dispensation. Validation on 2019 administrative data was done using XGBoost to evaluate discrimination, calibration and other relevant metrics on ranked predictions. Daily and quarterly predictions were simulated on 2019 data.
RESULTS: 65,063 study participants were included which represented 633,333 BZRA dispensation during 2018-2019. The validation set had 314,615 dispensations linked to 55,928 all-cause outcomes representing a pre-test probability of 17.8%. C-statistic for the XGBoost model was 0.75. Measuring risk at the end of 2019, the top 0.1 percentile of predicted risk had a LR + of 40.31 translating to a post-test probability of 90%. Daily and quarterly classification simulations resulted in uninformative predictions with positive likelihood ratios less than 10 in all risk prediction categories. Previous history of admissions was ranked highest in variable importance.
CONCLUSION: Developing ML models using only administrative health data may not provide health regulators with sufficient informative predictions to use as decision aids for potential interventions, especially if considering daily or quarterly classifications of BZRA risks in older adults. ML models may be informative for this context if yearly classifications are preferred. Health regulators should have access to other types of data to improve ML prediction.
PMID:37591010 | DOI:10.1016/j.ijmedinf.2023.105177
