Category: Statistics

JMIR Form Res. 2024 Nov 19;8:e64844. doi: 10.2196/64844.

ABSTRACT

BACKGROUND: Generative artificial intelligence (AI), particularly in the form of large language models, has rapidly developed. The LLaMA series are popular and recently updated from LLaMA2 to LLaMA3. However, the impacts of the update on diagnostic performance have not been well documented.

OBJECTIVE: We conducted a comparative evaluation of the diagnostic performance in differential diagnosis lists generated by LLaMA3 and LLaMA2 for case reports.

METHODS: We analyzed case reports published in the American Journal of Case Reports from 2022 to 2023. After excluding nondiagnostic and pediatric cases, we input the remaining cases into LLaMA3 and LLaMA2 using the same prompt and the same adjustable parameters. Diagnostic performance was defined by whether the differential diagnosis lists included the final diagnosis. Multiple physicians independently evaluated whether the final diagnosis was included in the top 10 differentials generated by LLaMA3 and LLaMA2.

RESULTS: In our comparative evaluation of the diagnostic performance between LLaMA3 and LLaMA2, we analyzed differential diagnosis lists for 392 case reports. The final diagnosis was included in the top 10 differentials generated by LLaMA3 in 79.6% (312/392) of the cases, compared to 49.7% (195/392) for LLaMA2, indicating a statistically significant improvement (P<.001). Additionally, LLaMA3 showed higher performance in including the final diagnosis in the top 5 differentials, observed in 63% (247/392) of cases, compared to LLaMA2’s 38% (149/392, P<.001). Furthermore, the top diagnosis was accurately identified by LLaMA3 in 33.9% (133/392) of cases, significantly higher than the 22.7% (89/392) achieved by LLaMA2 (P<.001). The analysis across various medical specialties revealed variations in diagnostic performance with LLaMA3 consistently outperforming LLaMA2.

CONCLUSIONS: The results reveal that the LLaMA3 model significantly outperforms LLaMA2 per diagnostic performance, with a higher percentage of case reports having the final diagnosis listed within the top 10, top 5, and as the top diagnosis. Overall diagnostic performance improved almost 1.5 times from LLaMA2 to LLaMA3. These findings support the rapid development and continuous refinement of generative AI systems to enhance diagnostic processes in medicine. However, these findings should be carefully interpreted for clinical application, as generative AI, including the LLaMA series, has not been approved for medical applications such as AI-enhanced diagnostics.

PMID:39561356 | DOI:10.2196/64844

J Med Internet Res. 2024 Nov 19;26:e56166. doi: 10.2196/56166.

ABSTRACT

BACKGROUND: Lack of adherence to prescribed medication is common among patients with depression in China, posing serious challenges to the health care system. Online health communities have been found to be effective in enhancing patient compliance. However, empirical evidence supporting this effect in the context of depression treatment is absent, and the influence of online health community content on patients’ attitudes toward medication adherence is also underexplored.

OBJECTIVE: This study aims to explore whether online depression communities (ODCs) can help ameliorate the problem of poor medication taking among patients with depression. Drawing on the stimulus-organism-response and feelings-as-information theories, we established a research model to examine the influence of useful institution-generated content (IGC) and positive user-generated content (UGC) on attitudes toward medication adherence when combined with the mediating role of perceived social support, perceived value of antidepressants, and the moderating role of hopelessness.

METHODS: A cross-sectional questionnaire survey method was used in this research. Participants were recruited from various Chinese ODCs, generating data for a main study and 2 robustness checks. Hierarchical multiple regression analyses and bootstrapping analyses were adopted as the primary methods to test the hypotheses.

RESULTS: We received 1515 valid responses in total, contributing to 5 different datasets: model IGC (n=353, 23.3%), model UGC (n=358, 23.63%), model IGC+UGC (n=270, 17.82%), model IGC-B (n=266, 17.56%), and model UGC-B (n=268, 17.69%). Models IGC and UGC were used for the main study. Model IGC+UGC was used for robustness check A. Models IGC-B and UGC-B were used for robustness check B. Useful IGC and positive UGC were proven to have positive impact on the attitudes of patients with depression toward medication adherence through the mediations of perceived social support and perceived value of antidepressants. The findings corroborated the role of hopelessness in weakening or even negating the positive effects of ODC content on the attitudes of patients with depression toward medication adherence.

CONCLUSIONS: This study provides the first empirical evidence demonstrating the relationship between ODC content and attitudes toward medication adherence, through which we offer a novel solution to the problem of poor medication adherence among patients with depression in China. Our findings also provide suggestions about how to optimize this new approach-health care practitioners should generate online content that precisely matches the informational needs of patients with depression, and ODC service providers should endeavor to regulate the community atmosphere. Nonetheless, we warn that ODC interventions cannot be used as the only approach to addressing the problem of poor medication taking among patients with severe depressive symptoms.

PMID:39561355 | DOI:10.2196/56166

J Clin Oncol. 2024 Nov 19:JCO2400304. doi: 10.1200/JCO.24.00304. Online ahead of print.

ABSTRACT

PURPOSE: Persistent smoking after cancer diagnosis causes adverse outcomes while smoking cessation can improve survival. Thus, integration of smoking assessment and cessation assistance into routine cancer care is critical. Aiming for incremental practice change that could be sustained and built upon through future quality improvement (QI) projects, the American College of Surgeons initiated Just ASK in 2022 to increase implementation of smoking assessment among its accredited Cancer Programs. This manuscript describes outcomes from Just ASK.

METHODS: Seven hundred sixty-two programs enrolled in this cohort study, followed Plan Do Study Act methodology, and used local QI teams to facilitate practice change. The primary outcome was the ask rate (ie, patients asked/patients seen). Programs completed three surveys across the 1-year study (89.8% retention), answering questions about their program plus organizational readiness, implementation barriers, implementation strategies, and clinical practices related to assessing smoking among patients newly diagnosed with cancer. Data analysis involved descriptive statistics and analysis of change over time (eg, McNemar chi-squares).

RESULTS: Programs (53.1% community-based) tended to report moderate organizational readiness, multiple implementation barriers, and adoption of 4.63 ± 1.49 of eight possible implementation strategies (eg, training staff/providers). Programs reported frequency of assessing smoking status, documenting it in the electronic health record, advising patients who smoke to quit, and documenting advice and treatment increased over time (all P < .001). The ask rate increased from baseline to mid to final survey (P < .01; 87.79% v 88.65% v 91.92%, respectively).

CONCLUSION: Just ASK is the latest, and by far the largest, endeavor to improve assessment of cancer patients’ smoking status. Participants reported significant advances within a short time span and study results underscore the potential for national accreditation organizations to transform oncology practice.

PMID:39561316 | DOI:10.1200/JCO.24.00304

Neurology. 2024 Dec 24;103(12):e209940. doi: 10.1212/WNL.0000000000209940. Epub 2024 Nov 19.

ABSTRACT

OBJECTIVES: In aquaporin-4 antibody-positive neuromyelitis optica spectrum disorder (AQP4-IgG NMOSD), disability accrual is mostly attributed to relapses. This study aimed to assess the prevalence of progression independent of relapse activity (PIRA) and relapse-associated worsening (RAW) in AQP4-IgG NMOSD.

METHODS: This was a retrospective cohort study of patients with AQP4-IgG NMOSD enrolled in the MSBase international data registry. Patients required a minimum of 3 recorded Expanded Disability Status Scale (EDSS) scores: baseline, event, and a 6-month confirmation score. Presence and absence of relapses between the baseline and event EDSS scores determined RAW and PIRA, respectively. Descriptive statistics were used to present the results.

RESULTS: A total of 181 patients followed for a median of 4.5 years (Q1 1.7, Q3 7.8) were included. Most patients were female (88.4%), and the median age at disease onset was 38.1 years. Overall, 4 patients (2.2%) developed 5 incidences of PIRA and 13 patients developed RAW (7.2%).

DISCUSSION: This multicenter study highlights that PIRA is very rare in AQP4-IgG NMOSD. Limitations of this study include the sole focus of overall EDSS to measure disability, lack of requirement for a second EDSS score to confirm baseline EDSS, and the absence of magnetic resonance imaging information for all patients.

PMID:39561307 | DOI:10.1212/WNL.0000000000209940

Clin Infect Dis. 2024 Nov 20:ciae561. doi: 10.1093/cid/ciae561. Online ahead of print.

ABSTRACT

BACKGROUND: Tuberculosis (TB) incidence remains disproportionately high in people migrating to Canada and other low TB incidence countries, but systematic TB screening and prevention in migrants is often cost-prohibitive for TB programs. We aimed to develop and validate a TB risk prediction model to inform TB screening decisions in foreign-born permanent residents of Canada.

METHODS: We developed and validated a proportional baselines landmark supermodel for TB risk prediction using health administrative data from British Columbia and Ontario, two distinct provincial healthcare systems in Canada. Demographic (age, sex, refugee status, year of entry, TB incidence in country of origin), TB exposure, and medical (HIV, kidney disease, diabetes, solid organ transplantation, cancer) covariates were used to derive and test models in British Columbia; one model was chosen for external validation in the Ontario cohort. The model’s ability to predict 2- and 5-year TB risk in the Ontario cohort was assessed using discrimination and calibration statistics.

RESULTS: The study included 715,423 individuals (including 1,407 people with TB disease) in the British Columbia derivation cohort, and 958,131 individuals (including 1,361 people with TB disease) in the Ontario validation cohort. The 2- and 5-year concordance statistic in the validation cohort was 0.77 (95%CI: 0.75-0.78) and 0.77 (95%CI: 0.76-0.78), respectively. Calibration-in-the-large values were 0.14 (95% CI: 0.08-0.21) and -0.05 (95% CI: -0.12-0.02) in 2- and 5-year prediction windows.

CONCLUSIONS: This prediction model, available online at https://tb-migrate.com, may improve TB risk stratification in people migrating to low incidence countries and may help inform TB screening policy and guidelines.

PMID:39561254 | DOI:10.1093/cid/ciae561

Fam Pract. 2024 Nov 19:cmae061. doi: 10.1093/fampra/cmae061. Online ahead of print.

ABSTRACT

INTRODUCTION: The evolving landscape of general practice (GP)/family medicine (FM) in the post-COVID-19 era, focussing on integrating telemedicine and remote consultations requires a new definition for this specialty. Hence, a broader consensus-based definition of post-COVID-19 GP/FM is warranted.

METHODS: This study involved a modified electronic Delphi technique involving 27 specialists working in primary care recruited via convenient and snowball sampling. The Delphi survey was conducted online between August 2022 and April 2023, utilizing the Google Forms platform. Descriptive statistics were employed to analyse consensus across Delphi rounds.

RESULTS: Twenty-six international experts participated in the survey. The retention rate through the second and third Delphi rounds was 96.2% (n = 25). The broader consensus definition emphasizes person-centred care, collaborative patient-physician partnerships, and a holistic approach to health, including managing acute and chronic conditions through in-person or remote access based on patient preferences, medical needs, and local health system organization.

CONCLUSION: The study highlights the importance of continuity of care, prevention, and coordination with other healthcare professionals as core values of primary care. It also reflects the role of GP/FM in addressing new challenges post-pandemic, such as healthcare delivery beyond standard face-to-face care (e.g. remote consultations) and an increasingly important role in the prevention of infectious diseases. This underscores the need for ongoing research and patient involvement to continually refine and improve primary healthcare delivery in response to changing healthcare landscapes.

PMID:39561247 | DOI:10.1093/fampra/cmae061

Circ Cardiovasc Qual Outcomes. 2024 Nov;17(11):e010629. doi: 10.1161/CIRCOUTCOMES.123.010629. Epub 2024 Nov 19.

ABSTRACT

BACKGROUND: There is limited international agreement on defining care quality for the millions of people hospitalized with heart failure worldwide. Our objective was to compare and measure agreement across existing internationally published quality indicators (QIs) for the care of adults hospitalized for heart failure.

METHODS: Systematic review and evidence gap map of internationally published articles reporting on QIs for adults hospitalized for heart failure, using PubMed, MEDLINE, EMBASE, and TRIP from inception to July 18, 2022. Narrative synthesis and descriptive statistics characterized included articles and QIs using the Donabedian Framework of Structural, Process, and Outcomes. The methodological quality of QI sets was assessed using the Appraisal of Indicators through Research and Evaluation instrument. Agreement about QIs was defined as having at least 3 different cardiovascular societies recommend its use. An evidence gap map displayed each QI according to its clinically relevant category, methodological quality, and reporting articles.

RESULTS: Fourteen articles from 11 societies reported 75 unique QIs; 53 QIs were process, 16 were structural, and 7 were outcome measures. There was limited agreement on individual QIs across sets as a minority were recommended by ≥3 societies (12%; 9/75 QIs). The most common QIs included postdischarge follow-up (73%, 8/11 societies), specific pharmacotherapy (64%, 7/11 societies), patient education (45%, 5/11 societies), assessment of left ventricular ejection fraction (45%, 5/11 societies), 30-day readmission rate (45%, 5/11 societies), cardiac rehabilitation (36%, 4/11 societies), and multidisciplinary management (27%, 3/11 societies).

CONCLUSIONS: There was little agreement on defining high-quality care and limited agreement on measures including postdischarge follow-up, specific pharmacotherapies, patient education, assessment of left ventricular ejection fraction, 30-day readmission, cardiac rehabilitation, and multidisciplinary management. These measures may define high-quality care and highlight opportunities to improve the quality of care for adults hospitalized for heart failure.

PMID:39561228 | DOI:10.1161/CIRCOUTCOMES.123.010629

PLOS Digit Health. 2024 Nov 19;3(11):e0000650. doi: 10.1371/journal.pdig.0000650. eCollection 2024 Nov.

ABSTRACT

OBJECTIVES: Despite the development of efficacious wellness interventions, sustainable wellness behavior change remains challenging. To optimize engagement, initiating small behaviors that build upon existing practices congruent with individuals’ lifestyles may promote sustainable wellness behavior change. In this study, we crowd-sourced helpful, flexible, and engaging wellness practices to identify a list of those commonly used for improving sleep, productivity, and physical, emotional, and social wellness from participants who felt they had been successful in these dimensions.

METHOD: We recruited a representative sample of 992 U.S. residents to survey the wellness dimensions in which they had achieved success and their specific wellness practices.

RESULTS: Responses were aggregated across demographic, health, lifestyle factors, and wellness dimension. Exploration of these data revealed that there was little overlap in preferred practices across wellness dimensions. Within wellness dimensions, preferred practices were similar across demographic factors, especially within the top 3-4 most selected practices. Interestingly, daily wellness practices differ from those typically recommended as efficacious by research studies and seem to be impacted by health status (e.g., depression, cardiovascular disease). Additionally, we developed and provide for public use a web dashboard that visualizes and enables exploration of the study results.

CONCLUSIONS: Findings identify personalized, sustainable wellness practices targeted at specific wellness dimensions. Future studies could leverage tailored practices as recommendations for optimizing the development of healthier behaviors.

PMID:39561203 | DOI:10.1371/journal.pdig.0000650

PLoS One. 2024 Nov 19;19(11):e0314005. doi: 10.1371/journal.pone.0314005. eCollection 2024.

ABSTRACT

Limited research has assessed the accuracy of imputation methods in aerobiological datasets. We conducted a simulation study to evaluate, for the first time, the effectiveness of Gappy Singular Value Decomposition (GSVD), a data-driven approach, comparing it with the moving mean interpolation, a statistical approach. Utilizing complete pollen data from two monitoring stations in northeastern Italy for 2022, we randomly generated missing data considering the combination of various proportions (5%, 10%, 25%) and gap lengths (3, 5, 7, 10 days). We imputed 4800 time series using the GSVD algorithm, specifically implemented for this study, and the moving mean algorithm of the “AeRobiology” R package. We assessed imputation accuracy by calculating the Root Mean Square Error and employed multiple linear regression models to identify factors independently affecting the error (e.g. pollen variability, simulation settings). The results showed that the GSVD was as good as the well-established moving mean method and demonstrated its strong generalization capabilities across different data types. However, the imputation error was primarily influenced by pollen characteristics and location, regardless of the imputation method used. High variability in pollen concentrations and the distribution of missing data negatively affected imputation accuracy. In conclusion, we introduced and tested a novel imputation method, demonstrating comparable performance to the statistical approach in aerobiological data reconstruction. These findings contribute to advancing aerobiological data analysis, highlighting the need for improving imputation methods.

PMID:39561200 | DOI:10.1371/journal.pone.0314005

PLoS Comput Biol. 2024 Nov 19;20(11):e1012592. doi: 10.1371/journal.pcbi.1012592. Online ahead of print.

ABSTRACT

Bovine tuberculosis (bTB) has significant socio-economic and welfare impacts on the cattle industry in parts of the world. In the United Kingdom and Ireland, disease control is complicated by the presence of infection in wildlife, principally the European badger. Control strategies tend to be applied to whole populations, but better identification of key sources of transmission, whether individuals or groups, could help inform more efficient approaches. Mechanistic transmission models can be used to better understand key epidemiological drivers of disease spread and identify high-risk individuals and groups if they can be adequately fitted to observed data. However, this is a significant challenge, especially within wildlife populations, because monitoring relies on imperfect diagnostic test information, and even under systematic surveillance efforts (such as capture-mark-recapture sampling) epidemiological events are only partially observed. To this end we develop a stochastic compartmental model of bTB transmission, and fit this to individual-level data from a unique > 40-year longitudinal study of 2,391 badgers using a recently developed individual forward filtering backward sampling algorithm. Modelling challenges are further compounded by spatio-temporal meta-population structures and age-dependent mortality. We develop a novel estimator for the individual effective reproduction number that provides quantitative evidence for the presence of superspreader badgers, despite the population-level effective reproduction number being less than one. We also infer measures of the hidden burden of infection in the host population through time; the relative likelihoods of competing routes of transmission; effective and realised infectious periods; and longitudinal measures of diagnostic test performance. This modelling framework provides an efficient and generalisable way to fit state-space models to individual-level data in wildlife populations, which allows identification of high-risk individuals and exploration of important epidemiological questions about bTB and other wildlife diseases.

PMID:39561196 | DOI:10.1371/journal.pcbi.1012592