Performance of ChatGPT-5 on the Polish State Specialization Examination in General Surgery: An Evaluation Study

Pol Przegl Chir. 2026 Jun 8;98(3):1-9. doi: 10.5604/01.3001.0055.7863.

ABSTRACT

<p><strong>Introduction:</strong> Artificial intelligence, particularly large language models like ChatGPT-5.0, is increasingly applied in medical education and decision support. At the same time, there is a lack of research assessing the effectiveness of the latest language models in high-stakes specialized examinations in Poland, which justifies the need for such an analysis.</p><p><strong>Aim: </strong>This study evaluates ChatGPT-5.0’s performance on the Polish Specialization Examination (PES) in general surgery.</p><p><strong>Methods:</strong> A total of 701 single-choice questions from six PES sessions (2023-2025) were analyzed. The questions were divided into various categories. Each question was independently posed to ChatGPT-5.0 three times, and responses were compared with official answer keys. AI performance was compared with that of residents.</p><p><strong>Results:</strong> ChatGPT-5.0 achieved scores ranging from 75.6% to 82.8%, with an overall accuracy of 81-84%, consistently exceeding both the 60% pass threshold and the average score of residents. While ChatGPT-5.0 occasionally outperformed the top-performing residents, the highest human scores were superior in most sessions. Confidence scores were positively correlated with answer accuracy.</p><p><strong>Conclusions:</strong> ChatGPT-5.0 demonstrates strong written exam performance in general surgery. These findings highlight the potential of AI in medical education and exam preparation, while underscoring the limitations of single-choice assessments for clinical competence.</p><p><strong>The significance of the study:</strong> This study provides the first systematic evaluation of ChatGPT-5.0 on the National Specialty Examination in General Surgery in Poland, offering new insights into how advanced AI models perform across clinical domains and cognitive task types, and highlighting their potential role in future surgical training frameworks.</p&gt.

PMID:42439001 | DOI:10.5604/01.3001.0055.7863