J Surg Res. 2025 Nov 22;316:53-58. doi: 10.1016/j.jss.2025.10.031. Online ahead of print.
ABSTRACT
INTRODUCTION: Standardized letters of recommendation (SLORs) have been adopted in colon and rectal surgery to reduce bias and improve consistency in applicant evaluation. Despite their widespread use, concerns remain about their limited ability to distinguish between applicants and their susceptibility to inflated ratings and narrative bias. This study aims to identify program directors’ perspectives on the current SLOR and guide the development of an improved tool to improve utility, fairness, and meaningful assessment in the selection process.
METHODS: A needs assessment survey was distributed to all Accreditation Council for Graduate Medical Education-accredited colon and rectal surgery residency program directors via the American Program Directors in Colon and Rectal Surgery listserv in Spring 2025. It included 7 Likert-style questions (including a 26-item rating matrix) and 5 free-text questions designed to assess satisfaction and perspectives on what should be included in a revised SLOR. Descriptive statistics were used to analyze quantitative responses; qualitative responses were thematically coded. Results were presented for comment at the American Program Directors in Colon and Rectal Surgery annual meeting and feedback was collected. A new SLOR form was created.
RESULTS: Fifty-two program directors responded (70% participation rate). While 62% found the current SLOR somewhat useful, only 27% rated it as very useful. Primary concerns included lack of discrimination due to inflated ratings, superlative narrative comments, and inconsistent use across institutions. Clinical judgment (98%), professionalism (93%), and interpersonal skills (91%) were identified as the most valued domains. Respondents reported mixed views on the value of narrative comments and identified the current “weakness” field as unhelpful or harmful. Suggestions for improvement included a more meaningful rating scale, structured free-text prompts with character limits, and clearer anchors for competency assessment. A revised SLOR incorporating these features was pilot-tested and refined.
CONCLUSIONS: Colorectal surgery program directors express limited satisfaction with the current SLOR due to poor discriminatory capacity and inconsistent narrative content. An updated form-developed through user feedback and best practices in evaluation-was created to enhance clarity, fairness, and utility in the application process. Its effectiveness will be assessed after implementation in the 2025 match cycle. These findings may guide SLOR redesigns in other specialties seeking to enhance holistic and equitable applicant assessment.
PMID:41275532 | DOI:10.1016/j.jss.2025.10.031
