Pract Radiat Oncol. 2021 Oct 17:S1879-8500(21)00284-8. doi: 10.1016/j.prro.2021.09.014. Online ahead of print.
ABSTRACT
PURPOSE: Perform a longitudinal analysis of the performance of our automated plan checking software by retrospectively evaluating the number of errors identified in plans delivered to patients in three, month-long, data collection periods taken between 2017 and 2020.
METHODS: 11 automated checks were retrospectively run on 1169 external beam radiotherapy treatment plans identified as meeting the following criteria: planning target volume (PTV) based multi-field photon plans receiving a status of treatment approved in either March 2017, March 2018, or March 2020. The number of “passes” (true positives) and “flags” were recorded. “Flags” were sub-categorized into “false negatives”, “false negatives due to naming conventions”, or “true negatives”. 2 × 2 contingency tables using a two-tailed Fischer’s exact test were utilized to determine whether there were nonrandom associations between the output of the automated plan checking software and whether the check was manual or automated at the original time of treatment approval.
RESULTS: A statistically significant decrease in flags between the pre- and post-automation datasets was observed for four contour-based checks, namely “adjacent structures overlap”, “empty structures and missing slices”, “overlap between body and couch”, and “laterality”, as well as a check that determined whether the plan’s global maximum dose was within the planning target volume. Review of the origins of false negatives was fed back into the design of the checks to improve the reliability of the system and help avoid warning fatigue.
CONCLUSIONS: Periodic and longitudinal review of the performance of automated software is essential for monitoring and understanding its impact on error rates as well as for optimization of the tool to adapt to regular changes of clinical practice. The automated plan checking software has demonstrated continuous contributions to the safe and effective delivery of external beam radiotherapy to our patient population, an impact that extends beyond its initial implementation and deployment.
PMID:34670137 | DOI:10.1016/j.prro.2021.09.014
