Multi-modal segment anything model (mmSAM) for tumor segmentation in multi-tracer oncologic PET/CT

EJNMMI Phys. 2026 May 17. doi: 10.1186/s40658-026-00887-z. Online ahead of print.

ABSTRACT

PURPOSE: Lesion segmentation is important in targeted radionuclide therapy dosimetry. In this study, we proposed a novel 2D multi-modal Segment Anything Model (mmSAM) for lesion segmentation in whole-body multi-tracer PET/CT images.

METHODS: The AutoPET 2024 dataset, including ¹⁸F-PSMA (369 subjects), ¹⁸F-FDG (170 subjects) and ⁶⁸Ga-PSMA (170 subjects) PET/CT images and their corresponding tumor masks, was used in this study. The ¹⁸F-PSMA PET/CT dataset was used as the primary dataset and was divided into 233: 36: 100 for training, validation and testing the mmSAM, using both PET and CT input. The transferability of the primary model was evaluated on 100 patients on other 2 datasets, without and with fine-tuning using 70 cases of ¹⁸F-FDG and ⁶⁸Ga-PSMA respectively. Standard single modal 2D SAM with only PET input, 3D nnUNet with two-channel PET/CT input and the thresholding-based method were also implemented for comparison. Mean Dice, the 95th percentile Hausdorff distance (mean HD95), mean standardized uptake value (mean |SUV_mean| error), metabolic tumor volume (mean |MTV| error), true positive rate (TPR), positive predictive value (PPV), and false discovery rate (FDR) were computed for all tumors. Statistical significance among different segmentation methods was evaluated using the Wilcoxon test.

RESULTS: mmSAM achieved the best performance as compared to other segmentation methods for the primary ¹⁸F-PSMA dataset (mean Dice/HD95/|SUV_mean| error/|MTV| error/TPR/PPV/FDR = 0.76/1.92 mm/5.10%/14.60%/100%/96.51%/3.49%, all p < 0.05), without fine-tuning (mean Dice/HD95/|SUV_mean| error/|MTV| error/TPR/PPV/FDR = 0.61/2.28 mm/ 15.83%/27.71%/100%/78.91%/21.09% for ¹⁸F-FDG; 0.77/1.33 mm/5.55%/17.96%/100% /97.78%/2.22% for ⁶⁸Ga-PSMA, p < 0.05), as well as with fine-tuning (mean Dice/HD95/|SUV_mean| error/|MTV| error/TPR/PPV/FDR = 0.65/1.81 mm/6.62%/13.93%/100%/ 80.50%/19.50% for ¹⁸F-FDG; 0.81/1.15 mm/4.62%/14.30%/100%/99.30%/0.70% for ⁶⁸Ga-PSMA, p < 0.05) on the cross-tracer datasets.

CONCLUSION: The proposed mmSAM is promising for lesion segmentation in multi-tracer oncologic PET/CT images. Fine-tuning significantly enhances segmentation accuracy for cross-tracer studies.

PMID:42144442 | DOI:10.1186/s40658-026-00887-z