JMIR Med Inform. 2026 Jul 15. doi: 10.2196/99639. Online ahead of print.
ABSTRACT
BACKGROUND: Clinical retrieval-augmented generation depends on embedding models. A companion study found that non-retrieval-trained encoders underperformed retrieval-trained general-purpose embeddings and produced near-degenerate embedding geometry, but did not localize the architectural origin, separate training-domain from training-objective effects, or test whether the degradation can be corrected without retraining.
OBJECTIVE: To (a) characterize layer-wise retrieval and geometric trajectories across 13 transformer configurations on clinical documents; (b) separate training-objective from training-domain effects through matched architectural comparisons; (c) re-analyze the panel under a per-query layer-wise linear mixed-effects framework; and (d) evaluate deployment-relevant post-hoc geometric correction.
METHODS: Layer-wise embeddings were extracted from 13 transformer configurations on 3 clinical corpora (n=100 documents each: MTSamples, PMC-Patients, and Mistral-7B-Instruct-generated synthetic notes) under 2 query formats (keyword and natural-language metadata-derived queries via GPT-4o). The panel was extended beyond the companion study by adding BERT-base-uncased and BioMistral-7B as matched controls. Retrieval performance (MRR@10, Recall@10) and geometric properties (participation ratio, average pairwise cosine, anisotropy) were measured at every layer. A per-query layer-wise linear mixed-effects model was fit independently per configuration as the primary inferential analysis. Corpus-only ZCA whitening was evaluated as the primary deployment-relevant intervention, with 5-fold cross-validation, an epsilon sweep, a lexical-overlap audit, and a chunking sensitivity analysis.
RESULTS: Document embeddings clustered into three anisotropy tiers: extreme (avg pairwise cosine>0.92) for non-retrieval-trained encoders and LLMs, moderate (0.65-0.92) for general retrievers and most LLMs, and reduced (<0.65) for BioLORD-2023, instruction-tuned E5-Mistral-7B, and Nomic-embed-text-nopfx. The per-query random-slope LME identified two layer-depth patterns: classical degradation with depth in three non-retrieval-trained encoders (all P<.001), versus net improvement with depth in the remaining 10 models (all P<.001), with the strongest negative coefficients in decoder LLMs. Formal matched-contrast tests confirmed statistically significant training-objective × layer-depth interactions in all three matched pairs (BERT-base-uncased vs BGE-base, P<.001; BioMistral-7B vs E5-Mistral-7B, P<.001; BioBERT vs BioLORD-2023, P<.001). Corpus-only ZCA whitening produced a two-tier pattern under 5-fold cross-validation: Tier 2 non-retrieval-trained models showed positive ΔMRR@10 (+0.066 to +0.304), while Tier 1 retrieval-trained models showed negative ΔMRR@10 (-0.021 to -0.051), confirming held-out generalization. BM25-versus-embedding Spearman rank correlations spanned -0.02-0.37, indicating substantial non-lexical contribution to retrieval. Same-source ranking stability replicated at 4-5× corpus scale (ρ=0.952 for PMC-500, ρ=0.929 for MTSamples-400) for the BERT-scale subset.
CONCLUSIONS: Anisotropy in transformer embeddings is widespread across architectural classes and is lower in configurations with retrieval-specific training. Corpus-only ZCA whitening is a deployment-compatible, retraining-free post-hoc correction candidate that improved retrieval for non-retrieval-trained models on this controlled benchmark but requires target-corpus validation before clinical deployment. The matched-comparison evidence supports training objective rather than training domain as the stronger explanatory axis, though residual confounding is not eliminated. The principal contribution is mechanistic: layer-level localization of embedding degradation and the geometric basis for the two-tier intervention response.
PMID:42455615 | DOI:10.2196/99639
