LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs
Bozorgtabar, Behzad, Mahapatra, Dwarikanath, Roy, Sudipta, Naseer, Muzammal, Razzak, Imran, Ge, Zongyuan
Medical vision-language models (VLMs) are strong zero-shot recognizers for medical imaging, but their reliability under domain shift hinges on calibrated uncertainty with guarantees. Split conformal prediction (SCP) offers finite-sample coverage, yet prediction sets often become large (low efficiency) and class-wise coverage unbalanced--high class-conditioned coverage gap (CCV), especially in few-shot, imbalanced regimes; moreover, naively adapting to calibration labels breaks exchangeability and voids guarantees. We propose LATA (Laplacian-Assisted Transductive Adaptation), a training- and label-free refinement that operates on the joint calibration and test pool by smoothing zero-shot probabilities over an image-image kNN graph using a small number of CCCP mean-field updates, preserving SCP validity via a deterministic transform. We further introduce a failure-aware conformal score that plugs into the vision-language uncertainty (ViLU) framework, providing instance-level difficulty and label plausibility to improve prediction set efficiency and class-wise balance at fixed coverage. LATA is black-box (no VLM updates), compute-light (windowed transduction, no backprop), and includes an optional prior knob that can run strictly label-free or, if desired, in a label-informed variant using calibration marginals once. Across three medical VLMs and nine downstream tasks, LATA consistently reduces set size and CCV while matching or tightening target coverage, outperforming prior transductive baselines and narrowing the gap to label-using methods, while using far less compute. Comprehensive ablations and qualitative analyses show that LATA sharpens zero-shot predictions without compromising exchangeability.
尚无 AI 解读，展示原文摘要
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表