← 研踪 PaperTrailarXiv:2606.05107 · v2
Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Have
首次提交 2026-06-04 01:10
版本更新 2026-11-09 20:46
雷达观测 2026-11-10 15:22
摘要
We propose a label-free approach to adapt powerful but generic vision foundation models to specialized scientific domains. Standard supervised fine-tuning is often ill-suited to these settings: labels are scarce, and task-specific training can reduce the generality needed to cope with complex domain shifts. We instead leverage metadata to adapt representations to new domains in a weakly-supervised manner. Our method, FINO, combines a standard self-supervised objective with flexible metadata guidance that handles both highly granular discrete metadata and continuous metadata. It encourages the representation to preserve informative factors while suppressing spurious ones. Across subcellular fluorescence microscopy, Earth observation, wildlife monitoring, and medical imaging, FINO outperforms matched self-supervised and fully supervised adaptation baselines. Complete-adaptation repeats confirm consistent performance across domains. Without task labels for backbone adaptation, FINO also obtains higher reported scores than selected specialised systems on three domains and matches the fourth under differing cross-paper protocols. Code is available at github.com/facebookresearch/dinov3/tree/FINO.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-09 20:46 · 最新观测版本
Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Have
We propose a label-free approach to adapt powerful but generic vision foundation models to specialized scientific domains. Standard supervised fine-tuning is often ill-suited to these settings: labels are scarce, and task-specific training can reduce the generality needed to cope with complex domain shifts. We instead leverage metadata to adapt representations to new domains in a weakly-supervised manner. Our method, FINO, combines a standard self-supervised objective with flexible metadata guidance that handles both highly granular discrete metadata and continuous metadata. It encourages the representation to preserve informative factors while suppressing spurious ones. Across subcellular fluorescence microscopy, Earth observation, wildlife monitoring, and medical imaging, FINO outperforms matched self-supervised and fully supervised adaptation baselines. Complete-adaptation repeats confirm consistent performance across domains. Without task labels for backbone adaptation, FINO also obtains higher reported scores than selected specialised systems on three domains and matches the fourth under differing cross-paper protocols. Code is available at github.com/facebookresearch/dinov3/tree/FINO.
查看此版本解读 →查看此版本原文 ↗