跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.25834 · v1

Rank2Replace: VLM-Ranked Slot Replacement for Out-of-Distribution Generalization

首次提交 2026-11-19 17:50
版本更新 2026-11-19 17:50
雷达观测 2026-11-20 19:57
arXiv ↗PDF ↗

摘要

Spurious correlations can give image predictors high in-distribution accuracy but poor generalization under distribution shift. We introduce Rank2Replace\textbf{Rank2Replace}, which transfers knowledge of task-relevant evidence from pretrained vision--language models (VLMs) to small predictors through training-time augmentation for out-of-distribution (OOD) generalization. The VLM ranks object-centric slots by task relevance and guides rank-dependent replacement with same-rank slot embeddings from another image. This augmentation retains the original image's label, encouraging reliance on preserved task evidence as incidental cues change, and supports combination with existing OOD methods. Across four datasets and six splits under shared frozen slot embeddings, the default cross-entropy configuration reaches macro-averaged average-group and worst-group accuracies of 80.43%80.43\% and 65.28%65.28\% without environment labels or dataset-specific replacement tuning. Slot corruption improves both macro-averaged metrics for all fourteen OOD training methods. Combining it with slot ranking and IRM reaches 81.96%81.96\% and 72.68%72.68\%. Controlled ablations support the ranking and replacement design, with gains across five ranking VLMs and six extractor--predictor combinations. Permutation-invariant predictors further improve both macro-averaged metrics over their unaugmented counterparts while using VLM guidance only during training for efficient inference.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-19 17:50 · 最新观测版本

Rank2Replace: VLM-Ranked Slot Replacement for Out-of-Distribution Generalization

Spurious correlations can give image predictors high in-distribution accuracy but poor generalization under distribution shift. We introduce Rank2Replace\textbf{Rank2Replace}, which transfers knowledge of task-relevant evidence from pretrained vision--language models (VLMs) to small predictors through training-time augmentation for out-of-distribution (OOD) generalization. The VLM ranks object-centric slots by task relevance and guides rank-dependent replacement with same-rank slot embeddings from another image. This augmentation retains the original image's label, encouraging reliance on preserved task evidence as incidental cues change, and supports combination with existing OOD methods. Across four datasets and six splits under shared frozen slot embeddings, the default cross-entropy configuration reaches macro-averaged average-group and worst-group accuracies of 80.43%80.43\% and 65.28%65.28\% without environment labels or dataset-specific replacement tuning. Slot corruption improves both macro-averaged metrics for all fourteen OOD training methods. Combining it with slot ranking and IRM reaches 81.96%81.96\% and 72.68%72.68\%. Controlled ablations support the ranking and replacement design, with gains across five ranking VLMs and six extractor--predictor combinations. Permutation-invariant predictors further improve both macro-averaged metrics over their unaugmented counterparts while using VLM guidance only during training for efficient inference.

查看此版本解读 →查看此版本原文 ↗