A²：更小的自监督ViT定位效果更好
$A^2$: Smaller Self-Supervised ViTs Localize Better than Larger Ones
Sreehari Rammohan, Huy Ha, Carl Vondrick
AI 速读 基于摘要
作者发现较小的自监督ViT的注意力图在定位前景物体上优于较大的ViT，并提出A²方法，将“看哪里”与“提取什么”解耦，以兼顾定位与表征。
cs.CV
查看原文摘要
Robust visual classification often depends on localizing the main foreground objects in an image while ignoring contextual distractors. Surprisingly, we find that the last layer attention maps of smaller self-supervised ViTs localize foreground objects better than those of larger ViTs. However, we still need large ViTs, because they extract richer representations from each patch. To get the best of both worlds, good localization and rich representations, we propose $A^2$, a simple method that leverages this inverse scaling finding by decoupling where to look (a small attention model) from what to extract (a large embedding model): we crop around the attention peaks of a small model and embed the crops with a larger model. $A^2$ uses entirely pretrained features, requires no group labels, and does not require per-dataset attention or backbone training. Across 5 benchmarks, $A^2$ is competitive with backbone-matched loss-level methods like DFR, and outperforms end-to-end attention training under stronger distribution shifts.