SuperActivator 机制:Transformer 将可靠概念信号集中于尾部
The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail
Cassandra Goldberg, Chaehyeon Kim, Adam Stein, Eric Wong
AI 速读 基于摘要
作者提出并理论刻画了 SuperActivator 机制,即概念对齐注意力会把最可靠的概念证据集中到少数高激活 token 上,并据此改进概念检测与定位。
cs.LG
查看原文摘要
Concept vectors aim to enhance model interpretability by linking learned internal representations with human-understandable semantics, but their practical utility is often limited by noisy and inconsistent activations. In this work, we discover the SuperActivator Mechanism: a transformer dynamic that amplifies concept activation gaps, concentrating the most reliable concept evidence into a small set of highly activated tokens, which we call SuperActivators. We theoretically characterize this mechanism, proving that concept-aligned attention produces two competing effects: a tail amplification effect, where gaps among already-high activations grow disproportionately fast, and an equalization effect that eventually dominates as attention concentrates on extreme tokens. We validate the predicted tail amplification empirically, finding that in-concept activation distributions develop well-separated upper tails consistently across models, image and text modalities, and concept types. Moreover, we demonstrate the practical implications of SuperActivators for concept detection and localization. First, using only high-activation tokens as concept indicators improves detection by up to 14 F1 points over competing baselines. Second, restricting detection to the extreme activation tail is not merely sufficient but essential: including activations beyond the top 5-10% of the in-concept distribution introduces noise that degrades performance. Finally, using SuperActivators as instance-specific attribution targets yields more accurate and faithful concept localization heatmaps than using global concept vectors.