← 研踪 PaperTrailarXiv:2611.00606 · v2
Sparse Autoencoder Attribution Graphs Are Decided at the Margin
首次提交 2026-10-31 12:03
版本更新 2026-11-15 04:57
雷达观测 2026-11-17 12:06
摘要
Attribution graphs predict sparse autoencoder (SAE) ablations to first order on the active features, but a TopK SAE passes only its k largest preactivations, so small changes often swap active features. On five public TopK SAEs, zeroing one feature changes the active set of the next layer's SAE in 74% to 89% of cases. Our margin graph adds the first order change to every downstream preactivation, then selects the k largest again. On Pythia-70M the margin graph cuts the median error of predicted activations, relative to their exact change, from the attribution graph's 0.739 to 0.121, 84% lower. Exact recomputation of 48 interventions runs 3.5 times faster than the margin graph on our GPU. Patched into Pythia-70M with the clean reconstruction error, the margin graph's activations predict the change in the subject verb agreement logit difference with 1.95 times the attribution graph's mean error, on interventions above a fixed floor. Between seeds of our SAEs, which share one auxiliary loss coefficient, sparse feature maps recover 85% of a graph's squared norm on active features but 47% with each graph masked to its own, 38 percentage points less. Unmasking the inactive tenth nearest the TopK threshold recovers 73% of the masking loss, against 11% for a random tenth, so the margin to that threshold decides how ablations change activations and what graphs carry between seeds.
作者备注:44 pages, 8 figures
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-15 04:57 · 最新观测版本
Sparse Autoencoder Attribution Graphs Are Decided at the Margin
Attribution graphs predict sparse autoencoder (SAE) ablations to first order on the active features, but a TopK SAE passes only its k largest preactivations, so small changes often swap active features. On five public TopK SAEs, zeroing one feature changes the active set of the next layer's SAE in 74% to 89% of cases. Our margin graph adds the first order change to every downstream preactivation, then selects the k largest again. On Pythia-70M the margin graph cuts the median error of predicted activations, relative to their exact change, from the attribution graph's 0.739 to 0.121, 84% lower. Exact recomputation of 48 interventions runs 3.5 times faster than the margin graph on our GPU. Patched into Pythia-70M with the clean reconstruction error, the margin graph's activations predict the change in the subject verb agreement logit difference with 1.95 times the attribution graph's mean error, on interventions above a fixed floor. Between seeds of our SAEs, which share one auxiliary loss coefficient, sparse feature maps recover 85% of a graph's squared norm on active features but 47% with each graph masked to its own, 38 percentage points less. Unmasking the inactive tenth nearest the TopK threshold recovers 73% of the masking loss, against 11% for a random tenth, so the margin to that threshold decides how ablations change activations and what graphs carry between seeds.
查看此版本解读 →查看此版本原文 ↗