← 研踪 PaperTrailarXiv:2605.21849 · v2
Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
首次提交 2026-05-21 08:46
版本更新 2026-11-04 13:43
雷达观测 2026-11-05 11:10
摘要
Mechanistic interpretability aims to explain a model's behavior by identifying causally responsible internal structures. Dictionary-based explainers such as sparse autoencoders and transcoders are a primary tool, but their faithfulness under out-of-distribution (OOD) shift has received little systematic attention. We show that distribution shift rotates the subspace that the model actively uses, misaligning the explainer's dictionary trained on in-distribution (ID) activations. We formalize this misalignment as the faithfulness gap, a geometric distance between the ID dictionary and the OOD-active subspace, and show that it bounds excess OOD projection loss. To reduce this gap, we propose the Geometry-Adaptive Explainer (GAE), which realigns the dictionary with the OOD-active subspace while preserving the original sparse feature activations. This requires only unlabeled OOD activations and no gradient updates. For the adapted dictionary, we also bound the excess projection loss caused by any remaining faithfulness gap. Empirically, GAE even matches or surpasses all training-based baselines in causal faithfulness across multiple models and OOD settings.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-04 13:43 · 最新观测版本
Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
Mechanistic interpretability aims to explain a model's behavior by identifying causally responsible internal structures. Dictionary-based explainers such as sparse autoencoders and transcoders are a primary tool, but their faithfulness under out-of-distribution (OOD) shift has received little systematic attention. We show that distribution shift rotates the subspace that the model actively uses, misaligning the explainer's dictionary trained on in-distribution (ID) activations. We formalize this misalignment as the faithfulness gap, a geometric distance between the ID dictionary and the OOD-active subspace, and show that it bounds excess OOD projection loss. To reduce this gap, we propose the Geometry-Adaptive Explainer (GAE), which realigns the dictionary with the OOD-active subspace while preserving the original sparse feature activations. This requires only unlabeled OOD activations and no gradient updates. For the adapted dictionary, we also bound the excess projection loss caused by any remaining faithfulness gap. Empirically, GAE even matches or surpasses all training-based baselines in causal faithfulness across multiple models and OOD settings.
查看此版本解读 →查看此版本原文 ↗