论文库

原文摘要

Modern LLM workflows move coordinate-indexed objects across checkpoints: steering vectors, sparse autoencoders, top-$k$ neuron sets, attribution lists, LoRA updates, optimizer states, and merge alignments. This requires fixing the model's residual-stream gauge. We show that the native coordinate-preserving orthogonal gauge is architecture-dependent: LayerNorm residual charts have permutation gauge $S_d$ up to a global sign flip, while RMSNorm charts with generic gains have signed-permutation gauge $B_d = S_d \ltimes \{\pm 1\}^d$. Permutation-only alignment is thus symmetry-incomplete for RMSNorm models. We introduce sign-marginalized Hungarian matching and prove that with decorrelated coordinates, signed-correlation matching has a permutation-accuracy ceiling equal to the true gauge's positive-sign fraction up to $O(d 2^{-d})$, which sign-marginalized matching removes. Omitting signs induces expected squared error $2\|ΔW\|_F^2$ on weight updates and loses orientation on vectors and AdamW first moments, while RMSNorm gains and AdamW second moments permute without signs. Controlled tests confirm these laws: a Qwen2.5-1.5B SST-2 LoRA adapter retains 0.944 accuracy (from 0.945) under $B_d$ transport vs. 0.523 under $S_d$, and steering preserves effects to $<10^{-4}$ at zero raw top-10 overlap. Across natural checkpoints, inherited slots suffice for same-base descendants (96.2%-100% retention), whereas cross-generation pairs support sparse prompt-stable subsets with causally decisive negative signs ($+0.185$ vs. $-0.010$ at $8σ$ on Qwen2/2.5-0.5B; $+0.189$ vs. $-0.021$ on 1.5B). Along fine-tuning trajectories, composing local $B_d$ gauges recovers 91.1% of cross-run Qwen2.5-1.5B coordinates at 1,500 steps vs. 60.3% for endpoint matching. Merge and interpretability audits confirm that coordinate names are reproducible only relative to an explicit $B_d$ gauge.