← 研踪 PaperTrailarXiv:2609.06951 · v4
Steering Interference Reflects the Model's Defaults, Not the Behavior Directions
首次提交 2026-09-07 10:45
版本更新 2026-11-09 00:03
雷达观测 2026-11-10 12:16
摘要
Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that on average it points to the model rather than the behavior being steered. Steers mostly lower what the unsteered answer shows and raise a set of behaviors the model falls back on (on Qwen2.5-7B, for example, chiefly what the judge scores as refusal, sycophancy, and poeticism). Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: on Qwen2.5-7B all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, on average, directions carrying no behavioral content, of a real steer's size and layer, move the same behaviors in a similar order as real steers do, clearly on five models, weakly on five. Second, much interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains little of the interference it takes part in. The one-way structure and the failure of geometry hold on all ten models. Base models share the one-way structure, yet only one of four has refusal or sycophancy among the five behaviors steers induce most, so post-training appears to install them. Neither geometry nor random directions predicts one pair of behaviors well. For single-layer additive difference-of-means steering, this implies that disentangling behavior directions cannot by itself make steering modular.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v4 · 2026-11-09 00:03 · 最新观测版本
Steering Interference Reflects the Model's Defaults, Not the Behavior Directions
Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that on average it points to the model rather than the behavior being steered. Steers mostly lower what the unsteered answer shows and raise a set of behaviors the model falls back on (on Qwen2.5-7B, for example, chiefly what the judge scores as refusal, sycophancy, and poeticism). Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: on Qwen2.5-7B all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, on average, directions carrying no behavioral content, of a real steer's size and layer, move the same behaviors in a similar order as real steers do, clearly on five models, weakly on five. Second, much interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains little of the interference it takes part in. The one-way structure and the failure of geometry hold on all ten models. Base models share the one-way structure, yet only one of four has refusal or sycophancy among the five behaviors steers induce most, so post-training appears to install them. Neither geometry nor random directions predicts one pair of behaviors well. For single-layer additive difference-of-means steering, this implies that disentangling behavior directions cannot by itself make steering modular.
查看此版本解读 →查看此版本原文 ↗