← 研踪 PaperTrailarXiv:2604.15559 · v2
Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation
首次提交 2026-04-17 06:23
版本更新 2026-11-16 03:00
雷达观测 2026-11-17 17:12
摘要
Recent work on subliminal learning has shown that language models can transmit semantic traits through data with no apparent connection to those traits. Whether this phenomenon extends to agentic systems, where policies are learned from trajectories rather than static text, remains an open question with critical safety implications. We present the first empirical evidence that unsafe agent behaviors transfer subliminally through model distillation, demonstrated across several settings. In the primary setting, we construct a teacher agent with a deletion bias, a tendency to perform destructive file-system actions through an API-style tool interface, and distill it into a student using only trajectories from ostensibly safe tasks, with all explicit deletion keywords filtered. In the secondary setting, we replicate the threat model with shell commands, replacing API calls with shell commands and operationalizing the bias as a preference for chmod over semantically equivalent alternatives (e.g., chown, setfacl) when issuing the first permission-related command. Despite sanitation, students inherit measurable biases. Over 1,500 trials per configuration, the API student chooses a destructive action first on 35.5% of ambiguous tasks (vs. 1.2% baseline), and the Bash student's chmod-first rate reaches 30-55% (vs. 0-10% baseline). In a multi-turn environment where tool calls execute, 33.9% of student episodes end with a resource irreversibly destroyed (vs. 6.4% baseline). On TerminalBench we observe a smaller effect (7.7% vs. 0% baseline). Explicit data sanitation is therefore insufficient as a defense; mitigating behavioral bias transfer in agentic systems will require fundamentally new strategies.
作者备注:Accepted at NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 03:00 · 最新观测版本
Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation
Recent work on subliminal learning has shown that language models can transmit semantic traits through data with no apparent connection to those traits. Whether this phenomenon extends to agentic systems, where policies are learned from trajectories rather than static text, remains an open question with critical safety implications. We present the first empirical evidence that unsafe agent behaviors transfer subliminally through model distillation, demonstrated across several settings. In the primary setting, we construct a teacher agent with a deletion bias, a tendency to perform destructive file-system actions through an API-style tool interface, and distill it into a student using only trajectories from ostensibly safe tasks, with all explicit deletion keywords filtered. In the secondary setting, we replicate the threat model with shell commands, replacing API calls with shell commands and operationalizing the bias as a preference for chmod over semantically equivalent alternatives (e.g., chown, setfacl) when issuing the first permission-related command. Despite sanitation, students inherit measurable biases. Over 1,500 trials per configuration, the API student chooses a destructive action first on 35.5% of ambiguous tasks (vs. 1.2% baseline), and the Bash student's chmod-first rate reaches 30-55% (vs. 0-10% baseline). In a multi-turn environment where tool calls execute, 33.9% of student episodes end with a resource irreversibly destroyed (vs. 6.4% baseline). On TerminalBench we observe a smaller effect (7.7% vs. 0% baseline). Explicit data sanitation is therefore insufficient as a defense; mitigating behavioral bias transfer in agentic systems will require fundamentally new strategies.
查看此版本解读 →查看此版本原文 ↗