Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation
Jacob Dang, Brian Y. Xie, Omar G. Younis
原文摘要
Recent work on subliminal learning has shown that language models can transmit semantic traits through data with no apparent connection to those traits. Whether this phenomenon extends to agentic systems, where policies are learned from trajectories rather than static text, remains an open question with critical safety implications. We present the first empirical evidence that unsafe agent behaviors transfer subliminally through model distillation, demonstrated across several settings. In the primary setting, we construct a teacher agent with a deletion bias, a tendency to perform destructive file-system actions through an API-style tool interface, and distill it into a student using only trajectories from ostensibly safe tasks, with all explicit deletion keywords filtered. In the secondary setting, we replicate the threat model with shell commands, replacing API calls with shell commands and operationalizing the bias as a preference for chmod over semantically equivalent alternatives (e.g., chown, setfacl) when issuing the first permission-related command. Despite sanitation, students inherit measurable biases. Over 1,500 trials per configuration, the API student chooses a destructive action first on 35.5% of ambiguous tasks (vs. 1.2% baseline), and the Bash student's chmod-first rate reaches 30-55% (vs. 0-10% baseline). In a multi-turn environment where tool calls execute, 33.9% of student episodes end with a resource irreversibly destroyed (vs. 6.4% baseline). On TerminalBench we observe a smaller effect (7.7% vs. 0% baseline). Explicit data sanitation is therefore insufficient as a defense; mitigating behavioral bias transfer in agentic systems will require fundamentally new strategies.
cs.AI