arXiv:2608.28327 · v2
How Diverse Are Layered Safeguards? Estimands and Measurement for Ensembles of LLM Jailbreak Defenses
摘要
Practitioners protect large language models (LLMs) by stacking safeguards: a likelihood filter, a guard classifier, hidden-state probes, a decoding-time intervention. Such a stack is an ensemble of learned detectors, and ensemble theory states when combination pays: when members fail on different inputs. Stacking is recommended throughout the LLM security literature, but this condition has not been measured for any pair of safeguards. The condition is three quantities, not one, and conflating them is why it has resisted measurement. Separately optimized attacks license only a claim about which \emph{behaviors} are jointly vulnerable; the identity that governs a deployed stack is written over shared \emph{inputs}; and neither predicts what an adversary attacking the assembled pipeline achieves. We define the three estimands, give an estimator for each, and supply two controls: the correlation between two independent runs of the same attack, which sets the level a pair must clear, and a multilevel model separating detector diversity from behavior difficulty. On a seven-layer stack and two open-weight targets, no pair of safeguards is distinguishable from the correlation that two independent runs of the attack produce with no safeguard present, and behavior difficulty explains much of what remains. A second attack family, an attacker model rewriting its own prompts, reaches that verdict independently on a thirtieth of the budget and overturns the practical one: the only member that blocked the first adversary admits 0.19 under the second, against 0.03. Diversity between neural safeguards is measurable, but it and any member's strength are properties of a (stack, adversary, target) triple.
作者备注:40 pages, 4 figures. Submitted to Machine Learning. v2: major revision with a new title. The survey in v1 is removed and the experiment is rebuilt; v1 results are superseded. Authors changed: v2 is by A. Alotaibi and M. Ahmed. M. S. Jabbar and S. Al-Azani, authors of v1, took part in writing parts of the survey, which is not in v2. Code: https://github.com/AbrarAlotaibi/defense-correlation