跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2608.28327 · v2

How Diverse Are Layered Safeguards? Estimands and Measurement for Ensembles of LLM Jailbreak Defenses

首次提交 2026-08-28 21:36
版本更新 2026-11-14 21:42
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

Practitioners protect large language models (LLMs) by stacking safeguards: a likelihood filter, a guard classifier, hidden-state probes, a decoding-time intervention. Such a stack is an ensemble of learned detectors, and ensemble theory states when combination pays: when members fail on different inputs. Stacking is recommended throughout the LLM security literature, but this condition has not been measured for any pair of safeguards. The condition is three quantities, not one, and conflating them is why it has resisted measurement. Separately optimized attacks license only a claim about which \emph{behaviors} are jointly vulnerable; the identity that governs a deployed stack is written over shared \emph{inputs}; and neither predicts what an adversary attacking the assembled pipeline achieves. We define the three estimands, give an estimator for each, and supply two controls: the correlation between two independent runs of the same attack, which sets the level a pair must clear, and a multilevel model separating detector diversity from behavior difficulty. On a seven-layer stack and two open-weight targets, no pair of safeguards is distinguishable from the correlation that two independent runs of the attack produce with no safeguard present, and behavior difficulty explains much of what remains. A second attack family, an attacker model rewriting its own prompts, reaches that verdict independently on a thirtieth of the budget and overturns the practical one: the only member that blocked the first adversary admits 0.19 under the second, against 0.03. Diversity between neural safeguards is measurable, but it and any member's strength are properties of a (stack, adversary, target) triple.

作者备注:40 pages, 4 figures. Submitted to Machine Learning. v2: major revision with a new title. The survey in v1 is removed and the experiment is rebuilt; v1 results are superseded. Authors changed: v2 is by A. Alotaibi and M. Ahmed. M. S. Jabbar and S. Al-Azani, authors of v1, took part in writing parts of the survey, which is not in v2. Code: https://github.com/AbrarAlotaibi/defense-correlation

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-14 21:42 · 最新观测版本

How Diverse Are Layered Safeguards? Estimands and Measurement for Ensembles of LLM Jailbreak Defenses

Practitioners protect large language models (LLMs) by stacking safeguards: a likelihood filter, a guard classifier, hidden-state probes, a decoding-time intervention. Such a stack is an ensemble of learned detectors, and ensemble theory states when combination pays: when members fail on different inputs. Stacking is recommended throughout the LLM security literature, but this condition has not been measured for any pair of safeguards. The condition is three quantities, not one, and conflating them is why it has resisted measurement. Separately optimized attacks license only a claim about which \emph{behaviors} are jointly vulnerable; the identity that governs a deployed stack is written over shared \emph{inputs}; and neither predicts what an adversary attacking the assembled pipeline achieves. We define the three estimands, give an estimator for each, and supply two controls: the correlation between two independent runs of the same attack, which sets the level a pair must clear, and a multilevel model separating detector diversity from behavior difficulty. On a seven-layer stack and two open-weight targets, no pair of safeguards is distinguishable from the correlation that two independent runs of the attack produce with no safeguard present, and behavior difficulty explains much of what remains. A second attack family, an attacker model rewriting its own prompts, reaches that verdict independently on a thirtieth of the budget and overturns the practical one: the only member that blocked the first adversary admits 0.19 under the second, against 0.03. Diversity between neural safeguards is measurable, but it and any member's strength are properties of a (stack, adversary, target) triple.

查看此版本解读 →查看此版本原文 ↗