跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.17918 · v1

Condensing Law of Security Information

首次提交 2026-11-13 16:57
版本更新 2026-11-13 16:57
雷达观测 2026-11-16 11:42
arXiv ↗PDF ↗

摘要

Scaling laws indicate that the generalization error of a large language model (LLM) decreases monotonically with model size once data is sufficient, and the corollary that larger models outperform smaller ones is now widely accepted. We argue that this corollary fails in security: decisions there hinge on anomalous inputs, which are long-tailed (e.g., zero-day attacks) and therefore poorly served by likelihood fitting. Worse still, the capability brought about by scale (e.g., inferring the intended whole from a corrupted part) leads a LLM silently \emph{repair} the anomaly before any downstream security head ever sees it. We formalize this as a tradeoff between two quantities: the cognitive capability K(W)=I(W;P)K(W)=I(W;P) that a model holds about the world distribution, and the security-information retention R(W)=I(Z;S)I(A;S)R(W)=\frac{I(Z;S)}{I(A;S)} that survives in its intermediate representation. We derive three information-theoretic bounds to connect them: an upper bound on restoration accuracy in terms of K(W)K(W), an upper bound on R(W)R(W) in terms of the restoration error, and a Fano lower bound on security-task error in terms of R(W)R(W). These theorems approximately imply that usable security information is a product of how much security information survives and how well it can be extracted, indicating that optimal usable security information can be achieved at a certain model size. We state this as the \emph{condensing law of security information}: usable security information is unimodal in model size, so security generalization error is U-shaped and is minimized at a finite N⋆N^{\star} rather than decreasing without bound. The experiments on various real-world LLMs validate the condensing law, indicating that there is an optimal model size to solve security issues.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-13 16:57 · 最新观测版本

Condensing Law of Security Information

Scaling laws indicate that the generalization error of a large language model (LLM) decreases monotonically with model size once data is sufficient, and the corollary that larger models outperform smaller ones is now widely accepted. We argue that this corollary fails in security: decisions there hinge on anomalous inputs, which are long-tailed (e.g., zero-day attacks) and therefore poorly served by likelihood fitting. Worse still, the capability brought about by scale (e.g., inferring the intended whole from a corrupted part) leads a LLM silently \emph{repair} the anomaly before any downstream security head ever sees it. We formalize this as a tradeoff between two quantities: the cognitive capability K(W)=I(W;P)K(W)=I(W;P) that a model holds about the world distribution, and the security-information retention R(W)=I(Z;S)I(A;S)R(W)=\frac{I(Z;S)}{I(A;S)} that survives in its intermediate representation. We derive three information-theoretic bounds to connect them: an upper bound on restoration accuracy in terms of K(W)K(W), an upper bound on R(W)R(W) in terms of the restoration error, and a Fano lower bound on security-task error in terms of R(W)R(W). These theorems approximately imply that usable security information is a product of how much security information survives and how well it can be extracted, indicating that optimal usable security information can be achieved at a certain model size. We state this as the \emph{condensing law of security information}: usable security information is unimodal in model size, so security generalization error is U-shaped and is minimized at a finite N⋆N^{\star} rather than decreasing without bound. The experiments on various real-world LLMs validate the condensing law, indicating that there is an optimal model size to solve security issues.

查看此版本解读 →查看此版本原文 ↗