跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2604.07238 · v2

On the Price of Privacy for Language Identification and Generation

首次提交 2026-04-09 00:04
版本更新 2026-11-16 23:32
雷达观测 2026-11-17 16:10
arXiv ↗PDF ↗

摘要

Large language models train on sensitive user data, so we ask what differential privacy (DP) costs in language learning. We study private language identification and generation in the agnostic statistical setting. For identification, privacy is free: at every fixed ε\varepsilon, pure DP and (ε,δ)(\varepsilon,δ)-DP with δ≥n−O(1)δ\ge n^{-O(1)} keep the non-private rate exp⁡(−r(n))\exp(-r(n)) for every r(n)=o(n)r(n)=o(n). For generation, the price depends on what is public. With a public witness bound, pure DP loses at most a factor Θ(min⁡{1,ε})Θ(\min\{1,\varepsilon\}) in the exponent, and on suitable two-language collections the optimal rate is exp⁡(−Θ(min⁡{1,ε} n))\exp(-Θ(\min\{1,\varepsilon\}\,n)). Without such a bound, pure DP can lose the exponential rate itself, already for two languages: every rate exp⁡(−o(n))\exp(-o(n)) stays achievable, but no exponential rate is. On these collections, (ε,δ)(\varepsilon,δ)-DP keeps an exponential rate if and only if log⁡(1/δ)=o(n)\log(1/δ)=o(n). Three facts about pure DP drive the proofs. It turns a robustness margin into an exponentially small error. It obeys a private form of Le Cam's two-point bound. And it cannot see the absence of a string.

作者备注:Extended version of a NeurIPS 2026 paper. 36 pages, 4 figures, 3 tables

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-16 23:32 · 最新观测版本

On the Price of Privacy for Language Identification and Generation

Large language models train on sensitive user data, so we ask what differential privacy (DP) costs in language learning. We study private language identification and generation in the agnostic statistical setting. For identification, privacy is free: at every fixed ε\varepsilon, pure DP and (ε,δ)(\varepsilon,δ)-DP with δ≥n−O(1)δ\ge n^{-O(1)} keep the non-private rate exp⁡(−r(n))\exp(-r(n)) for every r(n)=o(n)r(n)=o(n). For generation, the price depends on what is public. With a public witness bound, pure DP loses at most a factor Θ(min⁡{1,ε})Θ(\min\{1,\varepsilon\}) in the exponent, and on suitable two-language collections the optimal rate is exp⁡(−Θ(min⁡{1,ε} n))\exp(-Θ(\min\{1,\varepsilon\}\,n)). Without such a bound, pure DP can lose the exponential rate itself, already for two languages: every rate exp⁡(−o(n))\exp(-o(n)) stays achievable, but no exponential rate is. On these collections, (ε,δ)(\varepsilon,δ)-DP keeps an exponential rate if and only if log⁡(1/δ)=o(n)\log(1/δ)=o(n). Three facts about pure DP drive the proofs. It turns a robustness margin into an exponentially small error. It obeys a private form of Le Cam's two-point bound. And it cannot see the absence of a string.

查看此版本解读 →查看此版本原文 ↗