← 研踪 PaperTrailarXiv:2604.07238 · v2
On the Price of Privacy for Language Identification and Generation
首次提交 2026-04-09 00:04
版本更新 2026-11-16 23:32
雷达观测 2026-11-17 16:10
摘要
Large language models train on sensitive user data, so we ask what differential privacy (DP) costs in language learning. We study private language identification and generation in the agnostic statistical setting. For identification, privacy is free: at every fixed , pure DP and -DP with keep the non-private rate for every . For generation, the price depends on what is public. With a public witness bound, pure DP loses at most a factor in the exponent, and on suitable two-language collections the optimal rate is . Without such a bound, pure DP can lose the exponential rate itself, already for two languages: every rate stays achievable, but no exponential rate is. On these collections, -DP keeps an exponential rate if and only if . Three facts about pure DP drive the proofs. It turns a robustness margin into an exponentially small error. It obeys a private form of Le Cam's two-point bound. And it cannot see the absence of a string.
作者备注:Extended version of a NeurIPS 2026 paper. 36 pages, 4 figures, 3 tables
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 23:32 · 最新观测版本
On the Price of Privacy for Language Identification and Generation
Large language models train on sensitive user data, so we ask what differential privacy (DP) costs in language learning. We study private language identification and generation in the agnostic statistical setting. For identification, privacy is free: at every fixed , pure DP and -DP with keep the non-private rate for every . For generation, the price depends on what is public. With a public witness bound, pure DP loses at most a factor in the exponent, and on suitable two-language collections the optimal rate is . Without such a bound, pure DP can lose the exponential rate itself, already for two languages: every rate stays achievable, but no exponential rate is. On these collections, -DP keeps an exponential rate if and only if . Three facts about pure DP drive the proofs. It turns a robustness margin into an exponentially small error. It obeys a private form of Le Cam's two-point bound. And it cannot see the absence of a string.
查看此版本解读 →查看此版本原文 ↗