← 研踪 PaperTrailarXiv:2511.13502 · v4
SnapAudit: Active Auditing of Differentially Private In-Context Learning via Snapshot-Based Simulation
首次提交 2025-11-17 23:39
版本更新 2026-11-10 10:38
雷达观测 2026-11-11 11:04
摘要
In-context learning (ICL) enables LLMs to adapt to tasks using demonstrations that may contain sensitive data. Differentially private (DP) ICL mechanisms mitigate this risk by adding noise during aggregation. However, verifying their claimed privacy bounds through repeated end-to-end membership-inference attacks is costly and yields unstable estimates. We propose SnapAudit, an active auditing framework that decomposes a DP-ICL pipeline into a deterministic clean-inference stage and a stochastic DP-noise stage, and audits the full pipeline by combining a small snapshot of the former with bootstrap simulation of the latter. Because clean LLM outputs are near-deterministic at low temperature, tens of thousands of clean LLM calls suffice to approximate the snapshot distribution; SnapAudit then bootstraps hundreds of thousands of noisy trials at negligible additional cost, controlling finite-sample uncertainty via an empirical Bernstein correction. For embedding-based mechanisms, a multi-sweep search constructs maximally separable audit signals. Compared with passive auditing, SnapAudit yields tighter, more stable privacy estimates that closely match theoretical guarantees. Across privacy budgets, SnapAudit reuses snapshots across noise configurations, whereas passive auditing requires fresh end-to-end trials. Across four DP-ICL pipelines and five privacy budgets, this reduces total wall-clock auditing time by $4.3$--$5.5\times$. SnapAudit also uncovers two flaws in existing DP-ICL designs: (i) classical Gaussian noise calibrations underestimate leakage at large privacy budgets, allowing empirical leakage to exceed the theoretical bound; (ii) an embedding-aggregation mechanism's sensitivity analysis is incorrect when the number of partitions equals one, leading to undersized noise and a privacy violation.
作者备注：Accepted at NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v4 · 2026-11-10 10:38 · 最新观测版本
SnapAudit: Active Auditing of Differentially Private In-Context Learning via Snapshot-Based Simulation
In-context learning (ICL) enables LLMs to adapt to tasks using demonstrations that may contain sensitive data. Differentially private (DP) ICL mechanisms mitigate this risk by adding noise during aggregation. However, verifying their claimed privacy bounds through repeated end-to-end membership-inference attacks is costly and yields unstable estimates. We propose SnapAudit, an active auditing framework that decomposes a DP-ICL pipeline into a deterministic clean-inference stage and a stochastic DP-noise stage, and audits the full pipeline by combining a small snapshot of the former with bootstrap simulation of the latter. Because clean LLM outputs are near-deterministic at low temperature, tens of thousands of clean LLM calls suffice to approximate the snapshot distribution; SnapAudit then bootstraps hundreds of thousands of noisy trials at negligible additional cost, controlling finite-sample uncertainty via an empirical Bernstein correction. For embedding-based mechanisms, a multi-sweep search constructs maximally separable audit signals. Compared with passive auditing, SnapAudit yields tighter, more stable privacy estimates that closely match theoretical guarantees. Across privacy budgets, SnapAudit reuses snapshots across noise configurations, whereas passive auditing requires fresh end-to-end trials. Across four DP-ICL pipelines and five privacy budgets, this reduces total wall-clock auditing time by $4.3$--$5.5\times$. SnapAudit also uncovers two flaws in existing DP-ICL designs: (i) classical Gaussian noise calibrations underestimate leakage at large privacy budgets, allowing empirical leakage to exceed the theoretical bound; (ii) an embedding-aggregation mechanism's sensitivity analysis is incorrect when the number of partitions equals one, leading to undersized noise and a privacy violation.
查看此版本解读 →查看此版本原文 ↗