SnapAudit: Active Auditing of Differentially Private In-Context Learning via Snapshot-Based Simulation
Yuyang Xia, Ruixuan Liu, Li Xiong
原文摘要
In-context learning (ICL) enables LLMs to adapt to tasks using demonstrations that may contain sensitive data. Differentially private (DP) ICL mechanisms mitigate this risk by adding noise during aggregation. However, verifying their claimed privacy bounds through repeated end-to-end membership-inference attacks is costly and yields unstable estimates. We propose SnapAudit, an active auditing framework that decomposes a DP-ICL pipeline into a deterministic clean-inference stage and a stochastic DP-noise stage, and audits the full pipeline by combining a small snapshot of the former with bootstrap simulation of the latter. Because clean LLM outputs are near-deterministic at low temperature, tens of thousands of clean LLM calls suffice to approximate the snapshot distribution; SnapAudit then bootstraps hundreds of thousands of noisy trials at negligible additional cost, controlling finite-sample uncertainty via an empirical Bernstein correction. For embedding-based mechanisms, a multi-sweep search constructs maximally separable audit signals. Compared with passive auditing, SnapAudit yields tighter, more stable privacy estimates that closely match theoretical guarantees. Across privacy budgets, SnapAudit reuses snapshots across noise configurations, whereas passive auditing requires fresh end-to-end trials. Across four DP-ICL pipelines and five privacy budgets, this reduces total wall-clock auditing time by $4.3$--$5.5\times$. SnapAudit also uncovers two flaws in existing DP-ICL designs: (i) classical Gaussian noise calibrations underestimate leakage at large privacy budgets, allowing empirical leakage to exceed the theoretical bound; (ii) an embedding-aggregation mechanism's sensitivity analysis is incorrect when the number of partitions equals one, leading to undersized noise and a privacy violation.
cs.CR