arXiv:2608.03172 · v6
Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study
摘要
Structure-preserving de-identification replaces protected health information (PHI) with realistic same-type surrogates ("Anna S." becomes "Maria S.", not [NAME]). De-identified notes are what allow clinical text to be shared for model development, and many pipelines run a second PHI detector over them as a safety net. We ask a narrow, testable question: on the spans a de-identifier actually replaces, can downstream PHI detectors still find the surrogate as well as they found the original value? We call this detectability preservation and treat it as a proxy, not as general utility. We propose a paired protocol that (i) scores the same spans before and after replacement with an identical panel of 11 detectors from three families, (ii) measures the substitution effect only on replaced spans, keeping it separate from coverage, and (iii) bounds the effect with document-level cluster-bootstrap equivalence testing and posterior probabilities instead of significance tests. Applied to a commercial transform on seven benchmarks in seven languages (1,750 documents), pooled recall on replaced spans moves from 75.9% to 74.7% ( points, 90% CI ), within a -point margin under overlap matching. The effect is not uniform: it exceeds points for several detectors, for two identifier-dense non-English benchmarks, and under exact-boundary matching. Two-rater annotation of surrogates (), a same-slot counterfactual, and detector test-retest show that the residual has both generator-side and detector-side causes. On ICD-10 diagnosis classification of the same clinical documents, accuracy changes by at most 1 point under substitution, Faker surrogates, or redaction.
作者备注:Accepted to the Machine Learning for Health (ML4H) Symposium 2026, Proceedings track (PMLR 346), held in Sydney alongside NeurIPS 2026. 14 pages, 3 figures, 12 tables. Code, data and per-span predictions: https://github.com/Custodian-Labs/guardian-layer-phi-benchmark ; dashboard: https://custodianlabs.pages.dev