← 研踪 PaperTrailACL Anthology · ACL 2026:acl:2026.acl-long.1964 · 记录 #1
LLMSurgeon: Diagnosing Data Mixture of Large Language Models
来源日期（仅到月份） 2026-07
来源记录日期 2026-07
雷达观测 2026-11-01 21:48
摘要
The pretraining data mixture of Large Language Models (LLMs) constitutes their “digital DNA”, shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize \textbf{Data Mixture Surgery (DMS)}: given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose \textbf{LLMSurgeon}, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, LLMSurgeon estimates a calibrated \textit{soft} confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce \textbf{LLMScan}, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, LLMSurgeon recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the \textit{digital DNA} of foundation models without access to their training data. Code is available at: https://github.com/Yaxin9Luo/LLMSurgeon.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026-07 · 最新观测版本
LLMSurgeon: Diagnosing Data Mixture of Large Language Models
The pretraining data mixture of Large Language Models (LLMs) constitutes their “digital DNA”, shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize \textbf{Data Mixture Surgery (DMS)}: given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose \textbf{LLMSurgeon}, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, LLMSurgeon estimates a calibrated \textit{soft} confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce \textbf{LLMScan}, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, LLMSurgeon recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the \textit{digital DNA} of foundation models without access to their training data. Code is available at: https://github.com/Yaxin9Luo/LLMSurgeon.
查看此版本解读 →查看此版本原文 ↗