arXiv:2609.08789 · v2
Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers
摘要
Frontier AI developers publish safety frameworks that commit them to evidencing whether their models are dangerous. The European Union and California now treat these documents as instruments of accountability and impose duties on their revision. Neither requires the developer's account to identify each change to a commitment. We introduce the silent revision rate, the share of material changes to a framework's commitments that the developer's published account does not identify, and we release the versioned, hash-pinned corpus needed to compute it. The corpus holds every public version of the safety frameworks of the twelve developers that have published one, with each provider's changelog, redline or announcement. We trace 710 commitment instances across twelve consecutive version pairs, code them against a frozen codebook, and check the coding with three independent re-runs, a blind re-coding of every disputed row and a second coding of a stratified sample. First, 64% of material changes (95% CI 59 to 69) are silent under a strict standard and 30% under a lenient one. Second, the form of the account matters less than its granularity. Narrative and itemised accounts differ by eight points that eight pairs cannot separate from chance, whereas the rate falls with the number of discrete items a provider publishes. Third, 75% of traced changes weaken or remove a commitment, and weakenings are silent more often than strengthenings, at 70% against 52%. A justification explains why a framework changed and an enumeration states what changed, and only the latter makes revision auditable. We argue that publication duties should carry an enumeration duty, which one provider already meets, voluntarily and incompletely.
作者备注：31 pages, 3 figures, 10 tables. v2: coding checked by three re-runs, a blind re-coding and a second coding; strict rate revised from 67% to 64% and lenient from 53% to 30%; the v1 narrative-against-itemised difference is no longer distinguishable from chance. Data and code: https://github.com/louisyzhu/frontier-safety-framework-corpus (DOI 10.5281/zenodo.22670700)