← 研踪 PaperTrailarXiv:2605.05871 · v2
Retain-Neutral Surrogates for Min-Max Unlearning
首次提交 2026-05-07 16:38
版本更新 2026-11-12 02:41
雷达观测 2026-11-13 14:16
摘要
Machine unlearning aims to make a model forget designated training data while retaining performance on the rest. In min-max unlearning, an inner adversary perturbs the model to maximize the forget loss, and the model is updated to minimize the retain loss at the resulting surrogate point. When the forget and retain gradients are positively coupled, the standard perturbation also moves toward higher retain loss, and the retain step begins by recovering that loss. We propose Retain-Orthogonal Surrogate Unlearning (ROSU), which redefines the adversary to admit only perturbations orthogonal to the retain gradient. The one that raises the forget loss fastest has a closed form, and ROSU uses it to set the surrogate point and amplifies it into the forgetting step. Under positive coupling the ROSU surrogate avoids the retain cost that coupling induces in the standard surrogate, and the difference shrinks as the gradients approach orthogonality. ROSU has the smallest accuracy gap to retraining in every vision setting (CIFAR-10, CIFAR-100, Tiny-ImageNet) and is competitive with the baselines on language benchmarks (TOFU, WMDP). The redefinition helps most under high positive coupling. Code is available at https://github.com/unlearn-ml/machine-unlearning.
作者备注：Accepted at NeurIPS 2026 as a Conference Paper
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-12 02:41 · 最新观测版本
Retain-Neutral Surrogates for Min-Max Unlearning
Machine unlearning aims to make a model forget designated training data while retaining performance on the rest. In min-max unlearning, an inner adversary perturbs the model to maximize the forget loss, and the model is updated to minimize the retain loss at the resulting surrogate point. When the forget and retain gradients are positively coupled, the standard perturbation also moves toward higher retain loss, and the retain step begins by recovering that loss. We propose Retain-Orthogonal Surrogate Unlearning (ROSU), which redefines the adversary to admit only perturbations orthogonal to the retain gradient. The one that raises the forget loss fastest has a closed form, and ROSU uses it to set the surrogate point and amplifies it into the forgetting step. Under positive coupling the ROSU surrogate avoids the retain cost that coupling induces in the standard surrogate, and the difference shrinks as the gradients approach orthogonality. ROSU has the smallest accuracy gap to retraining in every vision setting (CIFAR-10, CIFAR-100, Tiny-ImageNet) and is competitive with the baselines on language benchmarks (TOFU, WMDP). The redefinition helps most under high positive coupling. Code is available at https://github.com/unlearn-ml/machine-unlearning.
查看此版本解读 →查看此版本原文 ↗