← 研踪 PaperTrailarXiv:2604.07962 · v2
We Should Distinguish Unlearning From Untraining
首次提交 2026-04-09 16:24
版本更新 2026-11-09 17:57
雷达观测 2026-11-10 15:22
摘要
There has been a recent surge in interest about the question of how we can ``delete'' specific data points or behaviours from a trained model, a goal referred to as ``machine unlearning''. We argue that the umbrella term ``unlearning'' actually spans two distinct problem formulations, but the distinction between them has not yet been observed in literature. This causes ambiguity around when an unlearning algorithm is expected to work, leads to the use of inappropriate metrics and baselines when comparing algorithms to one another, difficulty in interpreting results, and missed opportunities for pursuing critical research directions. In this paper, we argue for the position that a fundamental distinction must be made between two notions that we refer to as unlearning and untraining. On one hand, untraining aims to reverse the effect of having trained on a given ``forget set'', i.e. to remove the influence that that specific set of examples had on the model during training. On the other hand, unlearning aims not just to remove the influence of those given examples, but also of the entire underlying subdistribution from which they were sampled (and thus e.g. the concept or model behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each notion. By disambiguating technical definitions, our work aims to accelerate progress in this important field.
作者备注：NeurIPS 2026 Position Paper Track (oral)
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-09 17:57 · 最新观测版本
We Should Distinguish Unlearning From Untraining
There has been a recent surge in interest about the question of how we can ``delete'' specific data points or behaviours from a trained model, a goal referred to as ``machine unlearning''. We argue that the umbrella term ``unlearning'' actually spans two distinct problem formulations, but the distinction between them has not yet been observed in literature. This causes ambiguity around when an unlearning algorithm is expected to work, leads to the use of inappropriate metrics and baselines when comparing algorithms to one another, difficulty in interpreting results, and missed opportunities for pursuing critical research directions. In this paper, we argue for the position that a fundamental distinction must be made between two notions that we refer to as unlearning and untraining. On one hand, untraining aims to reverse the effect of having trained on a given ``forget set'', i.e. to remove the influence that that specific set of examples had on the model during training. On the other hand, unlearning aims not just to remove the influence of those given examples, but also of the entire underlying subdistribution from which they were sampled (and thus e.g. the concept or model behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each notion. By disambiguating technical definitions, our work aims to accelerate progress in this important field.
查看此版本解读 →查看此版本原文 ↗