跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.23229 · v1

Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching

环境LLM:通过温度匹配实现数据高效监督微调

首次提交 2026-11-18 02:59
版本更新 2026-11-18 02:59
雷达观测 2026-11-18 12:34
arXiv ↗PDF ↗

摘要

Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit T→∞T\to \infty reverting back to regular SFT and T=1T=1 to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to 21%21\% average relative improvement to standard SFT.

作者备注:Work in progress

AI 论文解读

基于摘要 · 非全文审阅

提出Ambient SFT框架,利用预训练模型作为噪声教师,通过匹配高温条件分布来增强微调数据,缓解小数据SFT中的过拟合和灾难性遗忘。

正在读取账号与研究记录…

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-18 02:59 · 最新观测版本

Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching

Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit T→∞T\to \infty reverting back to regular SFT and T=1T=1 to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to 21%21\% average relative improvement to standard SFT.

查看此版本解读 →查看此版本原文 ↗