← 研踪 PaperTrailarXiv:2605.24432 · v2
Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap
首次提交 2026-05-23 14:58
版本更新 2026-11-13 02:59
雷达观测 2026-11-13 14:15
摘要
Large Language Model (LLM) interactions are typically underspecified, with users clarifying all necessary details across multiple conversational turns. Yet recent work shows that LLMs perform far worse in this multi-turn setting than in a single turn with the same information available at once, a phenomenon termed "Lost-in-Conversation." However, bridging this gap effectively remains an open problem. Here we introduce Found in Conversation (FiC), a training framework where a model teaches itself to find and recover its single-turn competence given underspecified multi-turn prompts. We develop View-Asymmetric Self-Distillation, which distills across two views of the same task information (a single-turn view for the teacher and a multi-turn view for the student), transferring strong single-turn behavior into weak multi-turn behavior. This requires no stronger external teacher, which is unavailable as even frontier LLMs exhibit this gap. Across model families (Llama, Qwen, Phi, and OLMo) and sizes (3B-14B), FiC attains 92-100% of single-turn performance, yielding more efficient and helpful multi-turn conversations with single-turn capabilities largely intact.
作者备注：20 pages, 3 figures, 13 tables. NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-13 02:59 · 最新观测版本
Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap
Large Language Model (LLM) interactions are typically underspecified, with users clarifying all necessary details across multiple conversational turns. Yet recent work shows that LLMs perform far worse in this multi-turn setting than in a single turn with the same information available at once, a phenomenon termed "Lost-in-Conversation." However, bridging this gap effectively remains an open problem. Here we introduce Found in Conversation (FiC), a training framework where a model teaches itself to find and recover its single-turn competence given underspecified multi-turn prompts. We develop View-Asymmetric Self-Distillation, which distills across two views of the same task information (a single-turn view for the teacher and a multi-turn view for the student), transferring strong single-turn behavior into weak multi-turn behavior. This requires no stronger external teacher, which is unavailable as even frontier LLMs exhibit this gap. Across model families (Llama, Qwen, Phi, and OLMo) and sizes (3B-14B), FiC attains 92-100% of single-turn performance, yielding more efficient and helpful multi-turn conversations with single-turn capabilities largely intact.
查看此版本解读 →查看此版本原文 ↗