← 研踪 PaperTrailarXiv:2606.08810 · v3
What Simple Readouts Miss in Continuous Language Diffusion
首次提交 2026-06-08 04:00
版本更新 2026-11-09 16:08
雷达观测 2026-11-10 13:18
摘要
How much of a learned decoder can a simple readout preserve? On generated ELF and AURORA states, token-wise affine maps recover $97.17\%$ and $95.74\%$ of native lexical decisions. Contextual computation has already produced these states; only the terminal readout is simplified. The remaining disagreements matter beyond their count: equal-count rare-first and low-native-margin-first repairs have opposing pooled evaluator effects in both interfaces. On conditional summarization, rare-first improves reference similarity, while source consistency remains unresolved. Score accounting locates part of the benefit on unchanged text despite increased unigram cost, without identifying rarity as a cause. Repeatable repair effects do not ensure prediction transfer: a frozen predictor improves on two ELF checkpoints but fails on a distilled student. A separate task-fitted selector improves BERTScore over random selection by $0.261$ points on 256 fresh articles at a $50\%$ native-call budget; all-native decoding remains higher-scoring and acceleration is unestablished. Compressed and directional interface tests further distinguish fitting-state coverage, native decisions, and soft inputs. Together, the results motivate testing a simplification at three levels: preserved decisions, output consequences, and computational savings. Native decisions are fidelity targets, not semantic truth.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-09 16:08 · 最新观测版本
What Simple Readouts Miss in Continuous Language Diffusion
How much of a learned decoder can a simple readout preserve? On generated ELF and AURORA states, token-wise affine maps recover $97.17\%$ and $95.74\%$ of native lexical decisions. Contextual computation has already produced these states; only the terminal readout is simplified. The remaining disagreements matter beyond their count: equal-count rare-first and low-native-margin-first repairs have opposing pooled evaluator effects in both interfaces. On conditional summarization, rare-first improves reference similarity, while source consistency remains unresolved. Score accounting locates part of the benefit on unchanged text despite increased unigram cost, without identifying rarity as a cause. Repeatable repair effects do not ensure prediction transfer: a frozen predictor improves on two ELF checkpoints but fails on a distilled student. A separate task-fitted selector improves BERTScore over random selection by $0.261$ points on 256 fresh articles at a $50\%$ native-call budget; all-native decoding remains higher-scoring and acceleration is unestablished. Compressed and directional interface tests further distinguish fitting-state coverage, native decisions, and soft inputs. Together, the results motivate testing a simplification at three levels: preserved decisions, output consequences, and computational savings. Native decisions are fidelity targets, not semantic truth.
查看此版本解读 →查看此版本原文 ↗