← 研踪 PaperTrailarXiv:2609.33270 · v2
Structured Sparse Memory for Recurrent Reasoning
首次提交 2026-09-27 14:25
版本更新 2026-11-06 12:27
雷达观测 2026-11-09 10:16
摘要
Recurrent models trained from scratch have recently become competitive on ARC-style reasoning tasks, but the usual framing around small recurrent backbones overlooks two important parts of the system: task-conditioned memory and synthetic augmentation data. We study this regime through CHARM, a compact hybrid ARC model that combines recurrent reasoning with structured task memory, synthetic data, and inference-time aggregation. In existing approaches, task-conditioned memory supplies a large hidden source of capacity, reaching more than 30x the size of the recurrent backbone. We introduce a compositional sparse embedding (CoSE) for task conditioning that reduces learned task-memory parameters by over 90% while improving pass@2 in controlled ARC ablations. We also introduce Re-ARC2, a set of programs that generate and verify new examples for all ARC-AGI-2 training tasks. Combining these ingredients, our system reaches 84% pass@2 on ARC-AGI-1 and 46.7% pass@2 on ARC-AGI-2 public evaluation. Structured memory also enables adaptation with only 512 parameters per unseen puzzle, and its benefits generalize to other domains. Our code, dataset, and model checkpoints are available at https://github.com/water-vapor/charm.
作者备注：Accepted to NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-06 12:27 · 最新观测版本
Structured Sparse Memory for Recurrent Reasoning
Recurrent models trained from scratch have recently become competitive on ARC-style reasoning tasks, but the usual framing around small recurrent backbones overlooks two important parts of the system: task-conditioned memory and synthetic augmentation data. We study this regime through CHARM, a compact hybrid ARC model that combines recurrent reasoning with structured task memory, synthetic data, and inference-time aggregation. In existing approaches, task-conditioned memory supplies a large hidden source of capacity, reaching more than 30x the size of the recurrent backbone. We introduce a compositional sparse embedding (CoSE) for task conditioning that reduces learned task-memory parameters by over 90% while improving pass@2 in controlled ARC ablations. We also introduce Re-ARC2, a set of programs that generate and verify new examples for all ARC-AGI-2 training tasks. Combining these ingredients, our system reaches 84% pass@2 on ARC-AGI-1 and 46.7% pass@2 on ARC-AGI-2 public evaluation. Structured memory also enables adaptation with only 512 parameters per unseen puzzle, and its benefits generalize to other domains. Our code, dataset, and model checkpoints are available at https://github.com/water-vapor/charm.
查看此版本解读 →查看此版本原文 ↗