CPU-ARC:面向CPU推理复用的智能体推理缓存
CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse
Peter Belcak, Greg Heinrich, Saurav Muralidharan, Shizhe Diao, Pavlo Molchanov
AI 速读 基于摘要
作者提出智能体推理缓存(ARCs),把大模型的推理轨迹转化为可在CPU上执行的缓存,使已推理过的问题解法能高效复用于新实例,从而减少对大模型的重复调用。
cs.AI
查看原文摘要
Large language models reinvent the wheel all the time. They conflate task-solving with the execution of the solutions, which forces them to solve every problem from scratch. We show that once a task has been reasoned through, the application of its solution to a new problem instance can be done very efficiently and with little involvement from language models. We introduce agentic reasoning caching, a method for constructing robust CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces. ARCs show on-par performance with frontier LLMs across a range of difficult mathematical and scientific tasks, are fully explainable, continually trainable, robust to input perturbations, and 16-291x cheaper to invoke than the source model.