When Is Smaller Greener? Right-Sizing Language Models for Agentic AI
Sajid Siraj, Maria Barbati
AI agents use a large language model (LLM) to plan and act over many steps. Which model should an agent use, so that its tasks are completed reliably with the least energy, cost and time? We treat this as a multi-criteria decision problem in which comparisons per call can mislead: a task succeeds only if all its steps succeed, and failed attempts still use resources. We therefore compare models on resources per successfully completed task. We derive crossover task lengths at which two models exchange their efficiency ranking, determined by their per-step reliability and resource use. Each objective has its own crossover. Below the shortest crossover the less reliable model dominates, above the longest the more reliable one dominates, and in between both are Pareto-efficient and preferences decide. This supports a two-stage framework: a preference-free dominance screen, followed by a multi-criteria ranking of the efficient models, subject to reliability and feasibility constraints. We also give conditions under which this structure holds when reliability and resource use change during a task. In a pilot with 15 open-weight models on two synthetic agent tasks, with measured GPU energy, the predicted crossover fell within the observed range for 72 of the 88 observed crossovers. At 20 steps, a 3-billion-parameter model used a tenth of the energy of a 27-billion one per completed task; at 100 steps, a 1.7-billion model used almost four times as much as an 8-billion one. Smaller is greener only up to a task length that can be predicted.
尚无 AI 解读，展示原文摘要