← 研踪 PaperTrailarXiv:2602.03478 · v2
When LLM Routers Overpay: The Cost of Strong-Model Preference
首次提交 2026-02-03 20:51
版本更新 2026-11-05 22:55
雷达观测 2026-11-06 13:11
摘要
LLM routing aims to reduce inference costs by assigning easy queries to less expensive models and reserving stronger models for cases in which they are needed. However, we find that existing routers often violate this principle under loose cost budgets: as the budget increases, they increasingly select the strongest and most expensive model even when less expensive feasible candidates provide comparable or identical utility. We refer to this behavior as strong-model over-selection under loose budgets. Our analysis shows that ambiguity alone does not explain this phenomenon. Substantial overpayment can arise when differently priced models offer equivalent or near-equivalent utility and the router retains a persistent model-level preference for costly models; budget relaxation exposes this preference by making premium candidates feasible. Motivated by this diagnosis, we propose EquiRouter, a decision-aligned router that learns query-dependent model rankings, preserves meaningful utility differences, and explicitly prefers lower-cost models among equivalent candidates through a Cost-Aware Ranking Objective and a lightweight Query--Model Interaction Representation. Experiments across diverse text, multimodal, continuous-utility, and large-model-pool settings show that EquiRouter consistently improves cost efficiency while maintaining competitive peak and whole-curve performance. Code is available at https://github.com/LAMDA-Model-Reuse/EquiRouter.
作者备注：Accepted to NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-05 22:55 · 最新观测版本
When LLM Routers Overpay: The Cost of Strong-Model Preference
LLM routing aims to reduce inference costs by assigning easy queries to less expensive models and reserving stronger models for cases in which they are needed. However, we find that existing routers often violate this principle under loose cost budgets: as the budget increases, they increasingly select the strongest and most expensive model even when less expensive feasible candidates provide comparable or identical utility. We refer to this behavior as strong-model over-selection under loose budgets. Our analysis shows that ambiguity alone does not explain this phenomenon. Substantial overpayment can arise when differently priced models offer equivalent or near-equivalent utility and the router retains a persistent model-level preference for costly models; budget relaxation exposes this preference by making premium candidates feasible. Motivated by this diagnosis, we propose EquiRouter, a decision-aligned router that learns query-dependent model rankings, preserves meaningful utility differences, and explicitly prefers lower-cost models among equivalent candidates through a Cost-Aware Ranking Objective and a lightweight Query--Model Interaction Representation. Experiments across diverse text, multimodal, continuous-utility, and large-model-pool settings show that EquiRouter consistently improves cost efficiency while maintaining competitive peak and whole-curve performance. Code is available at https://github.com/LAMDA-Model-Reuse/EquiRouter.
查看此版本解读 →查看此版本原文 ↗