← 研踪 PaperTrailarXiv:2601.05903 · v2
HAPS: Hierarchical LLM Routing with Joint Architecture and Parameter Search
首次提交 2026-01-10 00:22
版本更新 2026-11-07 19:46
雷达观测 2026-11-10 11:14
摘要
Large language model (LLM) routing aims to exploit the specialized strengths of different LLMs for diverse tasks. However, existing approaches typically focus only on selecting LLM architectures, treating candidate models as fixed while overlooking parameter adaptation, which can substantially affect task performance. In this paper, we introduce HAPS, a hierarchical LLM routing framework that jointly searches over model architectures and parameters. Specifically, a high-level router selects among candidate LLM architectures, while a low-level router generates input-conditioned LoRA parameters for the selected architecture, expanding each candidate from a single fixed behavior into a region of reachable behaviors. The low-level router is built on the backbone of the high-level router, so that architecture selection and parameter generation share a common representation of the prompt. We further optimize the whole framework with a reinforcement learning objective. Extensive experiments show that HAPS consistently outperforms strong routing baselines across reasoning- and knowledge-intensive benchmarks, with analyses of its components, design choices, scalability, cost, and mixed open- and closed-source deployment further demonstrating the robustness of these gains. Our code is available at https://github.com/zihangtian/HAPS.
作者备注：Accepted at the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-07 19:46 · 最新观测版本
HAPS: Hierarchical LLM Routing with Joint Architecture and Parameter Search
Large language model (LLM) routing aims to exploit the specialized strengths of different LLMs for diverse tasks. However, existing approaches typically focus only on selecting LLM architectures, treating candidate models as fixed while overlooking parameter adaptation, which can substantially affect task performance. In this paper, we introduce HAPS, a hierarchical LLM routing framework that jointly searches over model architectures and parameters. Specifically, a high-level router selects among candidate LLM architectures, while a low-level router generates input-conditioned LoRA parameters for the selected architecture, expanding each candidate from a single fixed behavior into a region of reachable behaviors. The low-level router is built on the backbone of the high-level router, so that architecture selection and parameter generation share a common representation of the prompt. We further optimize the whole framework with a reinforcement learning objective. Extensive experiments show that HAPS consistently outperforms strong routing baselines across reasoning- and knowledge-intensive benchmarks, with analyses of its components, design choices, scalability, cost, and mixed open- and closed-source deployment further demonstrating the robustness of these gains. Our code is available at https://github.com/zihangtian/HAPS.
查看此版本解读 →查看此版本原文 ↗