ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling
Jianghao Lin Shanghai Jiaotong University, Yuanyuan Shi Shanghai Jiaotong University, Xin Peng, Renjie Ding LongShine, Hairui Wang, Yuxuan Peng longshine, Bizhe Bai, Weixi Song Westlake University, Zhejiang University and Shanghai Innovation Institute 等 13 位作者
Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with ToolPRM, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows “explore more but retain less”, since early JSON errors are unrecoverable.
尚无 AI 解读，展示原文摘要
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表