EVOTOOL: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
Shuo Yang, Caren Han University of Melbourne, University of Western Australia and University of Sydney, Xueqi Ma, Yan Li, Mohammad Reza Ghasemi Madani University of Melbourne, Eduard Hovy
原文摘要
LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging dueto delayed supervision and the difficulty ofcredit assignment in long-horizon trajectories.Existing optimization approaches tend to beeither monolithic, which are prone to entangling behaviors, or single-aspect, which ignorecross-module error propagation. To addressthese limitations, we propose EVOTOOL, a self-evolving framework that optimizes a modulartool-use policy via a gradient-free evolutionary paradigm. EVOTOOL decomposes agent’stool-use policy into four modules, includingPlanner, Selector, Caller, and Synthesizer, anditeratively improves them through three mechanisms. Trajectory-Grounded Blame Attribution uses diagnostic traces to localize failuresto a specific module. Feedback-Guided Targeted Mutation then edits only that modulevia natural-language critique. Diversity-AwarePopulation Selection preserves complementarycandidates to ensure solution diversity. Acrossfour diverse benchmarks, EVOTOOL outperforms strong baselines by over 5 points on bothGPT-4.1 and Qwen3-8B, while achieving superior efficiency and transferability.
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表