← 研踪 PaperTrailarXiv:2603.02792 · v3
From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors
首次提交 2026-03-03 17:27
版本更新 2026-11-04 20:55
雷达观测 2026-11-05 12:13
摘要
Large Language Models (LLMs) have already been widely adopted for automated algorithm design, demonstrating strong abilities in generating and evolving algorithms across various fields. Existing work has largely focused on examining their effectiveness in solving specific problems, with search strategies primarily guided by adaptive prompt designs. In this paper, through investigating the token-wise attribution of the prompts to LLM-generated algorithmic code, we show that the embedded code examples and their associated strategy instructions have the strongest influence on the generated algorithms. Building upon this insight, we propose leveraging existing benchmark algorithms as strong priors to guide LLM-driven optimization. We evaluate this approach with three LLMs on two black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization suite (bbob). Despite relying on a simple (1+1) elitist search, it obtains the best average rank in five of six benchmark--LLM settings against six state-of-the-art LLM-driven methods, including more sophisticated search strategies. Our findings highlight the value of integrating benchmarking studies to enhance both efficiency and robustness of the LLM-driven black-box optimization methods.
作者备注：Paper under review
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-04 20:55 · 最新观测版本
From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors
Large Language Models (LLMs) have already been widely adopted for automated algorithm design, demonstrating strong abilities in generating and evolving algorithms across various fields. Existing work has largely focused on examining their effectiveness in solving specific problems, with search strategies primarily guided by adaptive prompt designs. In this paper, through investigating the token-wise attribution of the prompts to LLM-generated algorithmic code, we show that the embedded code examples and their associated strategy instructions have the strongest influence on the generated algorithms. Building upon this insight, we propose leveraging existing benchmark algorithms as strong priors to guide LLM-driven optimization. We evaluate this approach with three LLMs on two black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization suite (bbob). Despite relying on a simple (1+1) elitist search, it obtains the best average rank in five of six benchmark--LLM settings against six state-of-the-art LLM-driven methods, including more sophisticated search strategies. Our findings highlight the value of integrating benchmarking studies to enhance both efficiency and robustness of the LLM-driven black-box optimization methods.
查看此版本解读 →查看此版本原文 ↗