Agentic Neural Architecture Search
Seokhoon Jeong, Mijung Kim, Taehwan Kim
原文摘要
Neural architecture search (NAS) methods have grown increasingly efficient, but they still operate within manually designed search spaces that require domain expertise and must be rebuilt for each new task. Large language models (LLMs) can write architectures in an open-ended code space, yet it is unclear how the work should be divided between LLM-driven design and NAS-driven search. We propose a way to combine the two. An LLM first designs a seed architecture and then decomposes it into a slotted architecture: a scaffold with named, interchangeable module slots that defines a bounded, task-specific search space, which a conventional NAS algorithm then explores without any manual engineering. We implement this in AgentNAS, a three-phase pipeline whose phases can be evaluated separately. On 17 tasks from NAS-Bench-360 and Unseen NAS, covering classification, dense regression, segmentation, and multi-label tagging, AgentNAS outperforms published baselines, including task-specific expert designs, on 11 tasks. Extensive ablation studies show that LLMs find strong architectures quickly but soon plateau, and that switching to NAS at this point yields further gains, indicating that the two approaches are complementary. Experiments with five LLMs show that seed architecture design is effective across all of them, whereas constructing a usable search space requires a minimum level of model capability. Our code is available at https://github.com/alroimfebruary/AgentNAS.
cs.AI