Designer-RSI: Evolving Procedural Memory for Graphic Design Agent
Hongyang Du, Lan Yan, Christian Flores, Asim Kadav
原文摘要
Deployed software agents face a continuous stream of user requests while their foundation-model weights remain frozen between releases. Recursive self-improvement (RSI) via external memory offers a practical path to continual adaptation, yet prior methods largely assume programmatic verifiers, cheap retries, or a single global prompt. Creative software breaks these assumptions, as open-ended visual briefs lack ground truth, VLM judges are noisy, and multi-step tool executions are costly and confounded by external assets. We present Designer-RSI, which evolves a modular SKILL.md skill bank from unlabeled design trajectories for an agent spanning over 230 raster, vector, and layout tools. Designer-RSI couples widening (acquiring new skills for recurring capability gaps) with deepening (revising failing skills by contrasting successful and failed runs), and screens updates through a Pareto-style matched replay gate that requires strict gains on sampled briefs without local regressions. Across held-out product briefs, eight external benchmarks, and two backbones, traffic-driven evolution outperforms static documentation skills, lifts execution success up to 99.8%, improves requirement fulfillment, and reaches up to a 67.6% blind win rate over the base agent. Ablations further show that skill acquisition and revision are strongly synergistic: neither suffices alone, and reliable gains require evolving coverage and execution quality together.
cs.AI · cs.CV