Review: Automated Design of Agentic Systems

This article reviews the ICLR 2025 paper 'Automated Design of Agentic Systems' by Hu, Lu, and Clune. The paper explores using a meta-agent (a 100-line Python program instructing GPT-4) to invent new agent architectures, rather than manually designing systems like Chain-of-Thought or LLM Debate. The meta-agent generates short Python programs that call GPT-3.5, evaluates them on benchmarks, and archives successful designs. Discovered agents beat state-of-the-art hand-designed systems on DROP, MGSM, and ARC benchmarks, and transfer well across domains. The review discusses the paper's claim that searching in code space is more expressive than graph-based approaches, and notes the high cost ($300-$500 per run) and choice of benchmarks.