Abstract
LLM-driven agent systems have shown strong potential in electronic design automation (EDA), capable of automating complex design workflows. However, existing evaluations mainly focus on isolated EDA tasks using individual language models, providing limited insight into the performance of different agent systems across complete EDA flows. This work presents FluxBench, a systematic evaluation tool for AI agents on end-to-end EDA workflows under unified prompts, tool environments, and technology library settings.
Our evaluation covers representative scenarios, including RTL generation with open-source toolchains and the RTL-to-GDS flow using closed-source commercial EDA tools for industrial applications. Through these workflows, we assess agents' capabilities in RTL code generation, iterative repair, tool-feedback utilization, logic synthesis, placement and routing (P&R), and Engineering Change Order (ECO) automation.
To further characterize the efficiency of agent systems, we introduce Token ROI, a cost-efficiency metric that measures effective improvements in EDA artifacts relative to token usage and runtime cost. Experimental results show that even when built on the same foundation model, different agent system architectures can exhibit performance gaps of up to 86.27%. Moreover, among systems with comparable task performance, Token ROI can differ by as much as $105.92\times$.
In the RTL-to-GDS flow using PicoRV32 as a case study, FluxEDA achieves an end-to-end score of up to 97.94, outperforming Claude Code equipped with domain-specific EDA skills by up to $8.39\times$. These results indicate that domain-specific skills alone are insufficient to enhance agent performance in large-scale EDA scenarios. Instead, both agent system design and foundation model capability play critical roles in enabling effective automated EDA workflows.
Blogger's Review: This paper highlights the potential and challenges of AI agents in EDA workflows through the FluxBench tool, emphasizing the importance of system design and foundation models. Past evaluations focused on single models are inadequate for complex design requirements, making a comprehensive analysis of different system architectures critical for future research.