Abstract
Large language model (LLM) agents perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is insufficient; a patch must maintain behavior, implement code optimization, and approach expert speedups.
Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test code patches that may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that provides an off-the-shelf coding agent with the necessary feedback to find real hotspots, improve beyond the first passing patch, and utilize profiler evidence rather than timing alone to determine what to optimize next.
On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches compared to OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, indicating that the gains stem from better feedback rather than additional test-time sampling.
Blogger's Review: The innovation of PerfAgent lies in its profiler-guided feedback mechanism, which enables code optimization to go beyond superficial performance tests and delve into actual performance bottlenecks. This technology significantly enhances the quality and efficiency of code optimization, making it worthy of attention and practice by developers.