AI coding agents are considered black boxes: we cannot inspect how they generate code, but we can observe what changes they make. This distinction is crucial for search-based software engineering (SBSE), where techniques like genetic improvement rely on mutation operators that reflect how code is actually transformed. Fewer than 1% of the 33,596 agent PRs in AIDev-pop target performance, making each case a rare glimpse into opaque agent behavior. We classify 1,254 performance-relevant diff hunks from 216 of these PRs, spanning five agent systems, against the 18-category syntactic mutation taxonomy of Even-Mendoza et al. (2025) using a dual-LLM intersection pipeline. The analysis reveals that three categories dominate: name modification (37.0%), object creation (26.4%), and type change (22.7%), which is markedly different from prior GI corpora where no change accounted for 84%. Each agent's deployed system commits to a distinctive mutation vocabulary, and each performance strategy activates a largely disjoint category subset. Thus, agent identity and target strategy serve as informative priors that narrow the effective SBSE operator space. Replication package available at: GitHub.
Blogger's Review: This paper reveals specific mutation patterns of AI agents in code optimization, highlighting both the potential and challenges they pose in software engineering. By meticulously categorizing performance-related pull requests, the researchers provide an empirical foundation for understanding AI agent behavior, which merits further exploration and experimentation in practical applications.