This paper examines whether AI agents truly understand computer architecture when tasked with hardware design. Existing benchmarks compare agents under a fixed problem framing, which cannot tell if an agent is reasoning about the machine or merely searching knobs without grasping their meaning. We introduce the AutoTuring experiment: the same agent optimizes the identical 15‑dimensional accelerator space twice—once with named architectural knobs and simulator counters, and once with anonymous variables in [0,1]. The legal space and reachable optima are kept identical, so only the semantic meaning differs. On a nine‑kernel FP16 GEMM workload, meaningful architectural information yields a clear advantage: the architect agent outperforms a modeled H200 by 5.4% and its blind counterpart by 12.3% on average, while using 70.1% fewer simulator calls. However, when a critic loop is added, most of the gap is recovered for the blind agent, indicating that structured critique can substitute for explicit architectural knowledge rather than complement it. We present these as preliminary findings—five to six runs per condition on a single modeled accelerator—emphasizing the comparative methodology over the accelerator itself.
Review