Simulation success does not guarantee structural correctness. We define and measure four evaluation levels—schema validity, topological validity, backend executability, and component‑set agreement—on a 150‑circuit trilingual benchmark using a pipeline built on a typed circuit interchange representation.
The levels are not nested. With gpt-4o-mini, 16 circuits (10.7%, 95% CI 6.7‑16.6) were rejected by the topological validator yet ran in ngspice without error or warning; 12 of these contained exactly the requested components, only one terminal was disconnected. Conversely, 7 circuits (4.7%) passed validation but were refused by ngspice. Ten failed both checks, 117 passed both, showing each check catches errors the other misses.
A minimal three‑component divider illustrates the gap: a dangling resistor reports 5.00 V instead of 2.50 V while ngspice stays silent.
A paired ablation experiment evaluated each arm from the same model sample rather than a fresh one, separating repair stages from sampling noise. On a stratified 45‑circuit subsample, model repair raised topological validity from 40.0% to 84.4% (+20 circuits, no regressions), moved executability by a net +6 (+7, ‑1), and improved component agreement by 2. One circuit moved in opposite directions at two levels in a single repair step.
Against a direct‑netlist baseline the pipeline executed 88.7% of circuits versus 47.3% for the baseline; under an accounting that credits the baseline for every failure we cannot confidently attribute to the netlist, the success rate drops to 62.7%.
Conclusion: structural validation and simulation should be reported as distinct evaluation stages for LLM‑generated circuits. A circuit that runs is not necessarily structurally valid, and a structurally valid circuit is not necessarily executable.
Review