Computer‑use agents are increasingly evaluated in realistic desktop settings, yet existing benchmarks cover professional engineering workflows only superficially. Mechanical computer‑aided design (CAD) imposes stricter demands: an agent must manipulate geometry and constraints over long interaction horizons while producing a native project that remains valid in dimensions, construction hierarchy, and downstream engineering state.
CADWorld is a long‑horizon computer‑use benchmark built on FreeCAD. It comprises 200 tasks spanning 11 mechanical‑CAD workflow categories, such as sketching, part modeling, assembly, CAM, finite‑element analysis (FEM), measurement, mesh processing, and technical drawing. Agents interact via screenshots and GUI actions, and task success is determined by executable checks on saved FreeCAD artifacts and auxiliary outputs, covering geometric properties, parametric structure, constraints, manufacturing state, and simulation results.
Across seven current agents evaluated on the full benchmark, the strongest agent achieves only $17.5\%$ success, compared with an $87.0\%$ pass rate for an expert reference. We observe that weaker agents often fail before producing a valid artifact, while stronger agents increasingly err on structural, geometric, and construction‑process requirements. CADWorld therefore exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows.
Project website: https://cad-world.github.io
Review