Prompt optimization aims to improve the image a user receives while keeping the requested scene unchanged. Conventional stylistic rewriting often weakens object attributes and relations even when global text‑image similarity stays high. To address this, we introduce StruPO—a structure‑aware prompt editor. StruPO uses a calibrated scene graph as a shared representation for policy conditioning and image‑grounded supervision. The graph explicitly marks what must be preserved, and operations such as deletion, reordering, and insertion provide token‑aligned ways to edit the prompt. Consequently, structural feedback from an external critique becomes a training signal for a reusable editing policy.
Learning from this signal requires handling uncertain grounding and stochastic image generation. StruPO retains grounding confidence via smooth reward shaping, and after each edit it averages multiple generated images before computing group‑relative advantages, thereby reducing noise. Experiments span three disjoint 2K‑prompt test sets and four frozen generation backbones, comparing against learned prompt optimizers, iterative structural feedback, diffusion‑model fine‑tuning, and inference‑time attention control. Results show StruPO improves the faithfulness‑aesthetics trade‑off, with consistent gains in relational fidelity. Independent automatic evaluators and a study with 15 annotators provide complementary evidence, and human preference margins grow with relation complexity. Ablations connect these gains to graph conditioning, local editing, and image‑grounded learning. After a single graph calibration, the policy produces reusable optimized text in one pass, eliminating the need for an iterative critique‑rewrite loop.
Review