Single-step retrosynthesis is a one‑to‑many inverse problem, yet most models are trained and evaluated against a single recorded set of reactants. Reference‑based training provides no explicit positive signal for unrecorded alternatives, and exact‑match evaluation discards chemically plausible predictions that do not match the reference. We propose RetroRLFF (Retrosynthesis via Reinforcement Learning from Forward Feedback), which optimizes reactant generation using both reference agreement and forward consistency with the target product. Starting from a supervised decoder‑only Transformer, RetroRLFF generates complete reactant sequences and employs a frozen forward reaction model to predict the resulting products. A sequence‑level reward combines reference agreement and forward consistency while penalizing invalid outputs. Direct Advantage‑Based Policy Optimization updates the policy without human preference annotations or a preference‑trained reward model. Experiments on the USPTO‑50K dataset show that RetroRLFF outperforms all 14 baselines in candidate‑level round‑trip accuracy at k = 1, 3, 5, 10. With roughly 50 million parameters, it achieves 90.1% top‑1 round‑trip accuracy, exceeding the strongest baseline R‑SMILES by 8.5 percentage points. Compared with its supervised policy, round‑trip accuracy rises from 83.1% to 90.1%, while exact‑match accuracy remains at 57.9% versus 57.5%. In an anonymized expert assessment of 131 randomly selected test cases, organic synthesis experts prefer RetroRLFF over EditRetro in 67.9% of comparisons. These results support forward feedback as a training signal for improving forward consistency while retaining top‑1 reference recovery. Expert assessment provides complementary evidence of synthetic feasibility, although neither forward consistency nor expert preference guarantees experimental feasibility.
Review