Large Language Models (LLMs) have been adopted as decision agents for Traffic Signal Control (TSC) because they can produce human‑readable reasoning. Existing LLM‑based TSC approaches optimize only against final rewards and cannot separate valid from flawed reasoning steps, causing useful and misleading steps to be updated together and weakening the model's ability to learn effective reasoning.
To address this gap, we introduce the ProcessLight framework, which decomposes signal decisions into verifiable semantic steps. Building on ProcessLight, we develop Step‑wise Traffic Process Policy Optimization (STeP‑PO), a reinforcement‑learning scheme that assigns credit at the step level. STeP‑PO evaluates local reasoning quality with step quality scores, measures each step's influence on the final action via step importance, and computes step‑level advantages over a semantic step‑tree structure. These advantages are then propagated to reasoning tokens, enabling fine‑grained policy updates beyond outcome‑only rewards.
Extensive experiments on several real‑world datasets show that our methods outperform baselines in both convergence speed and control performance. The code is released at https://github.com/wenzhaoabc/processlight.
Review