Semantic‑ID based generative recommendation treats retrieval and ranking as autoregressive generation over hierarchical item identifiers. The typical pipeline first applies supervised fine‑tuning (SFT) and then a reward‑based generative post‑training (GRPO). Vanilla GRPO, however, is poorly aligned with this tree‑structured task: under a frozen SFT checkpoint, the exact target often does not appear among the top‑16 candidates of a 50‑beam constrained ranking, and in harder cases none of the candidates stays on the target SID branch. This prompt‑level symptom raises a training concern—when on‑policy GRPO groups also miss the target, item‑level rewards may yield weak or degenerate variations even if some candidates follow part of the target path.
To address this failure mode we propose Difficulty‑Aware Semantic‑ID Optimization (DASO), framing it as an online rollout‑allocation problem. Instead of fixed difficulty buckets or uniformly injecting ground‑truth completions, DASO profiles each rollout group by prefix‑match depth, identifies bottleneck SID levels where candidates diverge from the target path, and reallocates a bounded portion of the group to prefix‑guided completions while retaining raw rollouts for contrast. A SID‑prefix reward supplies graded credit, and an auxiliary SFT anchor mitigates regression on examples already solved by the SFT checkpoint.
On public benchmarks DASO improves over MiniOneRec‑style GRPO on 11 of 12 metrics and achieves the best result on 9 of 12 metrics; it also boosts most level‑wise recall metrics on an internal recommendation task.
Blogger's Review: By incorporating fine‑grained tree awareness and dynamic rollout allocation, DASO effectively alleviates the sparse‑reward issue of traditional GRPO in hierarchical recommendation, demonstrating the strong potential of post‑training enhancements for generative recommender systems.