Conversational recommendation agents enable users to convey complex intents in natural language, e.g., “recommend Italian indie artists I haven’t heard before.” The main difficulty in cold‑start scenarios is planning – selecting, ordering, and invoking tools without real user data. We address this with two technical components.
The first component is a multi‑turn synthetic data generation pipeline. It converts single‑turn prompts into realistic multi‑turn conversations, allowing systematic pre‑launch evaluation. By using dialogue templates and stochastic variations, the generated data covers intent expression, tool calls, and error recovery.
The second component is a self‑improvement loop. At its core lies variance‑based contrastive optimization combined with a coding agent that iteratively fixes planning and tool‑use errors. The coding agent automatically pinpoints defects, generates corrective code, and re‑evaluates the conversation quality.
Results show an 8% quality gain over a highly tuned manual prompt. The system has been productionized at Spotify, dramatically speeding up iteration cycles for the conversational recommendation agent. Online A/B tests report a 14% increase in listening time, a 5% rise in weekly active users, and a 5% reduction in skip rate compared with the previous session‑refinement‑only experience.
This work offers a practical framework for accelerating the development of conversational recommendation agents in industry.
Review