Debt collection is a critical negotiation task in the financial industry, with significant practical relevance and academic value, serving as a rich behavioral testbed for human-centered dialogue systems. While large language models (LLMs) have shown promise in dialogue and negotiation, effectively evaluating their performance in complex scenarios remains a major challenge: existing benchmarks uniformly assume users to be static, rational agents with fixed preferences, failing to capture the rich behavioral heterogeneity inherent in real-world debt collection.
To bridge this gap, we propose DebtBench, the first public persona-enriched debt collection benchmark that highlights behavioral heterogeneity in negotiation. Moreover, we develop DebtGPT, a debt collection agent trained to jointly optimize financial recovery and interaction experience.
Our experimental results, using 16 state-of-the-art LLMs, find that most existing models struggle in this complex yet realistic scenario, whereas DebtGPT outperforms all open-source baselines and achieves performance on par with GPT-4o. The code and data are available at GitHub.
Blogger's Review: This paper introduces a new benchmark by incorporating behavioral heterogeneity into debt collection dialogue systems, pushing the application of dialogue systems in complex scenarios. The superior performance of DebtGPT not only validates the effectiveness of this approach but also provides an important reference framework for future research. Such innovative ideas deserve exploration and application in more domains.