We introduce SmartSalesBench, the first multi‑turn conversational benchmark tailored for assessing large language model (LLM) sales agents. Existing task‑oriented dialogue sets assume cooperative users and focus on task completion, whereas SmartSalesBench pits agents against buyer personas that deliberately resist and challenge, with conversations extending up to twenty turns. The benchmark comprises five agent profiles, twelve resistant buyer personas, fifteen smartphone products spanning six price tiers, and four controlled experiments run over three iterations, totaling 13,680 sessions. Both agents and buyers are generated via retrieval‑augmented generation (RAG) to ground the dialogue in factual information. Nine evaluation metrics are defined in three tiers: primary outcome metrics CR, BRS, RIR; secondary quality metrics SMS, PFS, ETI; and exploratory metrics ITS, ARI, SPI planned for future reporting. Evaluation combines automated LLM‑as‑judge scoring with human assessment (n=50, κ=0.71, Pearson r=0.81 with BRS).
Review