As large language models (LLMs) increasingly act through tools, they must reconcile user instructions, parametric knowledge, and dynamic environmental observations before taking actions. To measure this ability we introduce KC-Bench, a controlled multi‑turn benchmark that evaluates three categories of conflicts: world‑knowledge contradictions, input inconsistencies, and multi‑source temporal conflicts.
From over 1,000 generated candidates we manually curated 238 tasks that combine a user simulator, stateful tools, deterministic environment assertions, an open‑source natural‑language evaluator, and human trajectory verification. Each task requires the model to detect and resolve conflicts during dialogue before safely invoking a tool or producing an output.
We evaluated nine models, including DeepSeek‑V4‑Flash, GLM‑5.2, and MiniMax‑M3. Results reveal substantial cross‑domain variation: no model reliably handles factual correction, identity consistency checking, and temporal conflict resolution across all settings. In simulated environments, missed conflicts can propagate to erroneous tool calls or synthetic protected‑data flows.
KC‑Bench isolates model‑level conflict‑handling behavior rather than ranking complete agent frameworks, offering a reproducible diagnostic for developing conflict‑aware reasoning and execution safeguards.
Review