Full‑duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle overlaps, take the floor and yield in real‑time dialogue. Existing benchmarks mainly test these abilities with explicit turn‑management commands, while deployed assistants are often configured via roles or personas that require implicit inference of appropriate behavior. To address this gap we introduce DuplexSpeechBench‑IFEval (DSB‑IFEval) for evaluating implicit instruction following in real‑time spoken interaction. DSB‑IFEval comprises 1,038 test cases covering eight assistant roles and evaluates five conditioning protocols: default behavior, explicit behavioral instructions, persona‑implied behavior, combined persona‑rule conditioning, and instruction conflict. We measure floor management with a deterministic Instruction Adherence Score (IAS) and persona‑consistent content with an LLM‑judged Persona Adherence Score (PAS). Experiments on six real‑time speech systems reveal architecture‑dependent trade‑offs. Full‑duplex models such as F‑Actor and PersonaPlex are highly sensitive to whether behavior is stated explicitly or inferred from a persona, showing IAS drops of 9.7% and 4.5% respectively under persona‑only conditioning. In contrast, GPT‑Realtime, MiniCPM‑o and Fun‑Audio‑Chat strongly adhere to persona‑consistent content, yet their floor‑management behavior remains unchanged across explicit and persona‑only instructions and is limited on several proactive actions. Even when systems reliably follow conflicting directives according to their prescribed persona, they struggle to override them under safety conflicts. These findings indicate that inferring role‑implied behavior, executing it at the right conversational moment, and resolving competing instructions remain distinct challenges for full‑duplex voice agents.
Review