In modern applications, large language models (LLMs) increasingly collaborate with humans rather than merely serving as passive task providers. We introduce the HAS-Framework, a graph-based structure that treats humans and LLM-driven agents as first-class participants with explicit roles, permissions, communication paths, and action authority.
Building on this framework, HAS-Bench evaluates Human-Agent Systems under configurable human participation, encompassing agency levels, interaction channels, and persona policies. The benchmark measures not only task outcomes but also process-level collaboration behaviors, including clarification quality, feedback utilization, control calibration, safety, initiative, and interaction cost.
Experiments across six domains demonstrate that human participation can significantly enhance task completion and failure recovery, but the benefits depend on when, how, and by whom human input is exercised.
Blogger's Review: The introduction of HAS-Bench provides a fresh perspective on human-agent collaboration systems, highlighting the critical role of human participation in various contexts. By quantifying interactions between humans and LLMs, researchers can better understand how to optimize these systems for improved efficiency and safety. The framework's flexibility and configurability will serve as an important foundation for future research.