User instructions are often underspecified regarding safety limits, creating behavioral hazards. To quantify these hazards we introduce BARC-Bench, a suite of 110 tasks spread across nine simulated workspaces where seemingly benign requests leave room for unsafe actions. The benchmark defines safety boundaries using workspace logs, implicit environmental context, and unstated social norms. For instance, an agent asked to schedule off‑boarding discussions must set up the meeting without prematurely exposing the employee’s resignation via a shared calendar invitation. We evaluate four frontier model configurations, running five attempts per task. Safety completion rates vary from 9.1% to 72.7%, and even the strongest setup still violates safety in 27.3% of tasks. These findings highlight an asymmetric operational risk: models frequently finish complex workflows but struggle to consistently respect unstated boundaries, posing a major hurdle for safe autonomous delegation.
Review