AI agents that share a secret convention can coordinate through messages that appear innocent, making pure‑message monitoring insufficient. We introduce a behavioural test: when a message is re‑worded while preserving its meaning, a partner that knows the secret code changes its action, whereas an outsider occupying the same role does not.
We call the resulting metric Partner‑Outsider Positive Listening (PoPLi), defined as the difference in action change between the two agents. It is computed post‑hoc by replaying logged prompts and comparing the action shift caused by the original message versus its synonymic rewrite:$$\text{PoPLi}=\Delta_{\text{partner}}-\Delta_{\text{outsider}}$$
We evaluated PoPLi in a simulated market under five conditions: (1) no collusion; (2) joint reward with neutral messages; (3) same as (2) but with an openly stated intention; (4) a covert first‑letter code; (5) the code plus a stated intention. Results show that whenever the code is present, PoPLi ranges from 0.48 to 0.59 (95% intervals all above 0.29), while without the code it lies between -0.10 and -0.01. By contrast, message‑only monitors fail to separate the conditions: a monitor reading only the speaker’s messages answered “no” in all 50 markets, and a monitor reading every seller’s messages falsely flagged 9 out of 10 non‑collusive markets.
Thus PoPLi detects the use of a private channel rather than collusion per se, and it cannot detect collusion that relies solely on incentives or meaning‑based codes.
Review