Language agents degrade sharply in performance as reasoning chains lengthen, even when each individual step is simple. This phenomenon is traced to context dilution: an agent's investigative state (what it has confirmed, what it suspects, and what it still needs) exists only implicitly within an expanding context window, where early discoveries are overshadowed by later retrievals.
We introduce SLEUTH, which makes this state explicit and actionable through a structured epistemic working memory: the agent maintains Confirmed Facts grounded to sources, Active Hypotheses ranked by evidence, and Open Questions that directly drive its next action. Across five multi-hop benchmarks and five established baselines, SLEUTH's advantage increases with difficulty, from +5 points on HotpotQA to +11 on 4-hop chains, surpassing Reflexion without multiple episodes.
Analyzing the remaining gap reveals the evidence sufficiency problem: agents often find the answer but fail to commit, exhausting their budget on needless verification. A lightweight commitment trigger addresses this issue, but only when the agent already maintains a structured state: applying the same trigger to an unstructured agent yields no improvement, isolating organized epistemic state as a necessary condition for effective commitment.
Finally, enforcing protocol adherence on a weaker model recovers up to +19 points on the hardest problems, demonstrating that how an agent organizes its reasoning, rather than raw model capability, is the active ingredient for scaling multi-hop reasoning.
Blogger's Review: The introduction of SLEUTH offers a novel approach to enhancing multi-hop reasoning in language agents through structured working memory, significantly improving reasoning outcomes. This method underscores the importance of information organization, indicating that relying solely on model capability is insufficient; a well-structured state is essential for achieving efficient reasoning. I look forward to seeing more methodological research on optimizing such structured memory in the future.