CLUE (Closed‑Loop contextual Uncertainty rEsolution) is a framework that actively reduces contextual uncertainty for underspecified tasks expressed in natural language. It first leverages a large language model (LLM) to hypothesize task‑relevant concepts and candidate plans, then builds an online language‑embedded map to ground those hypotheses into concrete actions. The policy evaluates hypotheses through closed‑loop interaction, continuously refining its plan as new information is gathered, thereby jointly inferring success criteria, relevant cues, and their locations.
We deployed CLUE on a Boston Dynamics Spot robot across three real indoor and outdoor settings, covering 15 tasks that require object disambiguation, functional inference, and occlusion reasoning. CLUE achieved a success rate within 7 percentage points of an oracle policy and outperformed an LLM‑based planner without closed‑loop feedback by roughly fourfold. Additional experiments showed that merely building and querying a language‑enriched map yields about one‑third the success rate of CLUE while consuming over ten times more visual‑language model tokens.
These findings indicate that offline language maps alone are insufficient for complex contextual planning; closed‑loop interaction and online hypothesis testing are essential. Further information is available at https://zacravichandran.github.io/CLUE.
Review