Abstract
While we know that large language models (LLMs) can solve some planning problems, we do not understand the extent of these capabilities for robotics. One promising direction is to integrate the semantic knowledge of LLMs with the formal reasoning of task and motion planning (TAMP). However, designing such systems is complicated by the myriad of choices for how to integrate LLMs within TAMP.
We develop 16 algorithms that use LLMs to substitute key TAMP components. Our zero-shot experiments across 13750 evaluations and three domains reveal that LLM-based planners exhibit lower success rates and higher planning times than engineered systems. Providing geometric details increases the number of task-planning errors compared to pure PDDL descriptions, and (faster) direct LLM variants outperform (slower) reasoning variants in most cases.
Code and results are available at GitHub.
Blogger's Review: This article provides an in-depth study of the application of large language models in task and motion planning, revealing both the limitations and potential of LLMs in practical robotic planning. Although LLMs perform worse than engineered systems in some cases, the possibilities for integration still warrant further exploration, especially in improving success rates and reducing planning times.