Planning under uncertainty in continuous domains is crucial for autonomous systems but computationally demanding. Tree-based search methods like Monte Carlo Tree Search (MCTS) remain popular, yet their branching structure can result in sampling budgets that grow exponentially with lookahead depth in the worst cases. Continuous state or action spaces pose additional challenges, as planners must decide where to search within an infinite branching hierarchy.
We propose Graph Sparse Sampling (GSS), an online planning algorithm that shares sampled futures across multiple candidate decisions instead of sampling separate successors for each candidate action. This branch-free graph exposes large GPU-friendly batches while using heuristics to focus computation. We prove finite-sample performance guarantees for GSS covering full-rank or low-rank generative simulators via smoothed backups, and discrete or sampled continuous action spaces.
Under suitable overlap, regularity, and action-coverage conditions, these bounds have polynomial dependence on the planning horizon, formalizing when shared futures can avoid the exponential horizon dependence of tree-shaped sparse sampling. We demonstrate continuous-control simulations where GSS substantially outperforms tree-based planners on long horizons or achieves near-optimal performance, supporting no-branching graph planning as a complementary design principle for online control.
Blogger's Review: The Graph Sparse Sampling (GSS) algorithm presents a novel approach to mitigating computational complexity in continuous MDP planning by optimizing computation efficiency through shared sampling data. Its theoretical guarantees solidify its effectiveness, particularly in long-horizon predictions, highlighting the potential of graph structures in online control that warrants further exploration.