Perceptual task planning demands both accurate perception of uncertain scenes and the ability to generate action sequences that obey logical rules. Conventional pipelines discretize perception into symbolic facts before planning, discarding uncertainty and breaking task‑level feedback to perception. We introduce a generic, fully differentiable neuro‑soft‑symbolic framework that integrates visual perception and task planning within a single computational graph. The framework maintains a continuous soft symbolic state, lifts domain rules into a differentiable soft $T_P$ transition operator, and optimizes action logits over a short planning horizon. Gradients from the planning objective also update perception parameters, allowing task‑relevant perceptual representations to be refined during planning.
On the Blocksworld benchmark, our method solves all 40/40 LatPlan‑40 tasks and 596/600 PlanBench‑600 tasks, outperforming LatPlan (33/40) and a reasoning‑model baseline (587/600) while requiring substantially less computation and time. In a perceptual‑uncertainty ablation, unfreezing perception raises success from 59% to 83%. We further conduct task‑and‑motion simulations on Blocksworld scenes, validating the compatibility between decoded task plans and downstream robotic motion execution.
Key contributions include: (1) a continuous soft symbolic state that preserves perceptual uncertainty; (2) a differentiable soft $T_P$ operator that embeds logical rules into the gradient flow; (3) end‑to‑end optimization that jointly improves perception and planning.
Review