Simulating deformable objects is essential for various robotic manipulation applications, yet accurately predicting their dynamics remains challenging. We propose Physics-Guided Residual Dynamics (PGRD), a hybrid simulation framework that combines the advantages of physics-based and learning-based approaches.
Specifically, PGRD integrates an optimizable spring-mass simulator as a backbone with a learned neural network that predicts residual corrections to physics-based predictions. We adopt a velocity-based formulation to ensure stable simulation and utilize a sliding-window transformer architecture to capture temporal dependencies.
Our results show that PGRD achieves more accurate outcomes than both purely physics-based and learning-based methods across a diverse set of real-world deformable objects. Furthermore, we demonstrate PGRD's utility in two applications: manipulation planning via Model Predictive Control, including a language-conditioned setting with a generated goal image; and interactive simulation through action-conditioned video prediction using 3D Gaussian Splatting.
Blogger's Review: The introduction of the PGRD framework illustrates how the integration of physics and learning can enhance the accuracy of deformable object simulation, especially in complex robotic tasks. Its significant improvements in stability and accuracy provide new insights and tools for the field.