MIT researchers have developed a new method that enhances robots' ability to think ahead while acting, resulting in smoother motions and quicker reactions. This technique allows the AI model that plans a robot's motion to forecast its future position, enabling seamless transitions between current movements and subsequent actions. Existing methods often require robots to pause and think about their next steps, causing slow and jerky motions. By basing calculations on the future state rather than the current position, the MIT method enables much faster operation.
Crucially, this technique does not add computational overhead to the planning process and can be applied to various robotic hardware. The new method has doubled the speed of robots performing tasks like pick-and-place, while significantly reducing lag time between motions. It has also improved the performance of robotic arms in dynamic activities like playing table tennis and Whack-a-Mole. This system could be particularly useful for robots executing fast, agile maneuvers in challenging real-world environments, such as emergency response or search-and-rescue, and allows quicker reactions when recovering from mistakes.
"This work sets up a good foundation for efficient, fast, accelerated, and low-cost robotics applications. We look forward to expanding our work into the latest world action models to enhance capabilities as we push to make physical AI faster," says Song Han, an associate professor in MIT's Electrical Engineering and Computer Science Department and lead author of the paper on this method.
In state-of-the-art robotics applications, generative AI systems called vision-language-action (VLA) models act as the brain of a robot, planning its next moves and executing those actions. VLA models take environmental observations and task instructions from the robot's cameras, outputting the next few steps as a set, which are then executed on the robot hardware. However, VLA inference—the real-time process during which the model processes visual inputs, reasons about the task, and outputs actions—is computationally demanding, leading to significant pauses while the robot plans its next steps. MIT researchers developed a system called VLASH that enables a VLA to predict the future state of the robot and its environment while executing current actions. This overcomes a major hurdle faced by many methods that use current states to predict future moves.
To further increase speed, the researchers generate coarser action chunks, allowing the robot to execute larger steps that follow the same trajectory, a technique known as action quantization. Although this results in a slight dip in accuracy, it enables robots to complete tasks two to three times faster. They also developed a training-augmentation method that enables the VLA to learn to utilize future state information rather than current observations, achieving a fivefold acceleration in training without additional computational overhead.
Compared to baseline methods in simulations, VLASH consistently performed faster while maintaining accuracy in robotic maneuvers. The system also outperformed these methods on real hardware in tasks like pick-and-place, stacking, and sorting. For example, VLASH sorted colored cubes into a box twice as quickly as baseline methods while achieving the same 90 percent accuracy. In the future, researchers aim to combine VLASH with more powerful generative AI systems to enhance performance and unlock new applications. This work is supported, in part, by the MIT-IBM Computing Research Lab, Amazon, the National Science Foundation, and Nvidia.
Blogger's Review: This groundbreaking research from MIT offers a fresh perspective on real-time planning for robotic motion, enhancing speed and flexibility through predictive thinking. It not only reduces motion lag but also lays a foundation for future dynamic task applications, promising wide-ranging implications. The potential integration with more powerful generative AI systems may lead to revolutionary advancements in robotics technology.