Abstract
Vision-Language-Action (VLA) models have demonstrated impressive performance across various embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, leading to significant inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time.
In this paper, we propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system-level accelerations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling.
Extensive experiments demonstrate that Jetson-PI achieves 8.66x and 5.41x improvements in control frequency compared with naive PyTorch and vla.cpp on NVIDIA Jetson Orin, while outperforming VLASH by 14.8% in average success rate on the LIBERO benchmark.
The code of our asynchronous algorithm is available on GitHub, and our efficient llama.cpp-based inference engine is available on GitHub.
Blogger's Review: The introduction of Jetson-PI opens new possibilities for VLA model applications on low-power devices. By leveraging foresight correction and optimized scheduling, it significantly enhances the efficiency of robot control, pushing technological advancements and laying a foundation for future intelligent robotic applications.