arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.01031cs.ROcs.AIcs.LG
收藏

VLASH: 通过未来状态感知的异步推断实现实时VLAs

VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference

MIT(麻省理工学院);NVIDIA(英伟达);Tsinghua University(清华大学);UC Berkeley(加州大学伯克利分校);UCSD(加州大学圣地亚哥分校);Caltech(加州理工学院)

Jiaming Tang, Yufei Sun, Yilong Zhao, Shang Yang, Yujun Lin, Zhuoyang Zhang, James Hou, Yao Lu, Zhijian Liu, Song Han

AI总结:

VLASH通过未来状态感知的异步推断框架,实现高效、准确的实时VLAs控制,显著提升反应速度和任务执行精度。

摘要:

视觉-语言-动作模型(VLAs)在多样化的机器人任务中正变得越来越强大。然而,其在现实世界中的部署仍然缓慢且低效:演示视频通常被加速5-10倍以显得流畅,但会出现明显的动作停滞和对环境变化的延迟反应。异步推断提供了一种有前景的解决方案,通过使机器人同时执行动作和进行推断,以实现连续且低延迟的控制。然而,由于机器人和环境在推断过程中持续演变,预测和执行区间之间会出现时间错位。这导致了显著的动作不稳定性,而现有方法要么降低准确性,要么引入运行时开销来缓解这一问题。我们提出了VLASH,一种通用的异步推断框架,用于VLAs,能够实现流畅、准确且快速的反应控制,而无需额外的开销或架构更改。VLASH通过将机器人状态向前滚动并与之前生成的动作块结合,来估计未来的执行时间状态,从而弥合预测和执行之间的差距。实验表明,VLASH相比同步推断实现了高达2.03倍的加速,并将反应延迟降低了高达17.4倍,同时完全保持原始准确性。此外,它使VLAs能够处理快速反应、高精度的任务,如打乒乓球和玩“击鼓传花”游戏,其中传统的同步推断失败。代码可在https://github.com/mit-han-lab/vlash上获得。

英文摘要:

Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks. However, these models are typically deployed under synchronous inference, where the robot waits for model inference to complete before acting, and cannot perceive or respond to environmental changes during action execution. This not only introduces noticeable action stalls, but also significantly increases reaction latency, fundamentally limiting the applicability of VLAs to dynamic, real-time tasks. Asynchronous inference offers a promising solution to achieve continuous and low-latency control by enabling robots to execute actions and perform inference simultaneously. However, because the robot and environment continue to evolve during inference, a temporal misalignment arises between the prediction and execution intervals. This leads to significant action instability, while existing asynchronous methods either degrade accuracy or introduce runtime overhead to mitigate it. We propose VLASH, a simple yet effective method for asynchronous VLA inference that delivers smooth, accurate, and fast reaction control without architectural changes or additional runtime overhead. VLASH leverages the future execution-time state by rolling the robot state forward with the previous action chunk, thereby bridging the gap between prediction and execution. Experiments show that VLASH reduces reaction latency by up to 11.8x compared to synchronous inference and consistently outperforms all asynchronous baselines in accuracy. With action quantization, it further achieves 1.5-2.0x task completion speedup with minimal accuracy loss. Moreover, it empowers state-of-the-art VLAs such as $π_{0.5}$ to handle fast-reaction, high-precision tasks including playing ping-pong and playing whack-a-mole, where traditional synchronous inference fails. Code is available at https://github.com/mit-han-lab/vlash

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑