统一扩散VLA：通过联合离散去噪扩散过程实现视觉-语言-动作模型
Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process
AI总结：
本文提出统一扩散VLA模型，通过联合离散去噪扩散过程实现视觉语言动作的协同生成与执行，提升多模态任务的效率与性能。
摘要：
视觉-语言-动作（VLA）模型旨在理解和执行自然语言指令和视觉观察。最近的研究将未来图像整合到理解-执行循环中，产生了统一的VLA，能够联合理解、生成和执行——阅读文本和图像并生成未来的图像和动作。然而，这些模型要么依赖外部专家进行模态统一，要么将图像生成和动作预测视为独立过程，限制了这些任务之间直接协同的优势。我们的核心理念是通过同步去噪过程优化生成和动作，迭代细化使动作从初始化开始演进，在持续且充分的视觉引导下进行。我们基于提出的统一扩散VLA和联合离散去噪扩散过程（JD3P）进行理论构建，该过程将多种模态整合到单一去噪轨迹中，作为实现理解、生成和执行内在协同的关键机制。我们的模型和理论基于统一的多模态分词空间和混合注意力机制。我们进一步提出两阶段训练流水线和几种推理时的技术，以优化性能和效率。我们的方法在CALVIN、LIBERO和SimplerEnv等基准上取得了最先进的性能，比自回归方法快4倍的推理速度，并通过深入分析和实际应用验证其有效性。我们的项目页面可在https://irpn-eai.github.io/UD-VLA.github.io/上找到。
英文摘要：
Vision-language-action (VLA) models aim to understand natural language instructions and visual observations and to execute corresponding actions as an embodied agent. Recent work integrates future images into the understanding-acting loop, yielding unified VLAs that jointly understand, generate, and act -- reading text and images and producing future images and actions. However, these models either rely on external experts for modality unification or treat image generation and action prediction as separate processes, limiting the benefits of direct synergy between these tasks. Our core philosophy is to optimize generation and action jointly through a synchronous denoising process, where the iterative refinement enables actions to evolve from initialization, under constant and sufficient visual guidance. We ground this philosophy in our proposed Unified Diffusion VLA and Joint Discrete Denoising Diffusion Process (JD3P), which is a joint diffusion process that integrates multiple modalities into a single denoising trajectory to serve as the key mechanism enabling understanding, generation, and acting to be intrinsically synergistic. Our model and theory are built on a unified tokenized space of all modalities and a hybrid attention mechanism. We further propose a two-stage training pipeline and several inference-time techniques that optimize performance and efficiency. Our approach achieves state-of-the-art performance on benchmarks such as CALVIN, LIBERO, and SimplerEnv with 4$\times$ faster inference than autoregressive methods, and we demonstrate its effectiveness through in-depth analysis and real-world evaluations. Our project page is available at https://irpn-eai.github.io/UD-VLA.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
