统一视频-动作联合去噪用于灵巧动作与数据生成
Unified Video-Action Joint Denoising for Dexterous Action and Data Generation
AI总结：
提出Donk模型，通过联合建模交互视频与手部轨迹的分布，实现灵巧手的动作生成与数据增强。
摘要：
最近的世界动作模型通过将广泛的视觉动态先验与可执行的机器人动作对齐来利用视频基础模型。我们从分布的角度重新审视这种对齐。现有的公式通常将对齐的先验缩小为基于观测的未来动作策略分布。相比之下，我们通过在多条件机制下对交互视频和可执行手部轨迹的联合空间进行建模，保持更广泛的分布。我们提出了Donk，一个用于灵巧手的统一视频-动作去噪模型。通过语言、初始图像和初始手部状态，Donk采样未来视频和双手MANO轨迹作为动作策略。在没有图像条件的情况下，相同的去噪架构从文本条件分布中采样配对的视频-动作展开，将对齐的视频先验转化为数据引擎。在动作、视频和仅文本生成评估中，Donk在相同的统一训练方案下提高了灵巧轨迹的准确性，保持了强大的视频保真度，并产生了平滑的文本条件动作展开。
英文摘要：
Recent world action models leverage video foundation models by aligning broad visual-dynamics priors with executable robot actions. We revisit this alignment from a distributional perspective. Existing formulations typically narrow the aligned prior into an observation-conditioned policy distribution over future actions. In contrast, we keep the distribution broader by modeling the joint space of interaction videos and executable hand trajectories under multiple conditioning regimes. We propose Donk, a unified video-action denoising model for dexterous hands. With language, an initial image, and the initial hand state, Donk samples future videos and bimanual MANO trajectories as an action policy. Without the image condition, the same denoising architecture samples paired video-action rollouts from a text-conditioned distribution, turning the aligned video prior into a data engine. Across action, video, and text-only generation evaluations, Donk improves dexterous trajectory accuracy, preserves strong video fidelity, and produces smooth text-conditioned action rollouts under the same unified training recipe.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
