实时可控的自回归视频扩散
Real-Time Motion-Controllable Autoregressive Video Diffusion
AI总结：
AR-Drag是一种基于强化学习的实时图像到视频生成模型，通过自回归机制和轨迹奖励模型实现高效运动控制，提升视频生成质量和效率。
摘要：
实时可控的视频生成仍面临挑战，因为双向扩散模型固有的延迟以及缺乏有效的自回归（AR）方法。现有的AR视频扩散模型局限于简单的控制信号或文本到视频生成，并且在少数步骤生成时常常出现质量下降和运动伪影。为了解决这些挑战，我们提出了AR-Drag，这是第一个基于强化学习的少数步骤AR视频扩散模型，用于实时图像到视频生成的多样化运动控制。我们首先对基础I2V模型进行微调以支持基本运动控制，然后通过基于轨迹的奖励模型进一步改进。我们的设计通过自回归机制保持马尔可夫属性，并通过选择性地在去噪步骤中引入随机性来加速训练。广泛的实验表明，AR-Drag实现了高质量的视觉保真度和精确的运动对齐，相比最先进的可控VDMs显著减少了延迟，同时仅使用1.3B参数。额外的可视化可以在我们的项目页面上找到：https://kesenzhao.github.io/AR-Drag.github.io/。
英文摘要：
Real-time motion-controllable video generation remains challenging due to the inherent latency of bidirectional diffusion models and the lack of effective autoregressive (AR) approaches. Existing AR video diffusion models are limited to simple control signals or text-to-video generation, and often suffer from quality degradation and motion artifacts in few-step generation. To address these challenges, we propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control. We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement learning with a trajectory-based reward model. Our design preserves the Markov property through a Self-Rollout mechanism and accelerates training by selectively introducing stochasticity in denoising steps. Extensive experiments demonstrate that AR-Drag achieves high visual fidelity and precise motion alignment, significantly reducing latency compared with state-of-the-art motion-controllable VDMs, while using only 1.3B parameters. Additional visualizations can be found on our project page: https://kesenzhao.github.io/AR-Drag.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
