Moaw：释放视频扩散模型中的运动感知
Moaw: Unleashing Motion Awareness for Video Diffusion Models
AI总结：
Moaw通过训练视频扩散模型进行运动感知，实现零样本运动迁移，推动生成建模与运动理解的结合。
摘要：
视频扩散模型在大规模数据集上训练后，能够自然地捕捉帧间共享特征的对应关系。近期的研究利用这一特性，在零样本设置下用于光流预测和跟踪任务。受这些发现的启发，我们探讨监督训练是否能更充分地利用视频扩散模型的跟踪能力。为此，我们提出了Moaw框架，该框架旨在释放视频扩散模型中的运动感知能力，并利用其进行运动迁移。具体而言，我们训练了一个用于运动感知的扩散模型，将其模态从图像到视频生成转变为视频到密集跟踪。然后我们构建了一个带有运动标签的数据集，以识别编码最强运动信息的特征，并将这些特征注入到结构相同的视频生成模型中。由于两个网络的同质性，这些特征可以自然地以零样本方式适应，从而在不添加额外适配器的情况下实现运动迁移。我们的工作为连接生成建模和运动理解提供了新的范式，为更统一和可控的视频学习框架铺平了道路。
英文摘要：
Video diffusion models, trained on large-scale datasets, naturally capture correspondences of shared features across frames. Recent works have exploited this property for tasks such as optical flow prediction and tracking in a zero-shot setting. Motivated by these findings, we investigate whether supervised training can more fully harness the tracking capability of video diffusion models. To this end, we propose Moaw, a framework that unleashes motion awareness for video diffusion models and leverages it to facilitate motion transfer. Specifically, we train a diffusion model for motion perception, shifting its modality from image-to-video generation to video-to-dense-tracking. We then construct a motion-labeled dataset to identify features that encode the strongest motion information, and inject them into a structurally identical video generation model. Owing to the homogeneity between the two networks, these features can be naturally adapted in a zero-shot manner, enabling motion transfer without additional adapters. Our work provides a new paradigm for bridging generative modeling and motion understanding, paving the way for more unified and controllable video learning frameworks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
