arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.01800cs.CV
收藏

AtomoVideo:高保真图像到视频生成

AtomoVideo: High Fidelity Image-to-Video Generation

Alimama Tech, Alibaba Group(阿里巴巴集团阿里妈妈技术)

Litong Gong, Yiran Zhu, Weijie Li, Xiaoyang Kang, Biao Wang, Tiezheng Ge, Bo Zheng

AI总结:

本研究提出高保真图像到视频生成框架AtomoVideo,通过多粒度图像注入、适配训练等设计,在保时序稳定性的同时提升运动强度与保真度,可扩展至长视频预测,性能优于主流方法。

摘要:

近年来,依托性能优异的文本到图像生成技术,视频生成领域取得了快速的重大进展。本研究提出了一种用于图像到视频生成的高保真框架,命名为AtomoVideo。基于多粒度图像注入机制,我们提升了生成视频与给定输入图像的保真度。此外,得益于高质量数据集与训练策略,我们在保持优异时间一致性与稳定性的同时,实现了更强的运动幅度。我们的架构可灵活扩展至视频帧预测任务,通过迭代生成实现长序列预测。此外,由于采用了适配器(adapter)训练设计,本方法可与现有个性化模型及可控模块良好结合。经定量与定性评估,AtomoVideo相较主流方法取得了更优结果,更多示例可查看项目官网:https://atomo-video.github.io/。

英文摘要:

Recently, video generation has achieved significant rapid development based on superior text-to-image generation techniques. In this work, we propose a high fidelity framework for image-to-video generation, named AtomoVideo. Based on multi-granularity image injection, we achieve higher fidelity of the generated video to the given image. In addition, thanks to high quality datasets and training strategies, we achieve greater motion intensity while maintaining superior temporal consistency and stability. Our architecture extends flexibly to the video frame prediction task, enabling long sequence prediction through iterative generation. Furthermore, due to the design of adapter training, our approach can be well combined with existing personalized models and controllable modules. By quantitatively and qualitatively evaluation, AtomoVideo achieves superior results compared to popular methods, more examples can be found on our project website: https://atomo-video.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑