Decoupled Video Generation with Chain of Training-free Diffusion Model Experts
University of Sydney(悉尼大学);Southeast University(东南大学);Central South University(中南大学);Shanghai Jiaotong University(上海交通大学);Sensetime Research(商汤科技研究院);Hong Kong University of Science and Technology(香港科技大学)
Wenhao Li, Yichao Cao, Xiu Su, Xi Lin, Shan You, Mingkai Zheng, Yi Chen, Chang Xu
Video generation models hold substantial potential in areas such as filmmaking. However, current video diffusion models need high computational costs and produce suboptimal results due to extreme complexity of video generation task. In this paper, we propose \textbf{ConFiner}, an efficient video generation framework that decouples video generation into easier subtasks: structure \textbf{con}trol and spatial-temporal re\textbf{fine}ment. It can generate high-quality videos with chain of off-the-shelf diffusion model experts, each expert responsible for a decoupled subtask. During the refinement, we introduce coordinated denoising, which can merge multiple diffusion experts' capabilities into a single sampling. Furthermore, we design ConFiner-Long framework, which can generate long coherent video with three constraint strategies on ConFiner. Experimental results indicate that with only 10\% of the inference cost, our ConFiner surpasses representative models like Lavie and Modelscope across all objective and subjective metrics. And ConFiner-Long can generate high-quality and coherent videos with up to 600 frames.
VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification
Beihang University(北京航空航天大学);Zhongguancun Academy(中关村学院);Joy Future Academy(京东探索研究院);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学)
针对视频流模型扩展 AWR 时的冷启动与引导爆炸问题,提出 VideoAWR 解耦采样与优化锚点,结合解耦时间步校正,在 Wan2.1 上实现稳定优化与更优视频质量。
WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);KlingAI;The University of Hong Kong(香港大学)
Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising
University of Science and Technology of China(中国科学技术大学);University of Electronic Science and Technology of China(电子科技大学);Fudan University(复旦大学);Georgia Institute of Technology(佐治亚理工学院);Shanghai Jiao Tong University(上海交通大学)
IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Kling Team, Kuaishou Technology(快手技术 Kling 团队);Cardiff University(卡迪夫大学)
Dual-Expert Consistency Model for Efficient and High-Quality Video Generation
Nanjing University(南京大学);The University of Hong Kong(香港大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Chinese Academy of Sciences(中国科学院大学);S-Lab, Nanyang Technological University(南洋理工大学S实验室)
Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);The Chinese University of Hong Kong(香港中文大学);Huawei Cloud Computing Technologies Co., Ltd(华为云计算技术有限公司)
UniVerse-1: Unified Audio-Video Generation via Stitching of Experts
The Hong Kong University of Science and Technology(GuangZhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学)
Animate Your Thoughts: Decoupled Reconstruction of Dynamic Natural Vision from Slow Brain Activity
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhengzhou University(郑州大学);Beijing University of Posts and Telecommunications(北京邮电大学)
CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generation
The University of Hong Kong(香港大学);Kling Team, Kuaishou Technology(快手技术 Kling 团队);Tsinghua University(清华大学);Zhejiang University(浙江大学);Microsoft Research Project Page(微软研究院项目页面)
AesRM: Improving Video Aesthetics with Expert-Level Feedback
The University of Hong Kong(香港大学);Fudan University(复旦大学);Zhejiang University(浙江大学);Hong Kong University of Science and Technology(香港科学与技术大学);University of Sydney(悉尼大学);Alibaba Group(阿里巴巴集团)