视频大模型 - arXivDaily 专题

2501.11325 2025-01-22 cs.CV cs.AI 70%

CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation

Zheng Chong, Wenqing Zhang, Shiyue Zhang, Jun Zheng, Xiao Dong, Haoxiang Li, Yiling Wu, Dongmei Jiang, Xiaodan Liang

专题命中视频扩散模型：video generation(abstract)；long video(abstract)；分类 cs.CV

Comments 11 pages, 8 figures, 5 tables

2501.10018 2025-01-20 cs.CV 70%

DiffuEraser: A Diffusion Model for Video Inpainting

Xiaowen Li, Haolan Xue, Peiran Ren, Liefeng Bo

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments 11pages, 13figures

2403.18103 2025-01-09 cs.LG cs.CV 70%

Tutorial on Diffusion Models for Imaging and Vision

Stanley H. Chan

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

2501.00601 2025-01-07 cs.CV cs.AI cs.GR 70%

DreamDrive: Generative 4D Scene Modeling from Street View Images

Jiageng Mao, Boyi Li, Boris Ivanovic, Yuxiao Chen, Yan Wang, Yurong You, Chaowei Xiao, Danfei Xu, Marco Pavone, Yue Wang

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Project page: https://pointscoder.github.io/DreamDrive/

2501.00103 2025-01-03 cs.CV 70%

LTX-Video: Realtime Video Latent Diffusion

Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, Ofir Bibi

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

2412.18607 2024-12-25 cs.CV 70%

DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers

Yuntao Chen, Yuqi Wang, Zhaoxiang Zhang

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

2412.04000 2024-12-11 cs.CV 70%

IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation

Sejong Yang, Seoung Wug Oh, Yang Zhou, Seon Joo Kim

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Underreview

2411.18677 2024-12-02 cs.CV cs.AI cs.LG 70%

MatchDiffusion: Training-free Generation of Match-cuts

Alejandro Pardo, Fabio Pizzati, Tong Zhang, Alexander Pondaven, Philip Torr, Juan Camilo Perez, Bernard Ghanem

专题命中视频扩散模型：video diffusion(abstract)；text-to-video(abstract)；分类 cs.CV

Comments https://matchdiffusion.github.io

2411.10818 2024-11-19 cs.GR cs.CV 70%

FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations

Hmrishav Bandyopadhyay, Yi-Zhe Song

专题命中视频扩散模型：video diffusion(abstract)；text-to-video(abstract)；分类 cs.CV

Comments Code: https://github.com/hmrishavbandy/FlipSketch

2411.04924 2024-11-08 cs.CV 70%

MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views

Yuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, Jianfei Cai

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments NeurIPS 2024, Project page: https://donydchen.github.io/mvsplat360, Code: https://github.com/donydchen/mvsplat360

2410.16266 2024-10-22 cs.CV cs.AI 70%

3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors

Xi Liu, Chaoyi Zhou, Siyu Huang

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024 Spotlight

2409.12960 2024-09-20 cs.CV cs.GR 70%

LVCD: Reference-based Lineart Video Colorization with Diffusion Models

Zhitong Huang, Mohan Zhang, Jing Liao

专题命中视频扩散模型：video diffusion(abstract)；long video(abstract)；分类 cs.CV

Comments Accepted by ACM Transactions on Graphics and SIGGRAPH Asia 2024. Project page: https://luckyhzt.github.io/lvcd

2407.19453 2024-07-30 cs.CV 70%

FIND: Fine-tuning Initial Noise Distribution with Policy Optimization for Diffusion Models

Changgu Chen, Libing Yang, Xiaoyan Yang, Lianggangxu Chen, Gaoqi He, CHangbo Wang, Yang Li

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

2407.15153 2024-07-23 cs.CV 70%

Anchored Diffusion for Video Face Reenactment

Idan Kligvasser, Regev Cohen, George Leifman, Ehud Rivlin, Michael Elad

专题命中视频扩散模型：video generation(abstract)；long video(abstract)；分类 cs.CV

2407.12684 2024-07-18 cs.CV 70%

4Dynamic: Text-to-4D Generation with Hybrid Priors

Yu-Jie Yuan, Leif Kobbelt, Jiwen Liu, Yuan Zhang, Pengfei Wan, Yu-Kun Lai, Lin Gao

专题命中视频扩散模型：video diffusion(abstract)；text-to-video(abstract)；分类 cs.CV

2405.18326 2024-06-10 cs.CV 70%

VITON-DiT: Learning In-the-Wild Video Try-On from Human Dance Videos via Diffusion Transformers

Jun Zheng, Fuwei Zhao, Youjiang Xu, Xin Dong, Xiaodan Liang

专题命中视频扩散模型：video generation(abstract)；long video(abstract)；分类 cs.CV

Comments Project Page: https://zhengjun-ai.github.io/viton-dit-page/

2405.18132 2024-05-29 cs.CV 70%

EG4D: Explicit Generation of 4D Object without Score Distillation

Qi Sun, Zhiyang Guo, Ziyu Wan, Jing Nathan Yan, Shengming Yin, Wengang Zhou, Jing Liao, Houqiang Li

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

2405.17306 2024-05-29 cs.CV 70%

Controllable Longer Image Animation with Diffusion Models

Qiang Wang, Minghua Liu, Junjun Hu, Fan Jiang, Mu Xu

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments https://wangqiang9.github.io/Controllable.github.io/

2404.09967 2024-05-27 cs.CV cs.AI cs.LG 70%

Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model

Han Lin, Jaemin Cho, Abhay Zala, Mohit Bansal

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments First two authors contributed equally; Project page: https://ctrl-adapter.github.io/

2312.13763 2024-01-04 cs.CV cs.LG 70%

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, Karsten Kreis

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments Project page: https://research.nvidia.com/labs/toronto-ai/AlignYourGaussians/

2304.08818 2023-12-29 cs.CV cs.LG 70%

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, Karsten Kreis

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments Conference on Computer Vision and Pattern Recognition (CVPR) 2023. Project page: https://research.nvidia.com/labs/toronto-ai/VideoLDM/

2312.09109 2023-12-15 cs.CV cs.AI 70%

VideoLCM: Video Latent Consistency Model

Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, Nong Sang

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

2311.17009 2023-12-05 cs.CV 70%

Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer

Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kasten, Tali Dekel

专题命中视频扩散模型：video diffusion(abstract)；text-to-video(abstract)；分类 cs.CV

Comments Project page: https://diffusion-motion-transfer.github.io/

2311.16498 2023-11-29 cs.CV cs.GR 70%

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, Mike Zheng Shou

专题命中视频扩散模型：video diffusion(abstract)；long video(abstract)；分类 cs.CV

Comments Project Page at https://showlab.github.io/magicanimate

2303.13439 2023-03-24 cs.CV 70%

Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, Humphrey Shi

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments The project is available at: https://github.com/Picsart-AI-Research/Text2Video-Zero

2302.03011 2023-02-07 cs.CV 70%

Structure and Content-Guided Video Synthesis with Diffusion Models

Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, Anastasis Germanidis

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Project page at https://research.runwayml.com/gen1

2301.12661 2023-01-31 cs.SD cs.LG cs.MM eess.AS 70%

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

Rongjie Huang, Jiawei Huang, Dongchao Yang, Yi Ren, Luping Liu, Mingze Li, Zhenhui Ye, Jinglin Liu, Xiang Yin, Zhou Zhao

专题命中视频扩散模型：video generation(abstract)；text-to-video(abstract)；分类 cs.MM

Comments Audio samples are available at https://Text-to-Audio.github.io

2212.03250 2022-12-08 cs.CV 70%

Neural Cell Video Synthesis via Optical-Flow Diffusion

Manuel Serna-Aguilera, Khoa Luu, Nathaniel Harris, Min Zou

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments 9 pages, 2 tables, 7 figures

2206.07696 2022-11-15 cs.CV cs.LG stat.ML 70%

Diffusion Models for Video Prediction and Infilling

Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, Andrea Dittadi

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Published in TMLR (11/2022)

2608.06008 2026-08-07 cs.RO 新提交 67%

Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features

Adaptive-WAM：基于质量引导的中间视频扩散特征早期退出规划

Sining Ang, Yuguang Yang, Yan Wang

专题命中视频扩散模型：video generation(abstract)；video diffusion(abstract)

AI总结 Adaptive-WAM是基于Wan2.2-5B主干的质量感知多出口规划器，通过动态分配主干深度减少计算量，在NAVSIM、nuScenes等数据集上取得优异规划性能，延迟显著降低。

1. 视频扩散模型 1306 篇

CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation

DiffuEraser: A Diffusion Model for Video Inpainting

Tutorial on Diffusion Models for Imaging and Vision

DreamDrive: Generative 4D Scene Modeling from Street View Images

LTX-Video: Realtime Video Latent Diffusion

DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers

IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation

MatchDiffusion: Training-free Generation of Match-cuts

FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations

MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views

3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors

LVCD: Reference-based Lineart Video Colorization with Diffusion Models

FIND: Fine-tuning Initial Noise Distribution with Policy Optimization for Diffusion Models

Anchored Diffusion for Video Face Reenactment

4Dynamic: Text-to-4D Generation with Hybrid Priors

VITON-DiT: Learning In-the-Wild Video Try-On from Human Dance Videos via Diffusion Transformers

EG4D: Explicit Generation of 4D Object without Score Distillation

Controllable Longer Image Animation with Diffusion Models

Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

VideoLCM: Video Latent Consistency Model

Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer

MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model

Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

Structure and Content-Guided Video Synthesis with Diffusion Models

Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models

Neural Cell Video Synthesis via Optical-Flow Diffusion

Diffusion Models for Video Prediction and Infilling

Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features