视频大模型

Open-Sora: Democratizing Efficient Video Production for All

Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, Yang You

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Phased Consistency Models

Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, Xiaogang Wang, Hongsheng Li

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments NeurIPS 2024

TC4D: Trajectory-Conditioned Text-to-4D Generation

Sherwin Bahmani, Xian Liu, Wang Yifan, Ivan Skorokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, Andrea Tagliasacchi, David B. Lindell

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments ECCV 2024; Project Page: https://sherwinbahmani.github.io/tc4d

T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback

Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sugato Basu, Wenhu Chen, William Yang Wang

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments Project page: https://t2v-turbo.github.io/

Technical Report: Competition Solution For Modelscope-Sora

Shengfu Chen, Hailong Liu, Wenzhao Wei

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Advancing Video Quality Assessment for AIGC

Xinli Yue, Jianhui Sun, Han Kong, Liangchao Yao, Tianyi Wang, Lei Li, Fengyun Rao, Jing Lv, Fan Xia, Yuetang Deng, Qian Wang, Lingchen Zhao

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments 5 pages, 1 figure

Factorized-Dreamer: Training A High-Quality Video Generator with Limited and Low-Quality Data

Tao Yang, Yangming Shi, Yunwen Huang, Feng Chen, Yin Zheng, Lei Zhang

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Segment Anything for Videos: A Systematic Survey

Chunhui Zhang, Yawen Cui, Weilin Lin, Guanjie Huang, Yan Rong, Li Liu, Shiguang Shan

专题命中 视频生成 ：video generation(abstract)；video understanding(abstract)；分类 cs.CV

Comments https://github.com/983632847/SAM-for-Videos

DragVideo: Interactive Drag-style Video Editing

Yufan Deng, Ruida Wang, Yuhao Zhang, Yu-Wing Tai, Chi-Keung Tang

专题命中 视频生成 ：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

DreamGaussian4D: Generative 4D Gaussian Splatting

Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, Ziwei Liu

专题命中 视频生成 ：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Technical report. Project page is at https://jiawei-ren.github.io/projects/dreamgaussian4d Code is at https://github.com/jiawei-ren/dreamgaussian4d

APLA: Additional Perturbation for Latent Noise with Adversarial Training Enables Consistency

Yupu Yao, Shangqi Deng, Zihan Cao, Harry Zhang, Liang-Jian Deng

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models

Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, Lichao Sun

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments 37 pages, 18 figures; GitHub: https://github.com/lichao-sun/SoraReview

Distilling Vision-Language Models on Millions of Videos

Yue Zhao, Long Zhao, Xingyi Zhou, Jialin Wu, Chun-Te Chu, Hui Miao, Florian Schroff, Hartwig Adam, Ting Liu, Boqing Gong, Philipp Krähenbühl, Liangzhe Yuan

专题命中 视频生成 ：text-to-video(abstract)；video-language(abstract)；分类 cs.CV

Comments CVPR 2024. Project page: https://zhaoyue-zephyrus.github.io/video-instruction-tuning

Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts

Yuyang Zhao, Enze Xie, Lanqing Hong, Zhenguo Li, Gim Hee Lee

专题命中 视频生成 ：video generation(abstract)；video diffusion(abstract)；分类 cs.CV

Comments Project page: https://make-a-protagonist.github.io

LLM as an Art Director (LaDi): Using LLMs to improve Text-to-Media Generators

Allen Roush, Emil Zakirov, Artemiy Shirokov, Polina Lunina, Jack Gane, Alexander Duffy, Charlie Basil, Aber Whitcomb, Jim Benedetto, Chris DeWolfe

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments 12 pages, System Demonstration/Industry paper. Preprint

Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

Samaneh Azadi, Akbar Shah, Thomas Hayes, Devi Parikh, Sonal Gupta

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2304.07410

FlexLip: A Controllable Text-to-Lip System

Dan Oneata, Beata Lorincz, Adriana Stan, Horia Cucu

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 eess.IV

Comments 16 pages, 4 tables, 4 figures

Journal ref Sensors. 2022; 22(11):4104

NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion

Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, Nan Duan

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)；分类 cs.CV

VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer

Zineng Tang, Jaemin Cho, Hao Tan, Mohit Bansal

专题命中 视频生成 ：text-to-video(abstract)；video-language(abstract)；分类 cs.CV

Comments NeurIPS 2021 (19 pages)

Generative Video Transformer: Can Objects be the Words?

Yi-Fu Wu, Jaesik Yoon, Sungjin Ahn

专题命中 视频生成 ：video generation(abstract)；video reasoning(abstract)；分类 cs.CV

Comments Published in ICML 2021

Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation

Lincheng Li, Suzhen Wang, Zhimeng Zhang, Yu Ding, Yixing Zheng, Xin Yu, Changjie Fan

专题命中 视频生成 ：video generation(abstract)；long video(abstract)；分类 cs.CV

Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2

Tail-Aware HiFloat4: 面向Wan2.2的W4A4训练后量化

Zhanfeng Feng, Shuai Guo, Xin Di, Long Peng, Yang Cao, Zhengjun Zha

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)

AI总结 提出Tail-Aware HiFloat4方法，通过感知激活尾部的百分位校准和紧凑PTQ状态恢复，在HiFloat4数值格式下对Wan2.2进行W4A4训练后量化，减少罕见校准异常值的影响。

How do people watch AI-generated videos of physical scenes?

人们如何观看物理场景的AI生成视频？

Danqing Shi, Lan Jiang, Katherine M. Collins, Shangzhe Wu, Ayush Tewari, Miri Zilka

专题命中 视频生成 ：video generation(abstract)；video understanding(abstract)

AI总结 研究探讨了人们观看物理场景AI生成视频时的注视行为，发现高真实感视频的观看方式受观众感知真实性影响，而非视频实际真实性。

RecTable: Fast Modeling Tabular Data with Rectified Flow

Masane Fuchi, Tomohiro Takagi

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)

Comments 19 pages, 7 figures, 10 tables

SkyScript-100M: 1,000,000,000 Pairs of Scripts and Shooting Scripts for Short Drama

Jing Tang, Quanlu Jia, Yuqiang Xie, Zeyu Gong, Xiang Wen, Jiayi Zhang, Yalong Guo, Guibin Chen, Jiangping Yang

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)

Comments 18 pages, 12 figures

RoboDreamer: Learning Compositional World Models for Robot Imagination

Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, Chuang Gan

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)

A Review of Multi-Modal Large Language and Vision Models

Kilian Carolan, Laura Fennelly, Alan F. Smeaton

专题命中 视频生成 ：video generation(abstract)；text-to-video(abstract)

Comments 33 pages, 1 figure

Music-oriented Dance Video Synthesis with Pose Perceptual Loss

Xuanchi Ren, Haoran Li, Zijian Huang, Qifeng Chen

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV、eess.IV、cs.MM

VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning

VBVR-Pro：用于原生视觉推理的可扩展且可验证工具套件

Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, Zhongang Cai

机构 * Nanyang Technological University（南洋理工大学） ； University of California, Berkeley（加州大学伯克利分校） ； University of California, San Diego（加州大学圣地亚哥分校） ； The University of Tokyo（东京大学） ； The Chinese University of Hong Kong（香港中文大学） ； University of Michigan（密歇根大学） ； Johns Hopkins University（约翰霍普金斯大学） ； University of California, Davis（加州大学戴维斯分校） ； University of California, Los Angeles（加州大学洛杉矶分校） ； Carnegie Mellon University（卡内基梅隆大学） ； Columbia University（哥伦比亚大学） ； University of Toronto（多伦多大学） 纠错

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV、cs.MM

AI总结 本研究提出VBVR-Pro，一个可扩展可验证的原生视觉推理闭环测试平台，含300个生成任务、确定性评分器及多生成器机制研究，训练模型在多视觉推理基准上迁移性强，发布全部相关资源。

Comments Homepage: https://video-reason.com/

Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

超越试错：面向图像到视频一致性的智能体优化

Aman Tyagi, Hemanth Boinpally, Jonathan Chen, Douglas Gebert, Steven Hickson

机构 * Google Cloud（谷歌云） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV、cs.MM

AI总结 针对图像到视频模型试错效率低的问题，提出Agentic Self-Improvement框架，通过两阶段优化提升视频与文本一致性，生成视频胜率达69%，为视频生成模型提供实用可控的优化方法。