视频大模型

Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video

Ben Saunders, Necati Cihan Camgoz, Richard Bowden

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Stochastic Talking Face Generation Using Latent Distribution Matching

Ravindra Yadav, Ashish Sardana, Vinay P Namboodiri, Rajesh M Hegde

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments InterSpeech 2020

Condensed Movies: Story Based Retrieval with Contextual Embeddings

Max Bain, Arsha Nagrani, Andrew Brown, Andrew Zisserman

专题命中 视频生成 ：text-to-video(abstract)；分类 cs.CV

Comments Appears in: Asian Conference on Computer Vision 2020 (ACCV 2020) - Oral presentation

Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample

Shir Gur, Sagie Benaim, Lior Wolf

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms

Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Wei Feng, Yang Liu, Jianjun Zhao

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments 11 pages, 7 figures; This paper has been accepted to ACM-MM 2020

How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals

Umur Aybars Ciftci, Ilke Demir, Lijun Yin

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments To be published in the proceedings of 2020 IEEE/IAPR International Joint Conference on Biometrics (IJCB)

Lyric Video Analysis Using Text Detection and Tracking

Shota Sakaguchi, Jun Kato, Masataka Goto, Seiichi Uchida

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments 15 pages, 8 figures, DAS 2020

Self-Training for Domain Adaptive Scene Text Detection

Yudi Chen, Wei Wang, Yu Zhou, Fei Yang, Dongbao Yang, Weiping Wang

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Local-Global Video-Text Interactions for Temporal Grounding

Jonghwan Mun, Minsu Cho, Bohyung Han

专题命中 视频生成 ：text-to-video(abstract)；分类 cs.CV

Comments CVPR 2020; code available in https://github.com/JonghwanMun/LGI4temporalgrounding

Learning a Text-Video Embedding from Incomplete and Heterogeneous Data

Antoine Miech, Ivan Laptev, Josef Sivic

专题命中 视频生成 ：text-to-video(abstract)；分类 cs.CV

Comments The paper had a major update in January 2020 after a bug we found in the codebase that affected many results

Talking Face Generation by Conditional Recurrent Adversarial Network

Yang Song, Jingwen Zhu, Dawei Li, Xiaolong Wang, Hairong Qi

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments Project Page:http://web.eecs.utk.edu/~ysong18/projects/talkingface/talkingface.html

Order Matters: Shuffling Sequence Generation for Video Prediction

Junyan Wang, Bingzhang Hu, Yang Long, Yu Guan

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments This manuscript has been accepted at BMVC 2019. See the project at https://github.com/andrewjywang/SEENet

StoryGAN: A Sequential Conditional GAN for Story Visualization

Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, Jianfeng Gao

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Sliced Wasserstein Generative Models

Jiqing Wu, Zhiwu Huang, Dinesh Acharya, Wen Li, Janine Thoma, Danda Pani Paudel, Luc Van Gool

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments This paper is accepted by CVPR 2019, accidentally uploaded as a new submission (arXiv:1904.05408, which has been withdrawn). The code is available at this https URL https:// github.com/musikisomorphie/swd.git

Sliced Wasserstein Generative Models

Jiqing Wu, Zhiwu Huang, Dinesh Acharya, Wen Li, Janine Thoma, Danda Pani Paudel, Luc Van Gool

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments This paper is submitted to arxiv twice, thus withdraw one of the versions. See arXiv:1706.02631 instead

Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks

Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, Yusuke Uchida

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments Accepted to ECCV 2018 Workshop: Computer Vision for Fashion, Art and Design. Project page is at https://dena.com/intl/anime-generation

Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation

Lijie Fan, Wenbing Huang, Chuang Gan, Junzhou Huang, Boqing Gong

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments 10 pages

Every Smile is Unique: Landmark-Guided Diverse Smile Generation

Wei Wang, Xavier Alameda-Pineda, Dan Xu, Pascal Fua, Elisa Ricci, Nicu Sebe

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments Accepted as a poster in Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Parallel Multiscale Autoregressive Density Estimation

Scott Reed, Aäron van den Oord, Nal Kalchbrenner, Sergio Gómez Colmenarejo, Ziyu Wang, Dan Belov, Nando de Freitas

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Spectral video construction from RGB video: Application to Image Guided Neurosurgery

Md. Abul Hasnat, Jussi Parkkinen, Markku Hauta-Kasari

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments Experiments were conducted in 2011, Paper rewritten with recent review in 2015

Contextual RNN-GANs for Abstract Reasoning Diagram Generation

Arnab Ghosh, Viveka Kulharia, Amitabha Mukerjee, Vinay Namboodiri, Mohit Bansal

专题命中 视频生成 ：video generation(abstract)；分类 cs.CV

Comments To Appear in AAAI-17 and NIPS Workshop on Adversarial Training

Large-Scale Query-by-Image Video Retrieval Using Bloom Filters

Andre Araujo, Jason Chaves, Haricharan Lakshman, Roland Angst, Bernd Girod

专题命中 视频生成 ：long video(abstract)；分类 cs.MM

Unlocking Temporal Generalization in Hamiltonian Video Dynamics Models

解锁哈密顿视频动力学模型中的时间泛化

Eli Laird, Corey Clark

机构 * Department of Computer Science, Southern Methodist University（南卫理公会大学计算机科学系） 纠错

专题命中 视频生成 ：video generation(abstract,comments)

AI总结 研究世界模型在可变时间分辨率下预测动力学的问题，利用哈密顿生成网络（HGN），指出其在非保守环境中时间泛化失效的问题及原因，通过针对性修复实现稳定动力学预测，推荐连续时间视频生成中时间泛化的策略。

Comments To appear in the 1st Workshop on Physics-Aware Video Generation and Restoration at the 28th International Conference on Pattern Recognition

ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Construction

编码智能体作为世界模拟器

Hongyu Wang, Jingquan Wang, Ashvin Anilkumar, Bocheng Zou, Radu Serban, Dan Negrut

机构 * Department of Mechanical & Aerospace Engineering, University of Wisconsin-Madison（威斯康星大学麦迪逊分校机械与航空航天工程系） ； School of Computer, Data, and Information Sciences, University of Wisconsin-Madison（威斯康星大学麦迪逊分校计算机、数据与信息科学学院） 纠错

专题命中 视频生成 ：text-to-video(abstract)

AI总结 提出一个通过可执行模拟代码构建基于物理的世界模型的智能体框架，协调规划、代码生成、视觉审查和物理分析智能体，迭代修正代码以满足物理约束，在物理准确性、指令忠实度和视觉质量上超越视频模型。

Time-Correlated Video Bridge Matching

时间相关视频桥匹配

Viacheslav Vasilev, Arseny Ivanov, Nikita Gushchin, Maria Kovaleva, Alexander Korotin

机构 * Kandinsky Lab（Kandinsky实验室） ； Applied AI Institute（应用人工智能研究所） ； HSE University（高等经济大学） 纠错

专题命中 视频生成 ：video generation(abstract)

AI总结 本文提出时间相关视频桥匹配框架，解决视频生成中时间相关序列建模问题，通过建模序列间依赖性和时间相关性，提升视频生成质量与重建精度。

Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education

计算教育中学生对AI生成教学视频的感知与偏好

Esse Ciego, Shubbhi Taneja, Wilson Wong, Amanpreet Kapoor

专题命中 视频生成 ：video generation(abstract)

AI总结 本研究通过对170名计算专业学生的调查，探究其对Knowlify生成的Markdown教学AI视频的感知与偏好，发现学生认可视频质量但对其课堂广泛采用存顾虑，明确了AI视频的适用场景与潜在问题。

LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments

LeapBot-WA：通过预测性潜在对齐实现的世界锚定动作模型

Pei Liu, Nan Zheng, Lang Zhang, Daojie Peng, Yanan Zhang, Feilong Kong, Mingyue Feng, Jiachao Liu, Yaonong Wang, Qifeng Chen, Jun Ma

专题命中 视频生成 ：video generation(abstract)

AI总结 研究针对世界动作模型依赖像素级视频生成的瓶颈，提出LeapBot-WA，通过预测性潜在对齐建立新范式，引入ISAE弥合模态差距，设计MoT架构，在多数据集上表现出色，实现高效强大的潜在中心范式及零样本鲁棒性和现实世界迁移。

From Passive Video to Editable Experience: Physically Grounded Experience Synthesis for Embodied Intelligence

从被动视频到可编辑体验：具身智能的物理基础体验合成

Jia Luo

专题命中 视频生成 ：video generation(abstract)

AI总结 针对具身AI的数据瓶颈，提出Pegasus低资源框架，通过结构化知识传递将人类操作视频转化为机器人可学习数据，经多基准与机器人评估验证其跨具身翻译及数据生成的有效性。

GenHOI: Contact-Aware Humanoid-Object Interaction by Imitating Generated Videos without Task-Specific Training

GenHOI: 通过模仿生成视频实现接触感知的人形机器人-物体交互，无需任务特定训练

Zhihai Bi, Qiang Zhang, Guoyang Zhao, Jiahang Cao, Xueyin Luo, Yushan Zhang, Jinglan Xu, Ruoyu Geng, Yulin Li, Andrew F. Luo, Jun Ma

机构 * The University of Tokyo（东京大学） ； National University of Singapore（新加坡国立大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； Tsinghua University（清华大学） 纠错

专题命中 视频生成 ：video generation(abstract)

AI总结 提出GenHOI框架，通过模仿单个生成视频实现人形机器人零样本执行多种物体交互任务，无需任务特定训练或物理演示数据，利用接触事件和手-物接触区域编码为几何约束优化轨迹。

Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generation

自回归生成中自我修正盲点的频谱起源

Ingrid Petrova, Luan Vejsiu

机构 * European University of Tirana（地拉那欧洲大学） 纠错

专题命中 视频生成 ：video generation(abstract)

AI总结 研究大型自回归语言模型自我修正盲点问题，提出频谱代数理论SPARC，定义错误传播算子，推导激活阈值，证明基于强化学习的验证器-校正器训练收敛条件，实验验证定理，频谱预测与盲点率匹配。