视频大模型

Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation

Hongcheng Gao, Jiashu Qu, Jingyi Tang, Baolong Bi, Yue Liu, Hongyu Chen, Li Liang, Li Su, Qingming Huang

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation

Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, Kai-Wei Chang

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments 41 pages, 33 Figures

Is Your Video Language Model a Reliable Judge?

Ming Liu, Wensheng Zhang

专题命中 视频数据与评测 ：video language model(title,abstract)；分类 cs.CV

OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation

Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, Siyu Zhu

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments 11 pages, 8 figures, 5 tables

What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Quality

Zihan Wang, Songlin Li, Lingyan Hao, Xinyu Hu, Bowen Song

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection

Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, Si Liu

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments 14 pages, 14 figures

Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality

Tianle Zhang, Langtian Ma, Yuchen Yan, Yuchen Zhang, Kai Wang, Yue Yang, Ziyao Guo, Wenqi Shao, Yang You, Yu Qiao, Ping Luo, Kaipeng Zhang

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding

Houlun Chen, Xin Wang, Hong Chen, Zeyang Zhang, Wei Feng, Bin Huang, Jia Jia, Wenwu Zhu

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted by 38th NeurIPS Datasets & Benchmarks Track (NeurIPS 2024)

Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality

Ge Ya Luo, Gian Mario Favero, Zhi Hao Luo, Alexia Jolicoeur-Martineau, Christopher Pal

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models

Shicheng Li, Lei Li, Shuhuai Ren, Yuanxin Liu, Yi Liu, Rundong Gao, Xu Sun, Lu Hou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024. 19 pages, 3 figures, 8 tables. Data is available at https://github.com/lscpku/VITATECS

PiTe: Pixel-Temporal Alignment for Large Video-Language Model

Yang Liu, Pengxiang Ding, Siteng Huang, Min Zhang, Han Zhao, Donglin Wang

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

DeVAn: Dense Video Annotation for Video-Language Models

Tingkai Liu, Yunzhe Tao, Haogeng Liu, Qihang Fan, Ding Zhou, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Published in 62nd ACL (2024) Main Track

Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment

Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, Ning Liu

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

Comments Accepted by ACMMM 24

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments 38 pages, 44 figures

Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies

Hung-Ting Su, Chun-Tong Chao, Ya-Ching Hsu, Xudong Lin, Yulei Niu, Hung-Yi Lee, Winston H. Hsu

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments Project page: https://ander1119.github.io/TiM

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, Yu Qiao

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments CVPR 2024 highlight: updated version with Mistral and better performances for MVBench/NExT-QA/STAR/TVQA/EgoSchema/IntentQA

SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge

Andong Wang, Bo Wu, Sunli Chen, Zhenfang Chen, Haotian Guan, Wei-Ning Lee, Li Erran Li, Chuang Gan

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments CVPR

Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection

Yang Liu, Tongfei Shen, Dong Zhang, Qingying Sun, Shoushan Li, Guodong Zhou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by ICMR 2024

VidLA: Video-Language Alignment at Scale

Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan, Son Tran, Benjamin Z. Yao, Belinda Zeng, Mubarak Shah, Trishul Chilimbi

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2024

Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks

Pedro Rodriguez, Mahmoud Azab, Becka Silvert, Renato Sanchez, Linzy Labson, Hardik Shah, Seungwhan Moon

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

Comments EACL 2023 Camera Ready

Synopses of Movie Narratives: a Video-Language Dataset for Story Understanding

Yidan Sun, Qin Chao, Yangfeng Ji, Boyang Li

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments 25 pages, 17 figures

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2022

RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval

Xing Wu, Chaochen Gao, Zijia Lin, Zhongyuan Wang, Jizhong Han, Songlin Hu

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments EMNLP 2022

Egocentric Video-Language Pretraining

Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2022. Double champions at Ego4D and EPIC-Kitchens, CVPR 2022 challenges. 23 pages, 13 figures, 12 tables. Code: https://github.com/showlab/EgoVLP

Egocentric Video-Language Pretraining @ Ego4D Challenge 2022

Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Preprint. 4 pages, 2 figures, 5 tables. Code: https://github.com/showlab/EgoVLP. The Ego4D challenge technical report of EgoVLP arXiv:2206.01670. See EPIC challenge technical report arXiv:2207.01334 for overlap

Egocentric Video-Language Pretraining @ EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022

Kevin Qinghong Lin, Alex Jinpeng Wang, Rui Yan, Eric Zhongcong Xu, Rongcheng Tu, Yanru Zhu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Wei Liu, Mike Zheng Shou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments To appeared in CVPRW22. 5 pages, 2 figures, 2 tables. Code: https://github.com/showlab/EgoVLP. The EPIC challenge technical report of EgoVLP arXiv:2206.01670. See Ego4D challenge technical report arXiv:2207.01622

Object-aware Video-language Pre-training for Retrieval

Alex Jinpeng Wang, Yixiao Ge, Guanyu Cai, Rui Yan, Xudong Lin, Ying Shan, Xiaohu Qie, Mike Zheng Shou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments CVPR2022; Code: https://github.com/FingerRec/OA-Transformer

FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Framework

Santiago Castro, Ruoyao Wang, Pingxuan Huang, Ian Stewart, Oana Ignat, Nan Liu, Jonathan C. Stroud, Rada Mihalcea

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted at ACL 2022 Main conference. Camera-ready version

DTVNet+: A High-Resolution Scenic Dataset for Dynamic Time-lapse Video Generation

Jiangning Zhang, Chao Xu, Yong Liu, Yunliang Jiang

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments This work extends the previous DTVNet in ECCV'20, and we further present a high-quality and high-resolution Quick-Sky-Time dataset and carry out more adequate experiments and analysis

Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intents

Shuting Wang, Yunqi Liu, Zixin Yang, Ning Hu, Zhicheng Dou, Chenyan Xiong

机构 * School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院） ； Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院） ； Serendipity One Inc.（Serendipity One公司） 纠错

专题命中 视频数据与评测 ：video generation(title)；text-to-video(abstract)