视频大模型

StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, Maosong Sun

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment

Wei Li, Hehe Fan, Yongkang Wong, Mohan Kankanhalli, Yi Yang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments NeurIPS 2024 (Spotlight)

HENASY: Learning to Assemble Scene-Entities for Egocentric Video-Language Model

Khoa Vo, Thinh Phan, Kashu Yamazaki, Minh Tran, Ngan Le

专题命中 视频理解 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2024

Video Token Merging for Long-form Video Understanding

Seon-Ho Lee, Jue Wang, Zhikang Zhang, David Fan, Xinyu Li

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments 21 pages, NeurIPS 2024

Journal ref NeurIPS 2024

Adaptive Video Understanding Agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning

Sullam Jeoung, Goeric Huybrechts, Bhavana Ganesh, Aram Galstyan, Sravan Bodapati

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI

Sijie Cheng, Kechen Fang, Yangyang Yu, Sicheng Zhou, Bohao Li, Ye Tian, Tingguang Li, Lei Han, Yang Liu

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Structured Video-Language Modeling with Temporal Grouping and Spatial Grounding

Yuanhao Xiong, Long Zhao, Boqing Gong, Ming-Hsuan Yang, Florian Schroff, Ting Liu, Cho-Jui Hsieh, Liangzhe Yuan

专题命中 视频理解 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted to ICLR2024, see https://openreview.net/forum?id=5dlfiJIXoh for more details

TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations

Mingze Gao, Jingyu Liu, Mingda Li, Jiangtao Xie, Qingbin Liu, Bo Zhao, Xi Chen, Hui Xiong

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

CogVLM2: Visual Language Models for Image and Video Understanding

Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, Lei Zhao, Zhuoyi Yang, Xiaotao Gu, Xiaohan Zhang, Guanyu Feng, Da Yin, Zihan Wang, Ji Qi, Xixuan Song, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Yuxiao Dong, Jie Tang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Text-Conditioned Resampler For Long Form Video Understanding

Bruno Korbar, Yongqin Xian, Alessio Tonioni, Andrew Zisserman, Federico Tombari

专题命中 视频理解 ：video understanding(title)；long video(abstract)；分类 cs.CV

Comments Accepted to the ECCV24 conference

Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning

Zi-Yi Dou, Xitong Yang, Tushar Nagarajan, Huiyu Wang, Jing Huang, Nanyun Peng, Kris Kitani, Fu-Jen Chu

专题命中 视频理解 ：video-language(title,abstract)；分类 cs.CV

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Yanting Yang, Minghao Chen, Qibo Qiu, Jiahao Wu, Wenxiao Wang, Binbin Lin, Ziyu Guan, Xiaofei He

专题命中 视频理解 ：video-language(title,abstract)；分类 cs.CV

Comments ECCV 2024 camera-ready

Fine-grained Knowledge Graph-driven Video-Language Learning for Action Recognition

Rui Zhang, Yafen Lu, Pengli Ji, Junxiao Xue, Xiaoran Yan

专题命中 视频理解 ：video-language(title,abstract)；分类 cs.MM

Vamos: Versatile Action Models for Video Understanding

Shijie Wang, Qi Zhao, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, Chen Sun

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted to ECCV 2024 (European Conference on Computer Vision). Code and models are released at https://brown-palm.github.io/Vamos/

Zero-Shot Long-Form Video Understanding through Screenplay

Yongliang Wu, Bozheng Li, Jiawang Cao, Wenbo Zhu, Yi Lu, Weiheng Chi, Chuyun Xie, Haolin Zheng, Ziyue Su, Jay Wu, Xu Yang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Highest Score Award to the CVPR'2024 LOVEU Track 1 Challenge

PVUW 2024 Challenge on Complex Video Understanding: Methods and Results

Henghui Ding, Chang Liu, Yunchao Wei, Nikhila Ravi, Shuting He, Song Bai, Philip Torr, Deshui Miao, Xin Li, Zhenyu He, Yaowei Wang, Ming-Hsuan Yang, Zhensong Xu, Jiangtao Yao, Chengjing Wu, Ting Liu, Luoqi Liu, Xinyu Liu, Jing Zhang, Kexin Zhang, Yuting Yang, Licheng Jiao, Shuyuan Yang, Mingqi Gao, Jingnan Luo, Jinyu Yang, Jungong Han, Feng Zheng, Bin Cao, Yisi Zhang, Xuanxu Lin, Xingjian He, Bo Zhao, Jing Liu, Feiyu Pan, Hao Fang, Xiankai Lu

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments MOSE Challenge: https://henghuiding.github.io/MOSE/ChallengeCVPR2024, MeViS Challenge: https://henghuiding.github.io/MeViS/ChallengeCVPR2024

AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding

Alessandro Suglia, Claudio Greco, Katie Baker, Jose L. Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Code available https://github.com/alanaai/EVUD

VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding

Muhammad Maaz, Hanoona Rasheed, Salman Khan, Fahad Khan

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Technical Report

Memory Consolidation Enables Long-Context Video Understanding

Ivana Balažević, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, Olivier J. Hénaff

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

No Time to Waste: Squeeze Time into Channel for Mobile Video Understanding

Yingjie Zhai, Wenshuo Li, Yehui Tang, Xinghao Chen, Yunhe Wang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Foundation Models for Video Understanding: A Survey

Neelu Madan, Andreas Moegelmose, Rajat Modi, Yogesh S. Rawat, Thomas B. Moeslund

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, Ser-Nam Lim

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted at CVPR 2024. Project Page https://boheumd.github.io/MA-LMM/

Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, Li Yuan

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024 (Highlight)

MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens

Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, Mohamed Elhoseiny

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments 6 pages,8 figures

A Unified Framework for Human-centric Point Cloud Video Understanding

Yiteng Xu, Kecheng Ye, Xiao Han, Yiming Ren, Xinge Zhu, Yuexin Ma

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

OmniVid: A Generative Framework for Universal Video Understanding

Junke Wang, Dongdong Chen, Chong Luo, Bo He, Lu Yuan, Zuxuan Wu, Yu-Gang Jiang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

VideoAgent: Long-form Video Understanding with Large Language Model as Agent

Xiaohan Wang, Yuhui Zhang, Orr Zohar, Serena Yeung-Levy

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectives

Simone Alberto Peirone, Francesca Pistilli, Antonio Alliegro, Giuseppe Averta

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted at IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024. Project webpage at https://sapeirone.github.io/EgoPack

EPIC Fields: Marrying 3D Geometry and Video Understanding

Vadim Tschernezki, Ahmad Darkhalil, Zhifan Zhu, David Fouhey, Iro Laina, Diane Larlus, Dima Damen, Andrea Vedaldi

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV

Comments Published at NeurIPS 2023. 24 pages, 15 figures. Project Webpage: http://epic-kitchens.github.io/epic-fields

Cutup and Detect: Human Fall Detection on Cutup Untrimmed Videos Using a Large Foundational Video Understanding Model

Till Grutschus, Ola Karrar, Emir Esenov, Ekta Vats

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV