视频大模型

Mamba Fusion: Learning Actions Through Questioning

Zhikang Dong, Apoorva Beedu, Jason Sheinkopf, Irfan Essa

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Stony Brook University（石溪大学） ； Google Research（谷歌研究院） 纠错

专题命中 视频理解 ：video language model(abstract)；分类 cs.CV

AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation

Sitong Gong, Yunzhi Zhuge, Lu Zhang, Yifan Wang, Pingping Zhang, Lijun Wang, Huchuan Lu

机构 * School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院） ； School of Innovation and Entrepreneurship, Dalian University of Technology（大连理工大学创新创业学院） ； School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted to IEEE Transactions on Multimedia (TMM)

Video Quality Assessment for Online Processing: From Spatial to Temporal Sampling

Jiebin Yan, Lei Wu, Yuming Fang, Xuelin Liu, Xue Xia, Weide Liu

机构 * School of Information Technology, Jiangxi University of Finance and Economics（江西财经大学信息技术学院） ； Harvard Medical School, Harvard University（哈佛大学医学院） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Situational Scene Graph for Structured Human-centric Situation Understanding

Chinthani Sugandhika, Chen Li, Deepu Rajan, Basura Fernando

机构 * College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院） ； Centre for Frontier AI Research, Agency for Science, Technology and Research（新加坡科技研究局前沿人工智能研究中心） ； Institute of High-Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted for WACV 2025

VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning

Ji Soo Lee, Jongha Kim, Jeehye Na, Jinyoung Park, Hyunwoo J. Kim

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments AAAI 2025

Valley2: Exploring Multimodal Models with Scalable Vision-Language Design

Ziheng Wu, Zhenghao Chen, Ruipu Luo, Can Zhang, Yuan Gao, Zhentao He, Xian Wang, Haoran Lin, Minghui Qiu

机构 * ByteDance（字节跳动） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Zero-shot Shark Tracking and Biometrics from Aerial Imagery

Chinmay K Lalgudi, Mark E Leone, Jaden V Clark, Sergio Madrigal-Mora, Mario Espinoza

机构 * Stanford University（斯坦福大学） ； Flinders University（弗林德斯大学） ； Centro de Investigación en Ciencias del Mar y Limnología, Universidad de Costa Rica（哥斯达黎加大学海洋与湖沼科学研究中心） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, João Carreira, Olivier J. Hénaff

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

OV-HHIR: Open Vocabulary Human Interaction Recognition Using Cross-modal Integration of Large Language Models

Lala Shakti Swarup Ray, Bo Zhou, Sungho Suh, Paul Lukowicz

机构 * RPTU Kaiserslautern-Landau（莱茵兰-普法尔茨州立大学凯撒斯劳滕-兰道分校） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted in IEEE ICASSP 2025

Detection-Fusion for Knowledge Graph Extraction from Videos

Taniya Das, Louis Mahon, Thomas Lukasiewicz

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 12 pages, To be submitted to a conference

Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy

Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Bhargava Kumar, Amit Agarwal, Ishan Banerjee, Srikant Panda, Tejaswini Kumar

机构 * University of Washington（华盛顿大学） ； New York University（纽约大学） ； Columbia University（哥伦比亚大学） ； Liverpool John Moores University（利物浦约翰摩尔大学） ； Chennai Mathematical Institute（金奈数学研究所） ； Birla Institute of Technology（比拉理工学院） 纠错

专题命中 视频理解 ：text-to-video(abstract)；分类 cs.CV

Video Domain Incremental Learning for Human Action Recognition in Home Environments

Yuanda Hu, Xing Liu, Meiying Li, Yate Ge, Xiaohua Sun, Weiwei Guo

机构 * College of Design and Innovation, Tongji University（同济大学设计与创新学院） ； SUSTech School of Design, Southern University of Science and Technology（南方科技大学设计学院） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

PruneVid: Visual Token Pruning for Efficient Video Large Language Models

Xiaohu Huang, Hao Zhou, Kai Han

机构 * The University of Hong Kong（香港大学） ； Baidu Inc.（百度公司） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Efficient Video Large Language Models

InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Models

Cong Wei, Yujie Zhong, Haoxian Tan, Yingsen Zeng, Yong Liu, Zheng Zhao, Yujiu Yang

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院） ； Meituan Inc.（美团公司） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

ShotVL: Human-Centric Highlight Frame Retrieval via Language Queries

Wangyu Xue, Chen Qian, Jiayi Wu, Yang Zhou, Wentao Liu, Ju Ren, Siming Fan, Yaoxue Zhang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition

Yulin Wang, Haoji Zhang, Yang Yue, Shiji Song, Chao Deng, Junlan Feng, Gao Huang

机构 * Tsinghua University（清华大学） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） ； China Mobile Research Institute（中国移动研究院） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted by IEEE TPAMI. Journal version of arXiv:2105.03245 (AdaFocusV1, ICCV 2021 Oral), arXiv:2112.14238 (AdaFocusV2, CVPR 2022), and arXiv:2209.13465 (AdaFocusV3, ECCV 2022). Code and pre-trained models: https://github.com/LeapLabTHU/Uni-AdaFocus

Dynamics Based Neural Encoding with Inter-Intra Region Connectivity

Mai Gamal, Mohamed Rashad, Eman Ehab, Seif Eldawlatly, Mennatullah Siam

机构 * German University in Cairo（开罗德国大学） ； Ain Shams University（艾因夏姆斯大学） ； Nile University（尼罗河大学） ； American University in Cairo（开罗美国大学） ； University of British Columbia（不列颠哥伦比亚大学） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Title change

EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videos

Ryo Fujii, Masashi Hatano, Hideo Saito, Hiroki Kajita

机构 * Keio University（庆应义塾大学） ； Keio University School of Medicine（庆应义塾大学医学院） 纠错

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Early accepted by MICCAI 2024

Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body

Zeqing Wang, Qingyang Ma, Wentao Wan, Haojie Li, Keze Wang, Yonghong Tian

专题命中 视频理解 ：text-to-video(abstract)；分类 cs.CV

Comments 16 pages, 14 figures

Extending Video Masked Autoencoders to 128 frames

Nitesh Bharadwaj Gundavarapu, Luke Friedman, Raghav Goyal, Chaitra Hegde, Eirikur Agustsson, Sagar M. Waghmare, Mikhail Sirotenko, Ming-Hsuan Yang, Tobias Weyand, Boqing Gong, Leonid Sigal

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 10.5 pages of main paper, 25 pages total, 4 figures and 10 tables. To appear in NeurIPS'24

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models

Tingyu Qu, Mingxiao Li, Tinne Tuytelaars, Marie-Francine Moens

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments work in progress

Dense Connector for MLLMs

Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, Jingdong Wang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 27 pages, NeurIPS 2024

Journal ref NeurIPS 2024

Snakes and Ladders: Two Steps Up for VideoMamba

Hui Lu, Albert Ali Salah, Ronald Poppe

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments New updated experiment results

Video RWKV:Video Action Recognition Based RWKV

Zhuowen Yin, Chengru Li, Xingbo Dong

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

BehAVE: Behaviour Alignment of Video Game Encodings

Nemanja Rašajski, Chintan Trivedi, Konstantinos Makantasis, Antonios Liapis, Georgios N. Yannakakis

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

LLaVA-OneVision: Easy Visual Task Transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Project Homepage: https://llava-vl.github.io/blog/2024-08-05-llava-onevision/

Telling Stories for Common Sense Zero-Shot Action Recognition

Shreyank N Gowda, Laura Sevilla-Lara

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted in ACCV 2024!

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs

Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, Yunhe Wang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Tech report

ViFi-ReID: A Two-Stream Vision-WiFi Multimodal Approach for Person Re-identification

Chen Mao, Chong Tan, Jingqi Hu, Min Zheng

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Exploring Efficient Foundational Multi-modal Models for Video Summarization

Karan Samel, Apoorva Beedu, Nitish Sontakke, Irfan Essa

专题命中 视频理解 ：video language model(abstract)；分类 cs.CV

Comments 11 pages, 4 figures