视频大模型

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, João Carreira, Olivier J. Hénaff

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

OV-HHIR: Open Vocabulary Human Interaction Recognition Using Cross-modal Integration of Large Language Models

Lala Shakti Swarup Ray, Bo Zhou, Sungho Suh, Paul Lukowicz

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted in IEEE ICASSP 2025

Detection-Fusion for Knowledge Graph Extraction from Videos

Taniya Das, Louis Mahon, Thomas Lukasiewicz

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 12 pages, To be submitted to a conference

Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy

Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Bhargava Kumar, Amit Agarwal, Ishan Banerjee, Srikant Panda, Tejaswini Kumar

专题命中 视频理解 ：text-to-video(abstract)；分类 cs.CV

Video Domain Incremental Learning for Human Action Recognition in Home Environments

Yuanda Hu, Xing Liu, Meiying Li, Yate Ge, Xiaohua Sun, Weiwei Guo

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

PruneVid: Visual Token Pruning for Efficient Video Large Language Models

Xiaohu Huang, Hao Zhou, Kai Han

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Efficient Video Large Language Models

InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Models

Cong Wei, Yujie Zhong, Haoxian Tan, Yingsen Zeng, Yong Liu, Zheng Zhao, Yujiu Yang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

ShotVL: Human-Centric Highlight Frame Retrieval via Language Queries

Wangyu Xue, Chen Qian, Jiayi Wu, Yang Zhou, Wentao Liu, Ju Ren, Siming Fan, Yaoxue Zhang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition

Yulin Wang, Haoji Zhang, Yang Yue, Shiji Song, Chao Deng, Junlan Feng, Gao Huang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted by IEEE TPAMI. Journal version of arXiv:2105.03245 (AdaFocusV1, ICCV 2021 Oral), arXiv:2112.14238 (AdaFocusV2, CVPR 2022), and arXiv:2209.13465 (AdaFocusV3, ECCV 2022). Code and pre-trained models: https://github.com/LeapLabTHU/Uni-AdaFocus

Dynamics Based Neural Encoding with Inter-Intra Region Connectivity

Mai Gamal, Mohamed Rashad, Eman Ehab, Seif Eldawlatly, Mennatullah Siam

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Title change

EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videos

Ryo Fujii, Masashi Hatano, Hideo Saito, Hiroki Kajita

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Early accepted by MICCAI 2024

Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body

Zeqing Wang, Qingyang Ma, Wentao Wan, Haojie Li, Keze Wang, Yonghong Tian

专题命中 视频理解 ：text-to-video(abstract)；分类 cs.CV

Comments 16 pages, 14 figures

Extending Video Masked Autoencoders to 128 frames

Nitesh Bharadwaj Gundavarapu, Luke Friedman, Raghav Goyal, Chaitra Hegde, Eirikur Agustsson, Sagar M. Waghmare, Mikhail Sirotenko, Ming-Hsuan Yang, Tobias Weyand, Boqing Gong, Leonid Sigal

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 10.5 pages of main paper, 25 pages total, 4 figures and 10 tables. To appear in NeurIPS'24

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models

Tingyu Qu, Mingxiao Li, Tinne Tuytelaars, Marie-Francine Moens

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments work in progress

Dense Connector for MLLMs

Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, Jingdong Wang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 27 pages, NeurIPS 2024

Journal ref NeurIPS 2024

Snakes and Ladders: Two Steps Up for VideoMamba

Hui Lu, Albert Ali Salah, Ronald Poppe

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments New updated experiment results

Video RWKV:Video Action Recognition Based RWKV

Zhuowen Yin, Chengru Li, Xingbo Dong

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

BehAVE: Behaviour Alignment of Video Game Encodings

Nemanja Rašajski, Chintan Trivedi, Konstantinos Makantasis, Antonios Liapis, Georgios N. Yannakakis

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

LLaVA-OneVision: Easy Visual Task Transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Project Homepage: https://llava-vl.github.io/blog/2024-08-05-llava-onevision/

Telling Stories for Common Sense Zero-Shot Action Recognition

Shreyank N Gowda, Laura Sevilla-Lara

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Accepted in ACCV 2024!

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs

Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, Yunhe Wang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Tech report

ViFi-ReID: A Two-Stream Vision-WiFi Multimodal Approach for Person Re-identification

Chen Mao, Chong Tan, Jingqi Hu, Min Zheng

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Exploring Efficient Foundational Multi-modal Models for Video Summarization

Karan Samel, Apoorva Beedu, Nitish Sontakke, Irfan Essa

专题命中 视频理解 ：video language model(abstract)；分类 cs.CV

Comments 11 pages, 4 figures

Realizing Video Summarization from the Path of Language-based Semantic Understanding

Kuan-Chen Mu, Zhi-Yi Chin, Wei-Chen Chiu

专题命中 视频理解 ：video generation(abstract)；分类 cs.CV

Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties

Keunwoo Peter Yu, Zheyuan Zhang, Fengyuan Hu, Shane Storks, Joyce Chai

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 16 pages, LaTeX; Accepted to EMNLP 2024 Main

Unveiling the Invisible: Captioning Videos with Metaphors

Abisek Rajakumar Kalarani, Pushpak Bhattacharyya, Sumit Shekhar

专题命中 视频理解 ：video language model(abstract)；分类 cs.CV

OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding

Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, Shuicheng Yan

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments NeurIPS-2024. Project page: https://lxtgh.github.io/project/omg_llava/

MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning

Haotian Zhang, Mingfei Gao, Zhe Gan, Philipp Dufter, Nina Wenzel, Forrest Huang, Dhruti Shah, Xianzhi Du, Bowen Zhang, Yanghao Li, Sam Dodge, Keen You, Zhen Yang, Aleksei Timofeev, Mingze Xu, Hong-You Chen, Jean-Philippe Fauconnier, Zhengfeng Lai, Haoxuan You, Zirui Wang, Afshin Dehghan, Peter Grasch, Yinfei Yang

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP

Zeliang Zhang, Zhuo Liu, Mingqian Feng, Chenliang Xu

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments Short paper. Accepted by the Findings of EMNLP 2024

Interpretable Action Recognition on Hard to Classify Actions

Anastasia Anichenko, Frank Guerin, Andrew Gilbert

专题命中 视频理解 ：video understanding(abstract)；分类 cs.CV

Comments 5 pages, This manuscript has been accepted at the Human-inspired Computer Vision (HCV) ECCV 2024 Workshop. arXiv admin note: text overlap with arXiv:2107.05319