多模态大模型

ReasVQA: Advancing VideoQA with Imperfect Reasoning Process

Jianxin Liang, Xiaojun Meng, Huishuai Zhang, Yueqian Wang, Jiansheng Wei, Dongyan Zhao

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to main conference at NAACL 2025; 8 pages;

Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2025. Code is available at https://github.com/hmxiong/StreamChat

Neptune: The Long Orbit to Benchmarking Long Video Understanding

Arsha Nagrani, Mingda Zhang, Ramin Mehran, Rachel Hornung, Nitesh Bharadwaj Gundavarapu, Nilpa Jha, Austin Myers, Xingyi Zhou, Boqing Gong, Cordelia Schmid, Mikhail Sirotenko, Yukun Zhu, Tobias Weyand

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness

Jiaxing Zhao, Boyuan Sun, Xiang Chen, Xihan Wei

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Text-to-Game Engine for UGC-Based Role-Playing Games

Lei Zhang, Xuezheng Peng, Shuyi Yang, Feiyang Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 9 figures

VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition

Lan Chen, Haoxiang Yang, Pengpeng Shao, Haoyu Song, Xiao Wang, Zhicheng Zhao, Yaowei Wang, Yonghong Tian

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

Modality-Aware Shot Relating and Comparing for Video Scene Detection

Jiawei Tan, Hongxing Wang, Kang Dang, Jiaxin Li, Zhilong Ou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localization

Jeongseok Hyun, Su Ho Han, Hyolim Kang, Joon-Young Lee, Seon Joo Kim

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to WACV 2025

Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation

Xiaoqi An, Lin Zhao, Chen Gong, Jun Li, Jian Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2025

Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval

Jian Xiao, Zhenzhen Hu, Jia Li, Richang Hong

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Visual Object Tracking across Diverse Data Modalities: A Review

Mengmeng Wang, Teli Ma, Shuo Xin, Xiaojun Hou, Jiazheng Xing, Guang Dai, Jingdong Wang, Yong Liu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Code and models are available at https://github.com/NVlabs/VILA/tree/main/longvila

Doe-1: Closed-Loop Autonomous Driving with Large World Model

Wenzhao Zheng, Zetian Xia, Yuanhui Huang, Sicheng Zuo, Jie Zhou, Jiwen Lu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Code is available at: https://github.com/wzzheng/Doe

Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering

Sai Bhargav Rongali, Mohamad Hassan N C, Ankit Jha, Neha Bhargava, Saurabh Prasad, Biplab Banerjee

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Journal ref WACV2025

LinVT: Empower Your Image-level Large Language Model to Understand Videos

Lishuai Gao, Yujie Zhong, Yingsen Zeng, Haoxian Tan, Dengjie Li, Zheng Zhao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

What's the Move? Hybrid Imitation Learning via Salient Points

Priya Sundaresan, Hengyuan Hu, Quan Vuong, Jeannette Bohg, Dorsa Sadigh

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization

Yuliang Liu, Mingxin Huang, Hao Yan, Linger Deng, Weijia Wu, Hao Lu, Chunhua Shen, Lianwen Jin, Xiang Bai

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding

Kangsan Kim, Geon Park, Youngwan Lee, Woongyeong Yeo, Sung Ju Hwang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Long-Term Ad Memorability: Understanding & Generating Memorable Ads

Harini SI, Somesh Singh, Yaman K Singla, Aanisha Bhattacharyya, Veeky Baths, Changyou Chen, Rajiv Ratn Shah, Balaji Krishnamurthy

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Published in WACV-2025

Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing

Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://ivy-lvlm.github.io/Video-MA2MBA/

On Moving Object Segmentation from Monocular Video with Transformers

Christian Homeyer, Christoph Schnörr

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments WICCV2023

Journal ref Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 (880--891)

TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability

Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Generative Timelines for Instructed Visual Assembly

Alejandro Pardo, Jui-Hsien Wang, Bernard Ghanem, Josef Sivic, Bryan Russell, Fabian Caba Heilbron

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

HourVideo: 1-Hour Video-Language Understanding

Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, Li Fei-Fei

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Track; 28 pages

StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, Maosong Sun

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Simple and Effective Temporal Grounding Pipeline for Basketball Broadcast Footage

Levi Harris

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities

Md Farhan Ishmam, Md Sakib Hossain Shovon, M. F. Mridha, Nilanjan Dey

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation

Kyungjin Seo, Junghoon Seo, Hanseok Jeong, Sangpil Kim, Sang Ho Yoon

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024. Project Page Link: https://pimforce.hcitech.org/

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, Lidong Bing

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ZC, SL, HZ, YX, and XL contributed equally to this project. Code: https://github.com/DAMO-NLP-SG/VideoLLaMA2

Transformer for Object Re-Identification: A Survey

Mang Ye, Shuoyi Chen, Chenyue Li, Wei-Shi Zheng, David Crandall, Bo Du

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by International Journal of Computer Vision (IJCV) in October 2024