多模态大模型

Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework

Satyapriya Krishna, Ninareh Mehrabi, Abhinav Mohanty, Matteo Memelli, Vincent Ponzo, Payal Motwani, Rahul Gupta

专题命中 视频多模态 ：multimodal(abstract)；multimodal foundation model(abstract)

Label-Free Long-Horizon 3D UAV Trajectory Prediction via Motion-Aligned RGB and Event Cues

Hanfang Liang, Shenghai Yuan, Fen Liu, Yizhuo Yang, Bing Wang, Zhuyu Huang, Chenyang Shi, Jing Jin

机构 * Jianghan University（江汉大学） ； Nanyang Technological University（南洋理工大学） ； Beihang University (BUAA)（北京航空航天大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)

LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs

Boyuan Sun, Jiaxing Zhao, Xihan Wei, Qibin Hou

机构 * VCIP, School of Computer Science, Nankai University（南开大学计算机学院） ； Tongyi Lab, Alibaba Group（阿里巴巴集团 Tongyi 实验室） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 21 pages, 4 figures, 7 tables

FixDrive: Automatically Repairing Autonomous Vehicle Driving Behaviour for $0.08 per Violation

Yang Sun, Christopher M. Poskitt, Kun Wang, Jun Sun

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)

Comments Accepted by the 47th IEEE/ACM International Conference on Software Engineering (ICSE 2025)

Journal ref Proc. ICSE'25, pages 1921-1933. IEEE, 2025

xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs

Michael S. Ryoo, Honglu Zhou, Shrikant Kendre, Can Qin, Le Xue, Manli Shu, Jongwoo Park, Kanchana Ranasinghe, Silvio Savarese, Ran Xu, Caiming Xiong, Juan Carlos Niebles

机构 * Salesforce AI Research（Salesforce AI研究院） ； Stony Brook University（石溪大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding

Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, Liqiang Nie

机构 * Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）） ； Huawei Technologies Co., Ltd.（华为技术有限公司） ； Shandong University（山东大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning

Qiuchen Wang, Ruixue Ding, Yu Zeng, Zehui Chen, Lin Chen, Shihang Wang, Pengjun Xie, Fei Huang, Feng Zhao

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Unsupervised Transcript-assisted Video Summarization and Highlight Detection

Spyros Barbakos, Charalampos Antoniadis, Gerasimos Potamianos, Gianluca Setti

机构 * University of Thessaly（塞萨洛尼基大学） ； King Abdullah University of Science and Technology（卡布斯大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

VideoRAG: Retrieval-Augmented Generation over Video Corpus

Soyeong Jeong, Kangsan Kim, Jinheon Baek, Sung Ju Hwang

机构 * KAIST（韩国科学技术院） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL Findings 2025

Fostering Video Reasoning via Next-Event Prediction

Haonan Wang, Hongfu Liu, Xiangyan Liu, Chao Du, Kenji Kawaguchi, Ye Wang, Tianyu Pang

机构 * National University of Singapore（新加坡国立大学） ； Sea AI Lab（Sea AI实验室） 纠错

专题命中 视频多模态 ：MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

LifeIR at the NTCIR-18 Lifelog-6 Task

Jiahan Chen, Da Li, Keping Bi

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)

Question-Answering Dense Video Events

Hangyu Qin, Junbin Xiao, Angela Yao

机构 * National University of Singapore（新加坡国立大学） 纠错

专题命中 视频多模态 ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to SIGIR'25

ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Data

Chengsen Wang, Qi Qi, Zhongwen Rao, Lujia Pan, Jingyu Wang, Jianxin Liao

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)

VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation

Chaofan Zhang, Peng Hao, Xiaoge Cao, Xiaoshuai Hao, Shaowei Cui, Shuo Wang

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院） ； Samsung R&D Institute China–Beijing（三星中国北京研发中心） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)

VidCtx: Context-aware Video Question Answering with Image Models

Andreas Goulas, Vasileios Mezaris, Ioannis Patras

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted in IEEE ICME 2025. This is the authors' accepted version

Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1

Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Lu Qiu, Ying Shan, Xihui Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical Report (In Progress); Code released at: https://github.com/TencentARC/SEED-Bench-R1

Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video

Bin Li, Yixuan Weng, Bin Sun, Shutao Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 8 pages, 6 figures, 3 tables

Shushing! Let's Imagine an Authentic Speech from the Silent Video

Jiaxin Ye, Hongming Shan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、eess.AS

Comments Project Page: https://imagintalk.github.io

Valley: Video Assistant with Large Language model Enhanced abilitY

Ruipu Luo, Ziwang Zhao, Min Yang, Zheming Yang, Minghui Qiu, Tao Wang, Zhongyu Wei, Yanhao Wang, Cen Chen

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Can Large Language Models Grasp Concepts in Visual Content? A Case Study on YouTube Shorts about Depression

Jiaying "Lizzy" Liu, Yiheng Su, Praneel Seth

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)

Comments 11 pages

Journal ref CHI Conference on Human Factors in Computing Systems (CHI EA 2025)

PreMind: Multi-Agent Video Understanding for Advanced Indexing of Presentation-style Videos

Kangda Wei, Zhengyu Zhou, Bingqing Wang, Jun Araki, Lukas Lange, Ruihong Huang, Zhe Feng

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching

Yongqi Wang, Wenxiang Guo, Rongjie Huang, Jiawei Huang, Zehan Wang, Fuming You, Ruiqi Li, Zhou Zhao

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM、eess.AS

Comments accepted by NeurIPS 2024

VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection

Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, Si Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 14 pages, 14 figures

LLaVA-OneVision: Easy Visual Task Transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Homepage: https://llava-vl.github.io/blog/2024-08-05-llava-onevision/

Large Body Language Models

Saif Punjwani, Larry Heck

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering

Chuyi Shang, Amos You, Sanjay Subramanian, Trevor Darrell, Roei Herzig

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2024 (Main)

Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos

Jianrui Zhang, Mu Cai, Yong Jae Lee

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Page: https://vinoground.github.io

NL-Eye: Abductive NLI for Images

Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding

Xiao Wang, Jianlong Wu, Zijia Lin, Fuzheng Zhang, Di Zhang, Liqiang Nie

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Under peer review

Top-down Activity Representation Learning for Video Question Answering

Yanan Wang, Shuichiro Haruta, Donghuo Zeng, Julio Vizcarra, Mori Kurokawa

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments presented at MIRU2024