Long Video Understanding with Learnable Retrieval in Video-Language Models
机构 * School of Information Science and Technology, University of Science and Technology of China(信息科学与技术学院,中国科学技术大学) ; Microsoft Research Asia(微软亚洲研究院)
专题命中 视频理解 :video understanding(title,abstract);video-language(title,abstract);long video(title,abstract);video reasoning(abstract)
AI总结 针对长视频理解中LLM计算开销大和噪声干扰问题,提出可学习检索的视频语言模型R-VLM,通过软匹配损失端到端训练轻量MLP,选择最相关的K个视频片段作为上下文,减少令牌数并提升零样本问答性能。
Comments Accepted by IEEE Transactions on Multimedia (TMM)