多模态大模型

GAF-FusionNet: Multimodal ECG Analysis via Gramian Angular Fields and Split Attention

Jiahao Qin, Feng Liu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 1 figure, accepted by ICONIP 2024

VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video Prompting

Muhammet Furkan Ilaslan, Ali Koksal, Kevin Qinhong Lin, Burak Satar, Mike Zheng Shou, Qianli Xu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted for The 39th Annual AAAI Conference on Artificial Intelligence 2025 in Main Track, 19 pages, 24 figures

Apollo: An Exploration of Video Understanding in Large Multimodal Models

Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, Serena Yeung-Levy, Xide Xia

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments https://apollo-lmms.github.io

From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding

Heqing Zou, Tianze Luo, Guiyang Xie, Victor, Zhang, Fengmao Lv, Guangcong Wang, Junyang Chen, Zhuochen Wang, Hansheng Zhang, Huaijian Zhang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 11 pages

MEGL: Multimodal Explanation-Guided Learning

Yifei Zhang, Tianxu Jiang, Bo Pan, Jingyu Wang, Guangji Bai, Liang Zhao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer

Lu Zhang, Tiancheng Zhao, Heting Ying, Yibo Ma, Kyusong Lee

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL

A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal

Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun

专题命中 视频多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments This work has been submitted to the IEEE for possible publication

Multimodal Flare Forecasting with Deep Learning

Grégoire Francisco, Sabrina Guastavino, Teresa Barata, João Fernandes, Dario Del Moro

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Utilizing Large Language Models for Event Deconstruction to Enhance Multimodal Aspect-Based Sentiment Analysis

Xiaoyong Huang, Heli Sun, Qunshu Gao, Wenjie Huang, Ruichen Cao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering

Ting Yu, Kunhao Fu, Jian Zhang, Qingming Huang, Jun Yu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Transactions on Image Processing

Journal ref Transactions on Image Processing, vol. 33, pp. 3115-3129, 2024

Exploring Efficient Foundational Multi-modal Models for Video Summarization

Karan Samel, Apoorva Beedu, Nitish Sontakke, Irfan Essa

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 4 figures

Grounding Partially-Defined Events in Multimodal Data

Kate Sanders, Reno Kriz, David Etter, Hannah Recknor, Alexander Martin, Cameron Carpenter, Jingyang Lin, Benjamin Van Durme

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Preprint; 9 pages; 2024 EMNLP Findings

Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge

Yuxuan Wang, Yueqian Wang, Pengfei Wu, Jianxin Liang, Dongyan Zhao, Yang Liu, Zilong Zheng

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments To appear at EMNLP 2024

Neuromorphic Drone Detection: an Event-RGB Multimodal Approach

Gabriele Magrini, Federico Becattini, Pietro Pala, Alberto Del Bimbo, Antonio Porta

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at NeVi Workshop at ECCV24

MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction

Yan Feng, Alexander Carballo, Keisuke Fujii, Robin Karlsson, Ming Ding, Kazuya Takeda

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Learning Video Context as Interleaved Multimodal Sequences

Kevin Qinghong Lin, Pengchuan Zhang, Difei Gao, Xide Xia, Joya Chen, Ziteng Gao, Jinheng Xie, Xuhong Xiao, Mike Zheng Shou

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted by ECCV 2024

Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models

Bin Fu, Qiyang Wan, Jialin Li, Ruiping Wang, Xilin Chen

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 39 pages, 28 figures, 4 tables. Accepted at The 35th British Machine Vision Conference (BMVC 2024). Project page at https://fubin29.github.io/Blocks-as-Probes/

DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognition

Wei Ai, Yuntao Shou, Tao Meng, Nan Yin, Keqin Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 7 figures

SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentation

Qi Liu, Xinchen Liu, Kun Liu, Xiaoyan Gu, Wu Liu

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning

Jiajie Li, Garrett Skinner, Gene Yang, Brian R Quaranto, Steven D Schwaitzberg, Peter C W Kim, Jinjun Xiong

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MM-Forecast: A Multimodal Approach to Temporal Event Forecasting with Large Language Models

Haoxuan Li, Zhengmao Yang, Yunshan Ma, Yi Bin, Yang Yang, Tat-Seng Chua

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models

Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, Chunyuan Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://llava-vl.github.io/blog/2024-06-16-llava-next-interleave/

Temporal Grounding of Activities using Multimodal Large Language Models

Young Chol Song

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding

Tiancheng Zhao, Qianqian Zhang, Kyusong Lee, Peng Liu, Lu Zhang, Chunxin Fang, Jiajia Liao, Kelei Jiang, Yibo Ma, Ruochen Xu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 14 pages

VCoME: Verbal Video Composition with Multimodal Editing Effects

Weibo Gong, Xiaojie Jin, Xin Li, Dongliang He, Xinglong Wu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference

Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, Li Yuan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition

Xinzhe Ni, Yong Liu, Hao Wen, Yatai Ji, Jing Xiao, Yujiu Yang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICMR 2024 (oral)

Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection

Shengyang Sun, Xiaojin Gong

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICME 2024

Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samples

Kuofeng Gao, Jindong Gu, Yang Bai, Shu-Tao Xia, Philip Torr, Wei Liu, Zhifeng Li

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2401.11170

Multimodal Emotion Recognition by Fusing Video Semantic in MOOC Learning Scenarios

Yuan Zhang, Xiaomei Tao, Hanxu Ai, Tao Chen, Yanling Gan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM