多模态大模型

AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework

Run Shao, Cheng Yang, Qiujun Li, Qing Zhu, Yongjun Zhang, YanSheng Li, Yu Liu, Yong Tang, Dapeng Liu, Shizhong Yang, Haifeng Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 19 pages, 19 tables, 3 figures

Journal ref IEEE Transactions on Geoscience and Remote Sensing. 2025

MVP: Multimodal Emotion Recognition based on Video and Physiological Signals

Valeriya Strizhkova, Hadi Kachmar, Hava Chaptoukaev, Raphael Kalandadze, Natia Kukhilava, Tatia Tsmindashvili, Nibras Abo-Alzahab, Maria A. Zuluaga, Michal Balazia, Antitza Dantcheva, François Brémond, Laura Ferrari

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Preprint. Final paper accepted at Affective Behavior Analysis in-the-Wild (ABAW) at IEEE/CVF European Conference on Computer Vision (ECCV), Milan, September, 2024. 17 pages

Multimodal Large Models Are Effective Action Anticipators

Binglu Wang, Yao Tian, Shunzhou Wang, Le Yang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Multimodal joint prediction of traffic spatial-temporal data with graph sparse attention mechanism and bidirectional temporal convolutional network

Dongran Zhang, Jiangnan Yan, Kemal Polat, Adi Alhudhaif, Jun Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

M3-CVC: Controllable Video Compression with Multimodal Generative Models

Rui Wan, Qi Zheng, Yibo Fan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to ICASSP 2025

MANGO: Multimodal Acuity traNsformer for intelliGent ICU Outcomes

Jiaqing Zhang, Miguel Contreras, Sabyasachi Bandyopadhyay, Andrea Davidson, Jessica Sena, Yuanfang Ren, Ziyuan Guan, Tezcan Ozrazgat-Baslanti, Tyler J. Loftus, Subhash Nerella, Azra Bihorac, Parisa Rashidi

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

A Multimodal Approach For Endoscopic VCE Image Classification Using BiomedCLIP-PubMedBERT

Nagarajan Ganapathy, Podakanti Satyajith Chary, Teja Venkata Ramana Kumar Pithani, Pavan Kavati, Arun Kumar S

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 11 Pages, 2 Figures, Capsule Vision 2024 Challenge

Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking

Xiantao Hu, Ying Tai, Xu Zhao, Chen Zhao, Zhenyu Zhang, Jun Li, Bineng Zhong, Jian Yang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

GROOT-2: Weakly Supervised Multi-Modal Instruction Following Agents

Shaofei Cai, Bowei Zhang, Zihao Wang, Haowei Lin, Xiaojian Ma, Anji Liu, Yitao Liang

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Learning Complex Non-Rigid Image Edits from Multimodal Conditioning

Nikolai Warner, Jack Kolb, Meera Hahn, Vighnesh Birodkar, Jonathan Huang, Irfan Essa

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

ICANet: A Method of Short Video Emotion Recognition Driven by Multimodal Data

Xuecheng Wu, Mengmeng Tian, Lanhang Zhai

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MTSA-SNN: A Multi-modal Time Series Analysis Model Based on Spiking Neural Network

Chengzhi Liu, Zheng Tao, Zihong Luo, Chenghao Liu

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 6 pages, 6 figures, published to International Conference on Computer Supported Cooperative Work in Design

Journal ref International Conference on Pattern Recognition 2024

Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model

Benlin Liu, Yuhao Dong, Yiqin Wang, Zixian Ma, Yansong Tang, Luming Tang, Yongming Rao, Wei-Chiu Ma, Ranjay Krishna

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments project page: https://coarse-correspondence.github.io

Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration

Wei Ji, Li Li, Zheqi Lv, Wenqiao Zhang, Mengze Li, Zhen Wan, Wenqiang Lei, Roger Zimmermann

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Efficient Large Multi-modal Models via Visual Context Compression

Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, Alan Yuille

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments NeurIPS 2024 Camera Ready; Code is available at https://github.com/Beckschen/LLaVolta

MMSummary: Multimodal Summary Generation for Fetal Ultrasound Video

Xiaoqing Guo, Qianhui Men, J. Alison Noble

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments MICCAI 2024

Multi-modal Motion Prediction using Temporal Ensembling with Learning-based Aggregation

Kai-Yin Hong, Chieh-Chih Wang, Wen-Chieh Lin

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), accepted by IROS2024

Multimodal Learning for Embryo Viability Prediction in Clinical IVF

Junsik Kim, Zhiyi Shi, Davin Jeong, Johannes Knittel, Helen Y. Yang, Yonghyun Song, Wanhua Li, Yicong Li, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to MICCAI 2024

RoMemes: A multimodal meme corpus for the Romanian language

Vasile Păiş, Sara Niţă, Alexandru-Iulius Jerpelea, Luca Pană, Eric Curea

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 12 pages, 7 tables, 1 figure, submitted to The 19th International Conference on Linguistic Resources and Tools for Natural Language Processing (ConsILR 2024)

Temporal-Enhanced Multimodal Transformer for Referring Multi-Object Tracking and Segmentation

Changcheng Xiao, Qiong Cao, Yujie Zhong, Xiang Zhang, Tao Wang, Canqun Yang, Long Lan

专题命中 视频多模态 ：multimodal(title)；cross-modal(abstract)；分类 cs.CV

TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement

Zhiwei Lin, Hongbo Jin, Yongtao Wang, Yufei Wei, Nan Dong

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by ECAI2024

GEM-VPC: A dual Graph-Enhanced Multimodal integration for Video Paragraph Captioning

Eileen Wang, Caren Han, Josiah Poon

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognition

Jiazheng Xing, Chao Xu, Mengmeng Wang, Guang Dai, Baigui Sun, Yong Liu, Jingdong Wang, Jian Zhao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MapsTP: HD Map Images Based Multimodal Trajectory Prediction for Automated Vehicles

Sushil Sharma, Arindam Das, Ganesh Sistu, Mark Halton, Ciarán Eising

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments This paper is a preprint of a paper submitted to the 26th Irish Machine Vision and Image Processing Conference (IMVIP 2024). If accepted, the copy of record will be available at IET Digital Library

Journal ref Proceedings of the Irish Machine Vision and Image Processing Conference 2024

X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation

Pinxue Guo, Wanyun Li, Hao Huang, Lingyi Hong, Xinyu Zhou, Zhaoyu Chen, Jinglun Li, Kaixun Jiang, Wei Zhang, Wenqiang Zhang

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments ACMMM'2024

GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting

Tao Liu, Runze Yuan, Yi'ang Ju, Xun Xu, Jiaqi Yang, Xiangting Meng, Xavier Lagorce, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

EventDance++: Language-guided Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition

Xu Zheng, Lin Wang

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2403.14082

Portrait Video Editing Empowered by Multimodal Generative Priors

Xuan Gao, Haiyao Xiao, Chenglai Zhong, Shimin Hu, Yudong Guo, Juyong Zhang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by SIGGRAPH Asia 2024. Project Page: https://ustc3dv.github.io/PortraitGen/

Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild

Nicolas Richet, Soufiane Belharbi, Haseeb Aslam, Meike Emilie Schadt, Manuela González-González, Gustave Cortal, Alessandro Lameiras Koerich, Marco Pedersoli, Alain Finkel, Simon Bacon, Eric Granger

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 14 pages, 3 figures, ECCVw 2024

Video Token Sparsification for Efficient Multimodal LLMs in Autonomous Driving

Yunsheng Ma, Amr Abdelraouf, Rohit Gupta, Ziran Wang, Kyungtae Han

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages, 3 figures, 4 tables