多模态大模型

A Survey on Safe Multi-Modal Learning System

Tianyi Zhao, Liangliang Zhang, Yao Ma, Lu Cheng

专题命中 多模态训练与对齐 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Dense Multimodal Alignment for Open-Vocabulary 3D Scene Understanding

Ruihuang Li, Zhengqiang Zhang, Chenhang He, Zhiyuan Ma, Vishal M. Patel, Lei Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification

Yuan Zhang, Yaolei Qi, Xiaoming Qi, Yongyue Wei, Guanyu Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by MICCAI2024

MMA-UNet: A Multi-Modal Asymmetric UNet Architecture for Infrared and Visible Image Fusion

Jingxue Huang, Xilai Li, Tianshu Tan, Xiaosong Li, Tao Ye

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Multi-Modal Prototypes for Open-World Semantic Segmentation

Yuhuan Yang, Chaofan Ma, Chen Ju, Fei Zhang, Jiangchao Yao, Ya Zhang, Yanfeng Wang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments accepted in IJCV

TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data

Siyi Du, Shaoming Zheng, Yinsong Wang, Wenjia Bai, Declan P. O'Regan, Chen Qin

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 28 pages (including 9 pages of supplementary materials), accepted by ECCV 2024

LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition

Teng Wang, Lingquan Meng, Lei Cheng, Changyin Sun

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Enhance the Robustness of Text-Centric Multimodal Alignments

Ting-Yu Yen, Yun-Da Tsai, Keng-Te Liao, Shou-De Lin

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

MMR-Mamba: Multi-Modal MRI Reconstruction with Mamba and Spatial-Frequency Information Fusion

Jing Zou, Lanqing Liu, Qi Chen, Shujun Wang, Zhanli Hu, Xiaohan Xing, Jing Qin

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments 10 pages, 5 figure

Fine-grained Context and Multi-modal Alignment for Freehand 3D Ultrasound Reconstruction

Zhongnuo Yan, Xin Yang, Mingyuan Luo, Jiongquan Chen, Rusi Chen, Lian Liu, Dong Ni

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at MICCAI 2024. This is the submitted manuscript and the preprint has not undergone peer review (when applicable) or any post-submission improvements or corrections

Autonomous Soundscape Augmentation with Multimodal Fusion of Visual and Participant-linked Inputs

Kenneth Ooi, Karn N. Watcharasupat, Bhan Lam, Zhen-Ting Ong, Woon-Seng Gan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 eess.AS

Comments [v1] 5 pages, 2 figures. Submitted to 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. [v2] 5 pages, 2 figures. Fixed incorrect author list in citation #30

Journal ref ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Jun. 2023, pp. 1-5

MMBee: Live Streaming Gift-Sending Recommendations via Multi-Modal Fusion and Behaviour Expansion

Jiaxin Deng, Shiyao Wang, Yuchen Wang, Jiansong Qi, Liqin Zhao, Guorui Zhou, Gaofeng Meng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted at KDD 2024

MDF: A Dynamic Fusion Model for Multi-modal Fake News Detection

Hongzhen Lv, Wenzhong Yang, Fuyuan Wei, Jiaren Peng, Haokun Geng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.MM

Multimodal Learning With Intraoperative CBCT & Variably Aligned Preoperative CT Data To Improve Segmentation

Maximilian E. Tschuchnig, Philipp Steininger, Michael Gadermayr

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Submitted to SASHIMI2024 (MICCAI workshop)

Multimodal Prototyping for cancer survival prediction

Andrew H. Song, Richard J. Chen, Guillaume Jaume, Anurag J. Vaidya, Alexander S. Baras, Faisal Mahmood

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments ICML 2024

MMGPL: Multimodal Medical Data Analysis with Graph Prompt Learning

Liang Peng, Songyue Cai, Zongqian Wu, Huifang Shang, Xiaofeng Zhu, Xiaoxiao Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

MolFusion: Multimodal Fusion Learning for Molecular Representations via Multi-granularity Views

Muzhen Cai, Sendong Zhao, Haochun Wang, Yanrui Du, Zewen Qiang, Bing Qin, Ting Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 8 pages, 5 figures

Multimodal Physiological Signals Representation Learning via Multiscale Contrasting for Depression Recognition

Kai Shao, Rui Wang, Yixue Hao, Long Hu, Min Chen, Hans Arno Jacobsen

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images

Songhan Jiang, Zhengyu Gan, Linghan Cai, Yifeng Wang, Yongbing Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Multi-Modal Vision Transformers for Crop Mapping from Satellite Image Time Series

Theresa Follath, David Mickisch, Jan Hemmerling, Stefan Erasmi, Marcel Schwieder, Begüm Demir

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments 5 pages, 2 figures, 1 table. Accepted at IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2024. Our code is available at https://git.tu-berlin.de/rsim/mmtsvit

DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation

Yueru Luo, Shuguang Cui, Zhen Li

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at ICLR2024

Multimodal Deformable Image Registration for Long-COVID Analysis Based on Progressive Alignment and Multi-perspective Loss

Jiahua Li, James T. Grist, Fergus V. Gleeson, Bartłomiej W. Papież

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Converging Dimensions: Information Extraction and Summarization through Multisource, Multimodal, and Multilingual Fusion

Pranav Janjani, Mayank Palan, Sarvesh Shirude, Ninad Shegokar, Sunny Kumar, Faruk Kazi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 11 pages, 3 figures

M2CURL: Sample-Efficient Multimodal Reinforcement Learning via Self-Supervised Representation Learning for Robotic Manipulation

Fotios Lygerakis, Vedant Dave, Elmar Rueckert

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Project website: https://sites.google.com/view/M2CURL/home

FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare Prediction

Muhao Xu, Zhenfeng Zhu, Youru Li, Shuai Zheng, Yawei Zhao, Kunlun He, Yao Zhao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by KDD 2024 (Research Track)

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang, Jonghyun Choi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments ACL 2024

AstroCLIP: A Cross-Modal Foundation Model for Galaxies

Liam Parker, Francois Lanusse, Siavash Golkar, Leopoldo Sarra, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Geraud Krawezik, Michael McCabe, Ruben Ohana, Mariel Pettee, Bruno Regaldo-Saint Blancard, Tiberiu Tesileanu, Kyunghyun Cho, Shirley Ho

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.AI

Comments 18 pages, accepted in Monthly Notices of the Royal Astronomical Society, Presented at the NeurIPS 2023 AI4Science Workshop

Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling

Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, Gustavo Carneiro

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref CVPR2023

GAME-ON: Graph Attention Network based Multimodal Fusion for Fake News Detection

Mudit Dhawan, Shakshi Sharma, Aditya Kadam, Rajesh Sharma, Ponnurangam Kumaraguru

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.MM

Comments Accepted at SNAM 2024

Multimodal Approach for Harmonized System Code Prediction

Otmane Amel, Sedrick Stassin, Sidi Ahmed Mahmoudi, Xavier Siebert

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted for poster presentation at ESANN 2023

Journal ref Proceedings of the 31th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2023), pp. 181-186