多模态大模型

Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition

Qianrui Zhou, Hua Xu, Hao Li, Hanlei Zhang, Xiaohan Zhang, Yifan Wang, Kai Gao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.MM

Comments Accepted by AAAI 2024 (Main Track, Long Paper)

Enhancing Multimodal Large Language Models with Multi-instance Visual Prompt Generator for Visual Representation Enrichment

Wenliang Zhong, Wenyi Wu, Qi Li, Rob Barton, Boxin Du, Shioulin Sam, Karim Bouyarmane, Ismail Tutar, Junzhou Huang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking

Zefeng Zhang, Jiawei Sheng, Chuang Zhang, Yunzhi Liang, Wenyuan Zhang, Siqi Wang, Tingwen Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Comments Findings of ACL 2024

Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection

Kunpeng Wang, Zhengzheng Tu, Chenglong Li, Cheng Zhang, Bin Luo

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by TCSVT 2024

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

David Restrepo, Chenwei Wu, Constanza Vásquez-Venegas, Luis Filipe Nakayama, Leo Anthony Celi, Diego M López

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 6 figures, 5 tables

RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision

Jinzhong Wang, Xuetao Tian, Shun Dai, Tao Zhuo, Haorui Zeng, Hongjuan Liu, Jiaqi Liu, Xiuwei Zhang, Yanning Zhang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Attention-based sequential recommendation system using multimodal data

Hyungtaik Oh, Wonkeun Jo, Dongil Kim

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 18 pages, 4 figures, preprinted

SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals

Rahul Thapa, Bryan He, Magnus Ruud Kjaer, Hyatt Moore, Gauri Ganjoo, Emmanuel Mignot, James Zou

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models

Yang Jiao, Shaoxiang Chen, Zequn Jie, Jingjing Chen, Lin Ma, Yu-Gang Jiang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Technical Report

Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning

Jiachen Li, Qiaozi Gao, Michael Johnston, Xiaofeng Gao, Xuehai He, Suhaila Shakiah, Hangjie Shi, Reza Ghanadan, William Yang Wang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by ICML 2024. Project page: https://midas-icml.github.io

PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification

Zhenwei Wang, Qiule Sun, Bingbing Zhang, Pengfei Wang, Jianxin Zhang, Qiang Zhang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions

Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, Hanwang Zhang, Yueting Zhuang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by ICLR 2024 (Spotlight)

Multimodal Object Detection via Probabilistic a priori Information Integration

Hafsa El Hafyani, Bastien Pasdeloup, Camille Yver, Pierre Romenteau

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation

Bingyu Li, Da Zhang, Zhiyuan Zhao, Junyu Gao, Xuelong Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction

Jingyi Pan, Zipeng Wang, Lin Wang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Robotics and Automation Letters (RA-L)

Journal ref IEEE Robotics and Automation Letters, Volume 9 Issue 6, 5687 - 5694, June 2024

Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastoma

Ahmed Gomaa, Yixing Huang, Amr Hagag, Charlotte Schmitter, Daniel Höfler, Thomas Weissmann, Katharina Breininger, Manuel Schmidt, Jenny Stritzelberger, Daniel Delev, Roland Coras, Arnd Dörfler, Oliver Schnell, Benjamin Frey, Udo S. Gaipl, Sabine Semrau, Christoph Bert, Rainer Fietkau, Florian Putz

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models

Hongzhan Lin, Zixin Chen, Ziyang Luo, Mingfei Cheng, Jing Ma, Guang Chen

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Comments ACL 2024

IHC Matters: Incorporating IHC analysis to H&E Whole Slide Image Analysis for Improved Cancer Grading via Two-stage Multimodal Bilinear Pooling Fusion

Jun Wang, Yu Mao, Yufei Cui, Nan Guan, Chun Jason Xue

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

FORESEE: Multimodal and Multi-view Representation Learning for Robust Prediction of Cancer Survival

Liangrui Pan, Yijun Peng, Yan Li, Yiyi Liang, Liwen Xu, Qingchun Liang, Shaoliang Peng

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Enhanced Multimodal Content Moderation of Children's Videos using Audiovisual Fusion

Syed Hammad Ahmed, Muhammad Junaid Khan, Gita Sukthankar

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 8 pages, 3 figures, Accepted at The 37th International FLAIRS Conference

UniBEV: Multi-modal 3D Object Detection with Uniform BEV Encoders for Robustness against Missing Sensor Modalities

Shiming Wang, Holger Caesar, Liangliang Nan, Julian F. P. Kooij

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Intelligent Vehicles Symposium (IV 2024), camera-ready. Code: https://github.com/tudelft-iv/UniBEV

Generic Multi-modal Representation Learning for Network Traffic Analysis

Luca Gioacchini, Idilio Drago, Marco Mellia, Zied Ben Houidi, Dario Rossi

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusion

Yuntao Shou, Tao Meng, Fuchen Zhang, Nan Yin, Keqin Li

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL

Comments 10 pages, 6 figures

AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models

Zhiqiang Tang, Haoyang Fang, Su Zhou, Taojiannan Yang, Zihan Zhong, Tony Hu, Katrin Kirchhoff, George Karypis

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted at AutoML 2024 Conference

HiH: A Multi-modal Hierarchy in Hierarchy Network for Unconstrained Gait Recognition

Lei Wang, Bo Liu, Yinchi Ma, Fangfang Liang, Nawei Guo

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Leveraging Intra-modal and Inter-modal Interaction for Multi-Modal Entity Alignment

Zhiwei Hu, Víctor Gutiérrez-Basulto, Zhiliang Xiang, Ru Li, Jeff Z. Pan

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding

Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments CVPR2024

Journal ref CVPR2024

Neuro-Inspired Hierarchical Multimodal Learning

Xiongye Xiao, Gengshuo Liu, Gaurav Gupta, Defu Cao, Shixuan Li, Yaxing Li, Tianqing Fang, Mingxi Cheng, Paul Bogdan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments I am requesting the withdrawal of this submission due to an inadvertent duplication. The paper was submitted twice under different IDs, which was not intentional. The other submission (arXiv:2404.09403) contains the most updated and comprehensive version of the paper, and I would like to retain that as the sole version on the platform

Multimodal sensor fusion for real-time location-dependent defect detection in laser-directed energy deposition

Lequn Chen, Xiling Yao, Wenhe Feng, Youxiang Chew, Seung Ki Moon

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 eess.AS

Comments 8 pages, 10 figures. This paper has been accepted to be published in the proceedings of IDETC-CIE 2023

PEMMA: Parameter-Efficient Multi-Modal Adaptation for Medical Image Segmentation

Nada Saadi, Numan Saeed, Mohammad Yaqub, Karthik Nandakumar

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV