多模态大模型

Multimodal Fusion Strategies for Mapping Biophysical Landscape Features

Lucia Gordon, Nico Lang, Catherine Ressijac, Andrew Davies

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 4 figures, ECCV 2024 Workshop in CV for Ecology

Multimodal 3D Fusion and In-Situ Learning for Spatially Aware AI

Chengyuan Xu, Radha Kumaran, Noah Stier, Kangyou Yu, Tobias Höllerer

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 6 figures, accepted to IEEE ISMAR 2024

Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification

Salma Hassan, Hamad Al Hammadi, Ibrahim Mohammed, Muhammad Haris Khan

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs

Mustafa Shukor, Matthieu Cord

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2024. Code: https://github.com/mshukor/ima-lmms. Project page: https://ima-lmms.github.io/

Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats

Kuanrong Liu, Siyuan Liang, Jiawei Liang, Pengwen Dai, Xiaochun Cao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Alternative Telescopic Displacement: An Efficient Multimodal Alignment Method

Jiahao Qin, Yitao Xu, Zong Lu, Xiaojun Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 1 figures

Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models

Venkat Karramreddy, Liam Mitchell

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 10 figures

A Novel Adaptive Fine-Tuning Algorithm for Multimodal Models: Self-Optimizing Classification and Selection of High-Quality Datasets in Remote Sensing

Yi Ren, Tianyi Zhang, Zhixiong Han, Weibin Li, Zhiyang Wang, Wenbo Ji, Chenhao Qin, Chenbin Liang, Licheng Jiao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task

Yanjie Li, Jingyi Liu, Weijun Li, Lina Yu, Min Wu, Wenqiang Li, Meilan Hao, Su Wei, Yusong Deng

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments The Information Fusion has accepted this paper

Learning Noise-Robust Joint Representation for Multimodal Emotion Recognition under Incomplete Data Scenarios

Qi Fan, Haolin Zuo, Rui Liu, Zheng Lian, Guanglai Gao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by MRAC@ACM Multimedia 2024

Multi-modal Relation Distillation for Unified 3D Representation Learning

Huiqun Wang, Yiping Bao, Panwang Pan, Zeming Li, Xiao Liu, Ruijie Yang, Di Huang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV2024

CAST: Cross-modal Alignment Similarity Test for Vision Language Models

Gautier Dagan, Olga Loginova, Anil Batra

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion

Yingxuan Li, Ryota Hinami, Kiyoharu Aizawa, Yusuke Matsui

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM Multimedia 2024. Project page: https://liyingxuan1012.github.io/zeroshot-speaker-prediction ; Github repo: https://github.com/liyingxuan1012/zeroshot-speaker-prediction

Robust Semi-supervised Multimodal Medical Image Segmentation via Cross Modality Collaboration

Xiaogen Zhou, Yiyou Sun, Min Deng, Winnie Chiu Wing Chu, Qi Dou

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data

Spencer Whitehead, Jacob Phillips, Sean Hendryx

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

KAN-RCBEVDepth: A multi-modal fusion algorithm in object detection for autonomous driving

Zhihao Lai, Chuanhao Liu, Shihui Sheng, Zhiqiang Zhang

专题命中 多模态训练与对齐 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Multimodal Ensemble with Conditional Feature Fusion for Dysgraphia Diagnosis in Children from Handwriting Samples

Jayakanth Kunhoth, Somaya Al-Maadeed, Moutaz Saleh, Younes Akbari

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Self-Supervised Multimodal Learning: A Survey

Yongshuo Zong, Oisin Mac Aodha, Timothy Hospedales

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to IEEE T-PAMI

MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples

Tao Chen, Enwei Zhang, Yuting Gao, Ke Li, Xing Sun, Yan Zhang, Hui Li, Rongrong Ji

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments TOMM 2024

MU-MAE: Multimodal Masked Autoencoders-Based One-Shot Learning

Rex Liu, Xin Liu

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments IEEE MIPR 2024

LoginMEA: Local-to-Global Interaction Network for Multi-modal Entity Alignment

Taoyu Su, Xinghua Zhang, Jiawei Sheng, Zhenyu Zhang, Tingwen Liu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by ECAI 2024

LLAVADI: What Matters For Multimodal Large Language Models Distillation

Shilin Xu, Xiangtai Li, Haobo Yuan, Lu Qi, Yunhai Tong, Ming-Hsuan Yang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment

Taoyu Su, Jiawei Sheng, Shicheng Wang, Xinghua Zhang, Hongbo Xu, Tingwen Liu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by ACM MM 2024

MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning

Vishal Nedungadi, Ankit Kariryaa, Stefan Oehmcke, Serge Belongie, Christian Igel, Nico Lang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for ECCV 2024. Data and code: https://vishalned.github.io/mmearth Update arXiv v2 (ECCV): 1. Dataset fix: Removed duplicates and corrected ERA5 yearly statistics. 2. Data augmentation fix: Random crops are now aligned. 3. Test metrics fix: Metrics are now overall instead of mini-batch averages, matching GEO-Bench metrics. 4. Pretrained on MMEarth v001 & evaluated on GEO-Bench v1.0

Exploring Fusion Techniques in Multimodal AI-Based Recruitment: Insights from FairCVdb

Swati Swati, Arjun Roy, Eirini Ntoutsi

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

FoRA: Low-Rank Adaptation Model beyond Multimodal Siamese Network

Weiying Xie, Yusi Zhang, Tianlin Hui, Jiaqing Zhang, Jie Lei, Yunsong Li

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Pay Less On Clinical Images: Asymmetric Multi-Modal Fusion Method For Efficient Multi-Label Skin Lesion Classification

Peng Tang, Tobias Lasser

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Robust Multimodal Learning via Representation Decoupling

Shicai Wei, Yang Luo, Yuji Wang, Chunbo Luo

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ECCV2024 17 pages

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roesmann, Christian Kerl, Rinu Boney, Yusu Qian, Zirui Wang, Afshin Dehghan, Yinfei Yang, Zhe Gan, Peter Grasch

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Similarity Guided Multimodal Fusion Transformer for Semantic Location Prediction in Social Media

Zhizhen Zhang, Ning Wang, Haojie Li, Zhihui Wang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV、cs.CL