多模态大模型 - arXivDaily 专题

2403.04650 2024-09-10 cs.LG cs.AI 79%

Lightweight Cross-Modal Representation Learning

Bilal Faye, Hanane Azzag, Mustapha Lebbah, Djamel Bouchaffra

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.AI

Journal ref ESANN 2024

2409.01151 2024-09-04 cs.CV cs.LG 79%

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Yueqian Wang, Jianxin Liang, Yuxuan Wang, Huishuai Zhang, Dongyan Zhao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2312.16778 2024-09-04 cs.CL 79%

Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition

Yuntao Shou, Tao Meng, Wei Ai, Nan Yin, Keqin Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments 14 pages, 6 figures

2311.05348 2024-08-29 cs.CV 79%

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Fanyi Wang, Yanchun Xie, Yi-Jie Huang, Yaqian Li

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2408.13830 2024-08-28 cs.CV 79%

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Yuhong Jiao, Jiaqing Miao, Jinnan Gong, Hui He, Ping Liang, Cheng Luo, Ying Tan

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.14387 2024-08-27 cs.LG cs.AI 79%

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning

Sakhinana Sagar Srinivas, Chidaksh Ravuru, Geethan Sannidhi, Venkataramana Runkana

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.AI

Comments Paper published at the Deployable AI (DAI) workshop at AAAI-2024

2408.13622 2024-08-27 cs.LG cs.AI 79%

Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings

Sagar Srinivas Sakhinana, Geethan Sannidhi, Chidaksh Ravuru, Venkataramana Runkana

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

Comments Published at the ICLR 2024 Workshop on Practical ML for Low Resource Settings(PML4LRS)

2308.10421 2024-08-26 cs.CV 79%

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving

Jian Zou, Tianyu Huang, Guanglei Yang, Zhenhua Guo, Tao Luo, Chun-Mei Feng, Wangmeng Zuo

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Code available at https://github.com/hollow-503/UniM2AE

2407.19156 2024-08-20 cs.CV 79%

Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble

Juhan Cha, Minseok Joo, Jihwan Park, Sanghyeok Lee, Injae Kim, Hyunwoo J. Kim

专题命中多模态训练与对齐：multimodal(title)；multi-modal(abstract)；分类 cs.CV

2309.05919 2024-08-20 eess.IV cs.CV 79%

Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

Ling Huang, Su Ruan, Pierre Decazes, Thierry Denoeux

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.09064 2024-08-20 cs.CV cs.LG 79%

MoRA: LoRA Guided Multi-Modal Disease Diagnosis with Missing Modality

Zhiyi Shi, Junsik Kim, Wanhua Li, Yicong Li, Hanspeter Pfister

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

2408.08564 2024-08-19 cs.IR cs.CL 79%

Collaborative Cross-modal Fusion with Large Language Model for Recommendation

Zhongzhou Liu, Hao Zhang, Kuicai Dong, Yuan Fang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CL

Comments 10 pages, 4 figures, accepted by CIKM 2024

2308.07222 2024-08-15 cs.IR cs.AI 79%

MM-GEF: Multi-modal representation meet collaborative filtering

Hao Wu, Alejandro Ariza-Casabona, Bartłomiej Twardowski, Tri Kurniawan Wijaya

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2408.05751 2024-08-13 cs.IR cs.CV 79%

Advancing Re-Ranking with Multimodal Fusion and Target-Oriented Auxiliary Tasks in E-Commerce Search

Enqiang Xu, Xinhui Li, Zhigong Zhou, Jiahao Ji, Jinyuan Zhao, Dadong Miao, Songlin Wang, Lin Liu, Sulong Xu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.04547 2024-08-09 cs.MM 79%

Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation

Haoxiang Shi, Ziqi Liang, Jun Yu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.MM

Comments Accepted by INTERSPEECH 2024

2408.04170 2024-08-09 cs.CV 79%

M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction

Hui Luo, Jiashuang Huang, Hengrong Ju, Tianyi Zhou, Weiping Ding

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2408.02503 2024-08-06 cs.CL 79%

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model

Zhaowei Li, Wei Wang, YiQing Cai, Xu Qi, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL

2408.00779 2024-08-05 cs.LG cs.AI cs.ET cs.IT math.IT q-bio.BM 79%

Learning Structurally Stabilized Representations for Multi-modal Lossless DNA Storage

Ben Cao, Tiantian He, Xue Li, Bin Wang, Xiaohu Wu, Qiang Zhang, Yew-Soon Ong

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2312.14433 2024-08-01 cs.IR cs.MM 79%

Attribute-driven Disentangled Representation Learning for Multimodal Recommendation

Zhenyang Li, Fan Liu, Yinwei Wei, Zhiyong Cheng, Liqiang Nie, Mohan Kankanhalli

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

Comments ACM Multimedia 2024 Accepted

Journal ref In Proceedings of the 32st ACM International Conference on Multimedia (MM '24), 2024

2308.14484 2024-07-31 cs.CL 79%

Multimodal Detection of Bots on X (Twitter) using Transformers

Loukas Ilias, Ioannis Michail Kazelidis, Dimitris Askounis

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments IEEE Transactions on Information Forensics and Security (Accepted)

2407.19886 2024-07-30 cs.AI 79%

A Unified Graph Transformer for Overcoming Isolations in Multi-modal Recommendation

Zixuan Yi, Iadh Ounis

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2404.09640 2024-07-24 cs.CV 79%

CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning

Haojian Huang, Xiaozhen Qiao, Zhuo Chen, Haodong Chen, Bingyu Li, Zhe Sun, Mulin Chen, Xuelong Li

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024; 10 pages, 2 Tables, 9 Figures; Repo is available at: https://github.com/JethroJames/CREST

2407.15819 2024-07-23 cs.CV 79%

Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight

Ziyuan Huang, Kaixiang Ji, Biao Gong, Zhiwu Qing, Qinglong Zhang, Kecheng Zheng, Jian Wang, Jingdong Chen, Ming Yang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2407.14439 2024-07-22 cs.CV 79%

Token-level Correlation-guided Compression for Efficient Multimodal Document Understanding

Renshan Zhang, Yibo Lyu, Rui Shao, Gongwei Chen, Weili Guan, Liqiang Nie

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2407.14188 2024-07-22 eess.IV cs.CV 79%

TaGAT: Topology-Aware Graph Attention Network For Multi-modal Retinal Image Fusion

Xin Tian, Nantheera Anantrasirichai, Lindsay Nicholson, Alin Achim

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 2 figures, accepted by MICCAI 2024

2407.12338 2024-07-18 cs.IR cs.AI 79%

GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation

Guojiao Lin, Zhen Meng, Dongjie Wang, Qingqing Long, Yuanchun Zhou, Meng Xiao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments 11 pages, accepted by CIKM 2024

2407.12309 2024-07-18 cs.CL 79%

MEDFuse: Multimodal EHR Data Fusion with Masked Lab-Test Modeling and Large Language Models

Thao Minh Nguyen Phan, Cong-Tinh Dao, Chenwei Wu, Jian-Zhe Wang, Shun Liu, Jun-En Ding, David Restrepo, Feng Liu, Fang-Ming Hung, Wen-Chih Peng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

2403.03281 2024-07-18 cs.LG cs.AI 79%

Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuits

Sahil Sidheekh, Pranuthi Tenali, Saurabh Mathur, Erik Blasch, Kristian Kersting, Sriraam Natarajan

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2407.11683 2024-07-17 cs.CV 79%

Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning

Yunbin Tu, Liang Li, Li Su, Chenggang Yan, Qingming Huang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

2403.05060 2024-07-17 cs.MM cs.HC 79%

Multimodal Infusion Tuning for Large Models

Hao Sun, Yu Song, Xinyao Yu, Jiaqing Liu, Yen-Wei Chen, Lanfen Lin

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

1. 多模态训练与对齐 6918 篇

Lightweight Cross-Modal Representation Learning

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning

Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving

Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble

Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

MoRA: LoRA Guided Multi-Modal Disease Diagnosis with Missing Modality

Collaborative Cross-modal Fusion with Large Language Model for Recommendation

MM-GEF: Multi-modal representation meet collaborative filtering

Advancing Re-Ranking with Multimodal Fusion and Target-Oriented Auxiliary Tasks in E-Commerce Search

Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation

M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model

Learning Structurally Stabilized Representations for Multi-modal Lossless DNA Storage

Attribute-driven Disentangled Representation Learning for Multimodal Recommendation

Multimodal Detection of Bots on X (Twitter) using Transformers

A Unified Graph Transformer for Overcoming Isolations in Multi-modal Recommendation

CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning

Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight

Token-level Correlation-guided Compression for Efficient Multimodal Document Understanding

TaGAT: Topology-Aware Graph Attention Network For Multi-modal Retinal Image Fusion

GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation

MEDFuse: Multimodal EHR Data Fusion with Masked Lab-Test Modeling and Large Language Models

Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuits

Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning

Multimodal Infusion Tuning for Large Models