arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.07825cs.CVcs.AIcs.LG

缺失模态下的深度多模态学习:综述

Deep Multimodal Learning with Missing Modality: A Survey

The Australian National University(澳大利亚国立大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Adelaide University(阿德莱德大学);The University of Surrey(萨里大学)

Renjie Wu, Hu Wang, Hsiang-Ting Chen, Gustavo Carneiro

AI总结:

本文综述了缺失模态下的多模态学习方法,分析了其动机、技术细节、应用及挑战,为该领域的发展提供了全面的视角。

摘要:

在多模态模型训练和测试过程中,由于传感器限制、成本约束、隐私问题或数据丢失,某些数据模态可能缺失,从而影响性能。旨在处理缺失模态的多模态学习技术可以通过确保模型在某些模态不可用时仍具备鲁棒性来缓解这一问题。本文综述了最近在缺失模态下的多模态学习(MLMM)方面的进展,重点探讨了深度学习方法。本文提供了首个全面的综述,涵盖了MLMM与标准多模态学习设置之间的动机和区别,随后详细分析了当前的方法、应用和数据集,并以挑战和未来方向作为结尾。

英文摘要:

During multimodal model training and testing, certain data modalities may be absent due to sensor limitations, cost constraints, privacy concerns, or data loss, negatively affecting performance. Multimodal learning techniques designed to handle missing modalities can mitigate this by ensuring model robustness even when some modalities are unavailable. This survey reviews recent progress in Multimodal Learning with Missing Modality (MLMM), focusing on deep learning methods. It provides the first comprehensive survey that covers the motivation and distinctions between MLMM and standard multimodal learning setups, followed by a detailed analysis of current methods, applications, and datasets, concluding with challenges and future directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑