arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.09989cs.LG
收藏

迈向开放世界中的鲁棒多模态学习

Towards Robust Multimodal Learning in the Open World

Department of Computing(计算系)

Fushuo Huo

AI总结:

本文针对开放世界环境中现有多模态模型鲁棒性不足的问题,探究该场景下多模态学习鲁棒性的核心挑战,以缩小模型受控实验表现与实际部署要求的差距。

摘要:

机器学习的快速发展推动神经网络在众多领域取得了前所未有的成功。其中,多模态学习已成为一种变革性范式,它利用来自异构数据流(例如文本、视觉、音频)的互补信息来推进上下文推理与智能决策。尽管取得了这些进展,当前基于神经网络的模型在具有内在不可预测性的开放世界环境中往往表现不佳:不可预测的环境构成动态、不完整的模态输入以及虚假的分布关系会严重破坏系统可靠性。人类能够自然适应这类动态、模糊的场景,但人工智能系统在鲁棒性上表现出明显局限,尤其是在真实世界复杂度下处理多模态信号时。本研究探究开放世界场景下多模态学习鲁棒性的核心挑战,旨在弥合受控实验性能与实际部署需求之间的差距。

英文摘要:

The rapid evolution of machine learning has propelled neural networks to unprecedented success across diverse domains. In particular, multimodal learning has emerged as a transformative paradigm, leveraging complementary information from heterogeneous data streams (e.g., text, vision, audio) to advance contextual reasoning and intelligent decision-making. Despite these advancements, current neural network-based models often fall short in open-world environments characterized by inherent unpredictability, where unpredictable environmental composition dynamics, incomplete modality inputs, and spurious distributions relations critically undermine system reliability. While humans naturally adapt to such dynamic, ambiguous scenarios, artificial intelligence systems exhibit stark limitations in robustness, particularly when processing multimodal signals under real-world complexity. This study investigates the fundamental challenge of multimodal learning robustness in open-world settings, aiming to bridge the gap between controlled experimental performance and practical deployment requirements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑