注意差距：一个多模态主动学习中陷阱评估的框架
Mind the Gap: A Framework for Assessing Pitfalls in Multimodal Active Learning
AI总结：
本文提出一个多模态主动学习评估框架，通过合成数据集系统评估主动学习策略，发现模型易产生不平衡表示，现有方法无法缓解此问题，需设计模态感知的查询策略。
摘要：
多模态学习使神经网络能够整合异质信息，但在此设置中主动学习面临独特挑战，包括缺失模态、模态难度差异和交互结构变化。这些问题是单模态情况所没有的。虽然单模态主动学习策略的行为已得到良好描述，但其在多模态条件下的行为仍不明确。我们引入了一个新的框架来基准多模态主动学习，通过合成数据集隔离这些陷阱，允许系统评估而无需混淆噪声。使用此框架，我们比较了单模态和多模态查询策略，并在两个真实数据集上验证了我们的发现。我们的结果表明，模型始终发展出不平衡的表示，主要依赖于一个模态而忽视其他模态。现有查询方法无法缓解此影响，多模态策略并不总能优于单模态方法。这些发现突显了当前主动学习方法的局限性，并强调了需要模态感知查询策略来解决这些陷阱的必要性。代码和基准资源将公开发布。
英文摘要：
Multimodal learning enables neural networks to integrate information from heterogeneous sources, but active learning in this setting faces distinct challenges. These include missing modalities, differences in modality difficulty, and varying interaction structures. These are issues absent in the unimodal case. While the behavior of active learning strategies in unimodal settings is well characterized, their behavior under such multimodal conditions remains poorly understood. We introduce a new framework for benchmarking multimodal active learning that isolates these pitfalls using synthetic datasets, allowing systematic evaluation without confounding noise. Using this framework, we compare unimodal and multimodal query strategies and validate our findings on two real-world datasets. Our results show that models consistently develop imbalanced representations, relying primarily on one modality while neglecting others. Existing query methods do not mitigate this effect, and multimodal strategies do not consistently outperform unimodal ones. These findings highlight limitations of current active learning methods and underline the need for modality-aware query strategies that explicitly address these pitfalls. Code and benchmark resources will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
