arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.08306cs.MMcs.LG
收藏

面向多模态分类的平衡主动学习研究

Towards Balanced Active Learning for Multimodal Classification

Nanyang Technological University(南洋理工大学);Institute for Infocomm Research A*STAR(新加坡科技研究局资讯通信研究院);NVIDIA AI Tech Center(英伟达人工智能技术中心)

Meng Shen, Yizheng Huang, Jianxiong Yin, Heqing Zou, Deepu Rajan, Simon See

AI总结:

针对现有单模态主动学习策略应用于多模态分类时存在主导模态样本选择偏差的问题,提出三条平衡策略设计准则及基于模态主导度调制梯度嵌入的公平样本选择方法,在多类任务上性能优于现有方法。

摘要:

与单模态网络相比,多模态网络的参数空间更大,因此训练需要海量数据。主动学习是一种广泛应用的技术,通过仅筛选那些有助于提升模型性能的样本来降低数据标注成本。然而,当前的主动学习策略大多是为单模态任务设计的,当应用于多模态数据时,往往会导致样本选择偏向占主导地位的模态。这种不均衡性会阻碍平衡的多模态学习,而平衡学习对实现最优性能至关重要。为解决这一问题,我们提出了三条设计更平衡的多模态主动学习策略的指导原则。遵循这些原则,我们提出了一种新方法,通过利用模态间的主导程度调制梯度嵌入,实现更公平的数据选择。研究表明,所提方法避免了从主导模态进行贪婪式样本选择,从而实现了更平衡的多模态学习。在多种多模态分类任务上,我们的方法性能优于现有主动学习策略。总体而言,本工作强调了多模态主动学习中平衡样本选择的重要性,并为实现更平衡的多模态分类主动学习提供了实用解决方案。

英文摘要:

Training multimodal networks requires a vast amount of data due to their larger parameter space compared to unimodal networks. Active learning is a widely used technique for reducing data annotation costs by selecting only those samples that could contribute to improving model performance. However, current active learning strategies are mostly designed for unimodal tasks, and when applied to multimodal data, they often result in biased sample selection from the dominant modality. This unfairness hinders balanced multimodal learning, which is crucial for achieving optimal performance. To address this issue, we propose three guidelines for designing a more balanced multimodal active learning strategy. Following these guidelines, a novel approach is proposed to achieve more fair data selection by modulating the gradient embedding with the dominance degree among modalities. Our studies demonstrate that the proposed method achieves more balanced multimodal learning by avoiding greedy sample selection from the dominant modality. Our approach outperforms existing active learning strategies on a variety of multimodal classification tasks. Overall, our work highlights the importance of balancing sample selection in multimodal active learning and provides a practical solution for achieving more balanced active learning for multimodal classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑