一种鲁棒的不完整多模态低秩适应方法用于情感识别
A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition
AI总结：
本文提出MCULoRA方法，通过解耦模态组合的共享信息和动态调整参数微调，提升不完整多模态学习模型的训练效率和下游任务准确性。
摘要：
多模态情感识别（MER）在实际应用中常常由于传感器故障或隐私保护要求而遇到不完整的多模态问题。尽管现有方法试图通过额外梯度平衡每种模态组合的训练来解决各种不完整的多模态场景，但这些方法面临一个关键限制：不同模态组合的训练梯度相互冲突，最终降低最终预测模型的性能。在本文中，我们提出了一种基于模态组合的单模态解耦动态低秩适应方法，称为MCULoRA，这是一种新颖的参数高效训练不完整多模态学习模型的框架。MCULoRA包含两个关键模块，模态组合感知低秩适应（MCLA）和动态参数微调（DPFT）。MCLA模块有效解耦了不同模态组合的共享信息与个体模态组合的特征差异。DPFT模块根据每个模态表示空间的分离性调整模态组合的训练比例，优化不同模态组合之间的学习效率。我们对多个基准数据集进行了广泛的实验评估，证明MCULoRA在下游任务准确性上显著优于先前的不完整多模态学习方法。
英文摘要：
Multimodal Emotion Recognition (MER) often encounters incomplete multimodality in practical applications due to sensor failures or privacy protection requirements. While existing methods attempt to address various incomplete multimodal scenarios by balancing the training of each modality combination through additional gradients, these approaches face a critical limitation: training gradients from different modality combinations conflict with each other, ultimately degrading the performance of the final prediction model. In this paper, we propose a unimodal decoupled dynamic low-rank adaptation method based on modality combinations, named MCULoRA, which is a novel framework for the parameter-efficient training of incomplete multimodal learning models. MCULoRA consists of two key modules, modality combination aware low-rank adaptation (MCLA) and dynamic parameter fine-tuning (DPFT). The MCLA module effectively decouples the shared information from the distinct characteristics of individual modality combinations. The DPFT module adjusts the training ratio of modality combinations based on the separability of each modality's representation space, optimizing the learning efficiency across different modality combinations. Our extensive experimental evaluation in multiple benchmark datasets demonstrates that MCULoRA substantially outperforms previous incomplete multimodal learning approaches in downstream task accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
