arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.00758cs.LGcs.CVcs.SDeess.ASstat.ML
收藏

端到端多模态情感和性别识别的动态联合损失权重

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);NOTA Incorporated(NOTA公司)

Myungsu Chae, Tae-Ho Kim, Young Hoon Shin, June-Woo Kim, Soo-Young Lee

摘要:

多任务学习是一种提高多任务概括能力的方法。为了用一个神经网络模型执行多个分类任务,应该将每个任务的损失合并。以前的研究主要集中在多个预测任务上,使用联合损失和静态权重来训练模型,选择任务之间的权重,而没有通过统一或经验地设置它们来充分考虑。在这项研究中,我们提出了一种方法来计算联合损失使用动态权重,以提高整体性能,而不是个人的性能,任务。我们应用这种方法来设计一个端到端的多模态情感和性别识别模型,使用音频和视频数据。这种方法为训练过程结束时每个任务的损失提供了适当的权重。在我们的实验中,情感和性别识别与所提出的方法产生了较低的联合损失,这是计算为负对数似然,比使用静态权重的联合损失。此外,我们提出的模型具有更好的泛化能力比其他模型。据我们所知,这项研究是第一个证明使用动态权重的联合损失的力量,以最大限度地提高整体性能的情绪和性别识别任务。

英文摘要:

Multi-task learning is a method for improving the generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on multiple prediction tasks using joint loss with static weights for training models, choosing the weights between tasks without making sufficient considerations by setting them uniformly or empirically. In this study, we propose a method to calculate joint loss using dynamic weights to improve the total performance, instead of the individual performance, of tasks. We apply this method to design an end-to-end multimodal emotion and gender recognition model using audio and video data. This approach provides proper weights for the loss of each task when the training process ends. In our experiments, emotion and gender recognition with the proposed method yielded a lower joint loss, which is computed as the negative log-likelihood, than using static weights for joint loss. Moreover, our proposed model has better generalizability than other models. To the best of our knowledge, this research is the first to demonstrate the strength of using dynamic weights for joint loss for maximizing overall performance in emotion and gender recognition tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑