arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.23802cs.AIcs.CV
收藏

EMO-R3:多模态大语言模型中用于情感推理的反射强化学习

EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

School of Computer Science, Wuhan University(武汉大学计算机学院);MiLM Plus, Xiaomi Inc.(小米公司)

Yiyang Fang, Wenke Huang, Pei Fu, Yihao Yang, Kehua Su, Zhenbo Luo, Jian Luan, Mang Ye

AI总结:

针对多模态大语言模型情感推理难题,提出EMO-R3框架。引入结构化情感思维并设计反射情感奖励,经大量实验验证,该框架显著提升模型可解释性与情商,在多视觉情感理解基准测试中表现出色。

摘要:

多模态大语言模型在视觉推理和理解任务中取得了显著进展,但在捕捉人类情感的复杂性和主观性方面仍存在困难。基于监督微调的现有方法往往泛化能力有限且可解释性差,而诸如群体相对策略优化等强化学习方法与情感认知的内在特征不匹配。为应对这些挑战,我们提出了用于情感推理的反射强化学习(EMO-R3)框架,以增强多模态大语言模型的情感推理能力。具体而言,我们引入结构化情感思维,以指导模型以结构化和可解释的方式进行逐步情感推理,并设计了反射情感奖励,使模型能够基于视觉-文本一致性和情感连贯性重新评估其推理。大量实验表明,EMO-R3显著提高了多模态大语言模型的可解释性和情商,在多个视觉情感理解基准上取得了优异性能。

英文摘要:

Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human emotions. Existing approaches based on supervised fine-tuning often suffer from limited generalization and poor interpretability, while reinforcement learning methods such as Group Relative Policy Optimization fail to align with the intrinsic characteristics of emotional cognition. To address these challenges, we propose Reflective Reinforcement Learning for Emotional Reasoning (EMO-R3), a framework designed to enhance the emotional reasoning ability of MLLMs. Specifically, we introduce Structured Emotional Thinking to guide the model to perform step-by-step emotional reasoning in a structured and interpretable manner, and design a Reflective Emotional Reward that enables the model to re-evaluate its reasoning based on visual-text consistency and emotional coherence. Extensive experiments demonstrate that EMO-R3 significantly improves both the interpretability and emotional intelligence of MLLMs, achieving superior performance across multiple visual emotional understanding benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑