arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11665cs.CL
收藏

多任务强化学习用于增强多模态大语言模型作为裁判

Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge

Meta AI;Hong Kong University of Science and Technology(香港科技大学);Emory University(埃默里大学)

Junjie Wu, Xuan Kan, Zihao He, Shunwen Tan, Bo Pan, Kaitai Zhang

AI总结:

本文提出MT-RL-Judge框架,通过多任务强化学习优化多模态大语言模型作为裁判,提升判断一致性和与人类偏好的相关性,并在分布外任务中展现鲁棒泛化能力。

摘要:

多模态大语言模型(MLLMs)因其在各种视觉任务中与人类判断的强对齐性而被广泛采用为MLLM-as-a-Judge。然而,大多数现有裁判模型仅针对单任务场景进行优化,难以泛化到多样化上下文,这对可靠评估至关重要。为解决这一限制,我们提出多任务强化学习用于MLLM-as-a-Judge(MT-RL-Judge),该框架通过联合优化裁判模型跨多个任务,利用强化学习的泛化能力。实验结果表明,MT-RL-Judge在判断一致性和与人类偏好的相关性方面均优于强基线模型。此外,我们的方法在分布外任务中展现出稳健的泛化能力,进一步验证了其有效性。

英文摘要:

Multimodal Large Language Models (MLLMs) have been widely adopted as MLLM-as-a-Judges due to their strong alignment with human judgment across various visual tasks. However, most existing judge models are optimized for single-task scenarios and struggle to generalize to diverse contexts, which is a critical requirement for reliable evaluation. To address this limitation, we propose Multi-Task Reinforcement Learning for MLLM-as-a-Judge (MT-RL-Judge), a framework that jointly optimizes the judge model across multiple tasks, leveraging the generalization capabilities of RL. Experimental results against several strong baselines demonstrate that MT-RL-Judge outperforms strong baselines in both judgment consistency and correlation with human preferences. Furthermore, our approach exhibits robust generalization on out-of-distribution tasks, further validating its effectiveness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑