arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.03109cs.CL
收藏

一个模型,所有角色:多轮、多智能体自我对战强化学习用于对话社交智能

One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence

Microsoft Corporation(微软公司);University of Pennsylvania(宾夕法尼亚大学);University of Southern California(南加州大学);Penn State University(宾夕法尼亚州立大学)

Bowen Jiang, Taiwei Shi, Ryo Kamoi, Yuan Yuan, Camillo J. Taylor, Longqi Yang, Pei Zhou, Sihao Chen

AI总结:

OMAR通过多轮多智能体自我对战强化学习,使AI在无监督情况下发展出共情、说服等社交智能。

摘要:

本文介绍了OMAR:一个模型,所有角色,一种强化学习框架,使AI通过多轮、多智能体对话自我对战来发展社交智能。与传统范式依赖静态、单轮优化不同,OMAR允许一个模型同时扮演对话中的所有参与者,从动态社交互动中学习实现长期目标和复杂社会规范。为确保在长对话中的训练稳定性,我们实现了分层优势估计,计算回合级和token级优势。在SOTOPIA社交环境和Werewolf策略游戏中评估显示,训练后的模型发展出精细的、涌现的社交智能,如共情、说服和妥协寻求,证明了在竞争场景下学习协作的有效性。尽管我们识别出实际挑战如奖励黑客,但结果表明,丰富的社交智能可以在无人类监督的情况下涌现。我们希望这项工作能激励进一步研究AI在群体对话中的社交智能。

英文摘要:

This paper introduces OMAR: One Model, All Roles, a reinforcement learning framework that enables AI to develop social intelligence through multi-turn, multi-agent conversational self-play. Unlike traditional paradigms that rely on static, single-turn optimizations, OMAR allows a single model to role-play all participants in a conversation simultaneously, learning to achieve long-term goals and complex social norms directly from dynamic social interaction. To ensure training stability across long dialogues, we implement a hierarchical advantage estimation that calculates turn-level and token-level advantages. Evaluations in the SOTOPIA social environment and Werewolf strategy games show that our trained models develop fine-grained, emergent social intelligence, such as empathy, persuasion, and compromise seeking, demonstrating the effectiveness of learning collaboration even under competitive scenarios. While we identify practical challenges like reward hacking, our results show that rich social intelligence can emerge without human supervision. We hope this work incentivizes further research on AI social intelligence in group conversations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑