arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09249cs.AI
收藏

Social-R1: 向大语言模型中引入人类般的社交推理

Social-R1: Towards Human-like Social Reasoning in LLMs

Jincenzi Wu, Yuxuan Lei, Jianxun Lian, Yitian Huang, Lexin Zhou, Haotian Li, Xing Xie, Helen Meng

AI总结:

Social-R1通过多维奖励对齐模型推理与人类认知,利用ToMBench-Hard提供困难训练案例,使大模型在社交推理任务中实现超越和稳健泛化。

摘要:

尽管大语言模型在众多领域展现出显著的能力,但社交智能——感知社会线索、推断心理状态并生成适当回应的能力——仍然是一个关键挑战,尤其是在促进有效的人工智能协作和开发真正服务于人类需求的人工智能方面。当前模型往往依赖于表面模式而不是真正的社交推理。我们主张,培养人类般的社交智能需要通过具有挑战性的案例进行训练,这些案例难以通过捷径解决。为此,我们引入了ToMBench-Hard,一个对抗性基准,旨在为社交推理提供困难的训练示例。在此基础上,我们提出了Social-R1,一种强化学习框架,通过多维奖励对模型推理与人类认知进行对齐。与基于结果的强化学习不同,Social-R1监督整个推理过程,强制执行结构对齐、逻辑完整性和信息密度。结果表明,我们的方法使400亿参数模型超越了更大的模型,并在八个不同的基准上实现了稳健的泛化。这些发现表明,具有轨迹级对齐的挑战性训练案例为高效且可靠的社交智能提供了一条道路。

英文摘要:

While large language models demonstrate remarkable capabilities across numerous domains, social intelligence - the capacity to perceive social cues, infer mental states, and generate appropriate responses - remains a critical challenge, particularly for enabling effective human-AI collaboration and developing AI that truly serves human needs. Current models often rely on superficial patterns rather than genuine social reasoning. We argue that cultivating human-like social intelligence requires training with challenging cases that resist shortcut solutions. To this end, we introduce ToMBench-Hard, an adversarial benchmark designed to provide hard training examples for social reasoning. Building on this, we propose Social-R1, a reinforcement learning framework that aligns model reasoning with human cognition through multi-dimensional rewards. Unlike outcome-based RL, Social-R1 supervises the entire reasoning process, enforcing structural alignment, logical integrity, and information density. Results show that our approach enables a 4B parameter model to surpass much larger counterparts and generalize robustly across eight diverse benchmarks. These findings demonstrate that challenging training cases with trajectory-level alignment offer a path toward efficient and reliable social intelligence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑