发表机构
Graduate School of Artificial Intelligence, UNIST（蔚山国家科学技术研究院人工智能研究生院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
MARBO提出基于关系信念的偏好优化框架，使紧凑LLM智能体在社交推理游戏中通过信念引导决策与言语，实现更一致学习并持续超越现有基线。
AI 中文摘要
社交推理游戏（SDGs）要求智能体在部分可观测性下进行推理，通过维护关于隐藏角色和团队阵营的关系信念。虽然最近的LLM智能体方法通过提示和偏好优化改善了游戏表现，但它们往往在优化行动和游戏内言语时，没有明确地将这些行为建立在关系信念之上。这经常导致策略不一致的行为，尤其是对于紧凑型LLM智能体。我们引入了多智能体关系信念优化（MARBO），一种基于信念的偏好优化框架，利用关系信念来指导策略决策和游戏内言语。MARBO仅在行为得到可靠关系信念支持并导致策略上有利的社会结果时提供偏好反馈，从而鼓励在不确定性下进行更一致的学习。在代表性SDGs上的实验表明，MARBO使紧凑型LLM智能体能够持续优于现有基线。代码可在该https URL上获取。
英文摘要
Social deduction games (SDGs) require agents to reason under partial observability by maintaining relational beliefs about hidden roles and team alignments. While recent LLM-agent approaches improve gameplay through prompting and preference optimization, they often optimize actions and in-game speech without explicitly grounding them in such beliefs. This frequently leads to strategically inconsistent behavior, especially for compact LLM agents. We introduce Multi-Agent Relational Belief Optimization (MARBO), a belief-grounded preference optimization framework that leverages relational beliefs to guide strategic decisions and in-game speech. MARBO provides preference feedback only when behaviors are supported by reliable relational beliefs and lead to strategically favorable social outcomes, encouraging more consistent learning under uncertainty. Experiments on representative SDGs show that MARBO enables compact LLM agents to consistently outperform existing baselines. The Code is available on https://github.com/PleaseTakemeAway/MARBO.
Comments9 pages, accepted to EMNLP 2026
