TeamGym：面向具有分布式信息的AI智能体团队决策的Gym平台
TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information
AI总结：
研究团队推出TeamGym平台，针对团队决策问题测试8种语言模型，发现智能体存在沟通与检索差距，通过强化学习训练可提升决策质量并实现涌现协调。
摘要：
许多现实世界的决策需要团队整合分布在不同参与者之间的信息，但目前AI智能体的训练和基准测试仍主要是在孤立环境中进行的。我们推出TeamGym，这是一个用于评估和训练基于语言模型的智能体的环境，其处理的团队决策问题来自现实世界的法律、金融、医疗和软件工程语料库。TeamGym引入了一种方法来识别最小事实集，其中每个元素对于做出正确的最终决策都是必要的，而所有元素共同构成充分条件，我们将这些事实称为关键信息。这使得TeamGym既可以控制关键信息的分布，又可以跟踪智能体在做出团队决策时识别和整合关键信息的表现。我们在12个源语料库上测试了8种语言模型，发现当前AI智能体常常无法有效共享信息：与将相同信息提供给单个模型相比，将关键信息分布在不同参与者之间会使成功率平均降低22.9个百分点（这一现象称为沟通差距）。我们还发现，参与者无法自行有效检索与决策相关的信息：当参与者必须从源材料中识别和提取关键信息而非直接接收这些信息时，团队的成功率会进一步降低19.9个百分点（这一现象称为检索差距）。为了帮助弥补这些缺陷，我们使用TeamGym通过强化学习针对团队决策损失来训练策略。在两次策略训练实验中，我们发现训练提升了最终决策的质量，并在参与、说话者选择和沟通方面诱导出了涌现的协调，尽管奖励仅评估最终决策。
英文摘要：
Many real-world decisions require teams to integrate information distributed across participants. Yet AI agents are still primarily trained and benchmarked in isolation. We introduce TeamGym, an environment for evaluating and training language model-based agents on team decision-making problems constructed from real-world legal, finance, medical, and software engineering corpora. TeamGym introduces a method to identify a minimal set of facts, each element of which is necessary and all of which are collectively sufficient for a correct final decision, which we call pivotal information. This enables TeamGym to both control the distribution of pivotal information and track how well agents identify and integrate pivotal information when making team decisions. Examining 8 language models on 12 source corpora, we find that current AI agents often fail to effectively share information: Distributing pivotal information across participants lowers success rate by 22.9 percentage points on average relative to giving the same information to a single model (a communication gap). We also document that participants do not effectively retrieve decision-relevant information on their own: A team's success rate is lowered by a further 19.9 points when participants must identify and extract pivotal information from source material rather than receiving it directly (a retrieval gap). To help remedy these deficits, we use TeamGym to train policies through reinforcement learning on team decision loss. Across two policy-training experiments, we find that training improves final-decision quality and induces emergent coordination in participation, speaker selection, and communication, even though the reward evaluates only the final decision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
