多智能体环境中的空间推理任务(如事件预测、智能体类型识别或缺失数据填补)对多种应用(例如传感器网络上的自主监控,以及强化学习(RL)的子任务)非常重要。StarCraft II 游戏回放编码了智能(及对抗性)的多智能体行为,可为这些任务提供测试平台;然而,提取简单且标准化的表示以进行这些任务的原型设计非常费力,且阻碍了可复现性。相比之下,MNIST 和 CIFAR10 尽管极其简单,却使 ML 方法的快速原型设计和复现成为可能。遵循这些数据集的简单性,我们基于 StarCraft II 回放构建了一个基准空间推理数据集,该数据集展现了复杂的多智能体行为,同时仍像 MNIST 和 CIFAR10 一样易于使用。具体而言,我们仔细总结了 255 个连续游戏状态的窗口,从 60,000 场回放中创建了 360 万张摘要图像,包括所有相关的元数据,如游戏结果和玩家种族。我们开发了三种复杂度递减的格式:包含每种单位类型一个通道的高光谱图像(类似于多光谱地理空间图像)、模仿 CIFAR10 的 RGB 图像,以及模仿 MNIST 的灰度图像。我们展示了如何使用该数据集进行空间推理方法的原型设计。所有数据集、提取代码和数据集加载代码可在 https://starcraftdata.davidinouye.com 找到。
英文摘要:
Spatial reasoning tasks in multi-agent environments such as event prediction, agent type identification, or missing data imputation are important for multiple applications (e.g., autonomous surveillance over sensor networks and subtasks for reinforcement learning (RL)). StarCraft II game replays encode intelligent (and adversarial) multi-agent behavior and could provide a testbed for these tasks; however, extracting simple and standardized representations for prototyping these tasks is laborious and hinders reproducibility. In contrast, MNIST and CIFAR10, despite their extreme simplicity, have enabled rapid prototyping and reproducibility of ML methods. Following the simplicity of these datasets, we construct a benchmark spatial reasoning dataset based on StarCraft II replays that exhibit complex multi-agent behaviors, while still being as easy to use as MNIST and CIFAR10. Specifically, we carefully summarize a window of 255 consecutive game states to create 3.6 million summary images from 60,000 replays, including all relevant metadata such as game outcome and player races. We develop three formats of decreasing complexity: Hyperspectral images that include one channel for every unit type (similar to multispectral geospatial images), RGB images that mimic CIFAR10, and grayscale images that mimic MNIST. We show how this dataset can be used for prototyping spatial reasoning methods. All datasets, code for extraction, and code for dataset loading can be found at https://starcraftdata.davidinouye.com
SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systems
Technical University of Darmstadt(达姆施塔特技术大学);German Research Center for Artificial Intelligence(德国人工智能研究中心);Lab1141(Lab1141实验室);Centre for Cognitive Science, Darmstadt(达姆施塔特认知科学中心)
Neural Amortized Inference for Nested Multi-agent Reasoning
Dartmouth College(达特茅斯学院);Google Research(谷歌研究院);New York University(纽约大学);University of Virginia(弗吉尼亚大学);Massachusetts Institute of Technology(麻省理工学院);Johns Hopkins University(约翰斯·霍普金斯大学)
Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree
South China University of Technology(华南理工大学);Key Laboratory of Big Data and Intelligent Robot (SCUT), Ministry of Education(大数据与智能机器人重点实验室(SCUT));Hong Kong Polytechnic University(香港理工大学)
Adaptive Reasoning Executor: A Collaborative Agent System for Efficient Reasoning
Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化院);School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Academy of Artificial Intelligence for Science(上海人工智能科学研究院)
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering
Digital Research Center of Sfax, SMARTS Laboratory(突尼斯斯法克斯数字研究中心,SMARTS实验室);École Polytechnique de Tunisie, University of Carthage(突尼斯理工学院,卡塔赫纳大学);Computer Vision Center, Universitat Autònoma de Barcelona(巴塞罗那自治大学计算机视觉中心)
Visual Para-Thinker++: A Single-Policy Multi-Agent Framework for Visual Reasoning
Zhejiang University(浙江大学);Hunan University(湖南大学);Tianjin University(天津大学);University of Chinese Academy of Sciences(中国科学院大学);Shanghai Jiao Tong University(上海交通大学);Jilin University(吉林大学)