Recent advancements in autonomous multi-agent systems (MAS) based on large language models (LLMs) have enhanced the application scenarios and improved the capability of LLMs to handle complex tasks. Despite demonstrating effectiveness, existing studies still evidently struggle to evaluate, analysis, and reproducibility of LLM-based MAS. In this paper, to facilitate the research on LLM-based MAS, we introduce an open, scalable, and real-time updated platform for accessing and analyzing the LLM-based MAS based on the games Who is Spy?" (WiS). Our platform is featured with three main worths: (1) a unified model evaluate interface that supports models available on Hugging Face; (2) real-time updated leaderboard for model evaluation; (3) a comprehensive evaluation covering game-winning rates, attacking, defense strategies, and reasoning of LLMs. To rigorously test WiS, we conduct extensive experiments coverage of various open- and closed-source LLMs, we find that different agents exhibit distinct and intriguing behaviors in the game. The experimental results demonstrate the effectiveness and efficiency of our platform in evaluating LLM-based MAS. Our platform and its documentation are publicly available at https://whoisspy.ai/.
Choose Your Game Wisely: Measuring Game-Theoretic Structures in Real-World Vehicle Interactions
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；School of Airspace Science and Engineering, Shandong University（山东大学空天科学与工程学院）；Global Institute of Future Technology, Shanghai Jiao Tong University（上海交通大学未来技术全球研究院）；Global College, Shanghai Jiao Tong University（上海交通大学全球学院）
Pruning as a Cooperative Game: Surrogate-Assisted Layer Contribution Estimation for Large Language Models
Shenzhen Future Network of Intelligence Institute（深圳未来网络智能研究院）；Guangdong Provincial Key Laboratory of Future Networks of Intelligence（广东省未来网络智能重点实验室）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Beijing Normal University（北京师范大学）；Alibaba Group（阿里巴巴集团）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University（浙江大学）
Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Core
Key Laboratory of Intelligence, Games and Information Processing in Shaanxi Province（陕西省智能信息处理重点实验室）；Unmanned System Research Institute（无人系统研究院）；Northwestern Polytechnical University（西北工业大学）；School of Mathematics and Statistics（数学与统计学院）；Information Science Academy（信息科学学院）；China Electronics Technology Group Corporation（中国电子科技集团有限公司）
Enhancing Commentary Strategies for Imperfect Information Card Games: A Study of Large Language Models in Guandan Commentary
Guangdong University of Technology（广东工业大学）；East China Jiaotong University（华东交通大学）；Emory University（埃默里大学）；University of Minnesota - Twin Cities（明尼苏达大学双城分校）；South China University of Technology（华南理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Amazon（亚马逊公司）；University of Toronto（多伦多大学）
