Promoting behavioural diversity is critical for solving games with non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e.g., Rock-Paper-Scissors). Yet, there is a lack of rigorous treatment for defining diversity and constructing diversity-aware learning dynamics. In this work, we offer a geometric interpretation of behavioural diversity in games and introduce a novel diversity metric based on determinantal point processes (DPP). By incorporating the diversity metric into best-response dynamics, we develop diverse fictitious play and diverse policy-space response oracle for solving normal-form games and open-ended games. We prove the uniqueness of the diverse best response and the convergence of our algorithms on two-player games. Importantly, we show that maximising the DPP-based diversity metric guarantees to enlarge the gamescape -- convex polytopes spanned by agents' mixtures of strategies. To validate our diversity-aware solvers, we test on tens of games that show strong non-transitivity. Results suggest that our methods achieve at least the same, and in most games, lower exploitability than PSRO solvers by finding effective and diverse strategies.
Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Games
Shanghai Jiao Tong University（上海交通大学）；Netease Fuxi AI Lab（网易伏羲人工智能实验室）；University College London（伦敦大学学院）
衡量和促进政策多样性对于解决存在战略周期且没有一致赢家的具有强非传递动态的游戏至关重要（例如，石头剪刀布）。考虑到这一点，通过开放式学习来维护一个多样化的政策库是一个有吸引力的解决方案，它可以生成自动课程，以避免被利用。然而，在传统的开放式学习算法，有没有广泛接受的定义的多样性，这使得它很难构建和评估的多样性政策。在这项工作中，我们总结了以前的多样性的概念，并致力于提供一个统一的多样性措施，在多智能体开放式学习，包括马尔可夫游戏中的所有元素，基于行为多样性（BD）和响应多样性（RD）。在轨迹分布层次上，我们将状态-动作空间中的BD重新定义为占有测度的差异。对于奖励动力学，我们提出RD通过政策在遇到不同对手时的反应来表征多样性。我们还表明，目前的许多多样性措施属于BD或RD的类别之一，但不是两者兼而有之。利用这种统一的多样性测度，设计了开放式学习中寻求最佳对策时相应的多样性促进目标和种群有效性。我们验证我们的方法在两个相对简单的游戏，如矩阵游戏，非传递混合模型，和复杂的\textit{Google Research Football}环境。在矩阵博弈和非传递混合模型中，我们发现的种群具有最低的可利用性和最高的种群有效性;在\textit{Google Research Football}中，我们发现的种群在与不同水平的对手进行交互时具有最大的目标差。
Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization
Manning College of Information & Computer Sciences, University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校信息与计算机科学学院）；University of British Columbia（不列颠哥伦比亚大学）；Vector Institute（矢量研究所）；Amherst College（阿默斯特学院）；Stochastic Labs（随机实验室）；Stability AI（Stability AI公司）
An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue
Protolife, Inc.（普罗托生命公司）；Reed College（里德学院）；Keele University（基尔大学）；University of Tokyo（东京大学）；University of Southern Denmark（南丹麦大学）；Santa Fe Institute（圣塔菲研究所）；Uber AI Labs（优步人工智能实验室）；Monash University（莫纳什大学）
Exploring open-ended gameplay features with Micro RollerCoaster Tycoon
OriGen.AI, Tandon School of Engineering New York University（纽约大学坦登工程学院OriGen.AI）；Tandon School of Engineering New York University（纽约大学坦登工程学院）；Cross Labs Cross Compass, Ltd.（Cross Compass有限公司Cross Labs）
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Raffles Institution（莱佛士书院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Lee Kong Chian School of Medicine, Nanyang Technological University（南洋理工大学李光前医学院）；Centre of AI in Medicine (C-AIM), Nanyang Technological University（南洋理工大学人工智能医学中心）
Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
University College London（伦敦大学学院）；Huawei R&D U.K.（华为英国研发部）；Huawei Canada（华为加拿大公司）；Shanghai Jiao Tong University（上海交通大学）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家实验室，南京大学）；School of Artificial Intelligence, Nanjing University（人工智能学院，南京大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong, Shenzhen（香港大学深圳校区）；Southern University of Science and Technology（南方科技大学）
