集体智慧导论
An Introduction to Collective Intelligence
摘要：
本文综述了如何设计一个“集体智能”（COIN）的新兴科学。COIN是一个大型多智能体系统，其中：（i）几乎没有集中式通信或控制;（ii）提供了一个世界效用函数，对整个系统的可能历史进行评级。特别是，我们对每个代理运行强化学习（RL）算法的COIN感兴趣。而不是使用传统的建模方法（例如，模型的系统动力学，并手动调整代理合作），我们的目标是解决硬币设计问题隐式，通过“自适应”的RL算法的每个代理的字符。这种方法引入了一个全新的，深刻的设计问题：假设RL算法能够实现高奖励，当这些代理追求时，个体代理的奖励函数将导致高世界效用？换句话说，什么样的奖励函数能最好地确保我们不会出现像公地悲剧、布雷斯悖论或流动性陷阱这样的现象？虽然还很年轻，但专门专注于硬币设计问题的研究已经在人工领域取得了成功，特别是在分组路由，领导者-追随者问题以及亚瑟的El Farol酒吧问题的变体中。预计随着它的成熟并借鉴与COIN相关的其他学科，这项研究将大大扩展人类工程师可解决的任务范围。此外，除了借鉴它们之外，这种完全发展的硬币设计科学还可以为其他已经建立的科学领域提供更多的见解，例如经济学，博弈论和人口生物学。
英文摘要：
This paper surveys the emerging science of how to design a ``COllective INtelligence'' (COIN). A COIN is a large multi-agent system where: (i) There is little to no centralized communication or control; and (ii) There is a provided world utility function that rates the possible histories of the full system. In particular, we are interested in COINs in which each agent runs a reinforcement learning (RL) algorithm. Rather than use a conventional modeling approach (e.g., model the system dynamics, and hand-tune agents to cooperate), we aim to solve the COIN design problem implicitly, via the ``adaptive'' character of the RL algorithms of each of the agents. This approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research specifically concentrating on the COIN design problem has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed scie nce of COIN design may provide much insight into other already established scientific fields, such as economics, game theory, and population biology.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
