多智能体系统中的涌现:一个安全视角
Emergence in Multi-Agent Systems: A Safety Perspective
AI总结:
本文从安全视角研究多智能体系统中的涌现效应,通过识别全局规范与局部近似之间的错位建立分析框架,并提出调整底层参数化以缓解非预期行为偏差和降低涌现故障风险。
摘要:
在多智能体系统(MAS)中,由于执行是去中心化的且依赖于局部信息,可能会出现涌现效应。这些效应可能从轻微的行为偏差到灾难性的系统故障不等。为了正式定义这些效应,我们识别了全局固有规范(真实规范)与其局部近似(例如不同奖励分量或观测的配置)之间的错位。利用既有的安全术语,我们开发了一个框架来理解这些涌现效应。为了展示由此产生的含义,我们使用了两个可广泛配置的示例网格世界场景,在这些场景中,当独立推导时,不充分的规范会导致非预期的行为偏差。认识到全局适应可能并不总是可行的,我们提出调整底层参数化以缓解这些问题,从而提高系统的一致性并降低涌现故障的风险。
英文摘要:
Emergent effects can arise in multi-agent systems (MAS) where execution is decentralized and reliant on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these effects, we identify misalignments between the global inherent specification (the true specification) and its local approximation (such as the configuration of different reward components or observations). Using established safety terminology, we develop a framework to understand these emergent effects. To showcase the resulting implications, we use two broadly configurable exemplary gridworld scenarios, where insufficient specification leads to unintended behavior deviations when derived independently. Recognizing that a global adaptation might not always be feasible, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving the system's alignment and reducing the risk of emergent failures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。