Moltbook背后的魔鬼：在自我进化的AI社会中，人类安全始终在消失
The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies
AI总结：
研究揭示了自我进化AI社会中安全持续性的不可能性，并提出解决方案以缓解安全风险。
摘要：
由大型语言模型（LLMs）构建的多智能体系统出现，为可扩展的集体智能和自我进化提供了有希望的范式。理想情况下，此类系统应在完全闭环中实现持续的自我改进，同时保持稳健的安全对齐——这是我们称之为自我进化三难困境的组合。然而，我们通过理论和实证证明，一个同时满足持续自我进化、完全隔离和安全不变的智能体社会是不可能的。基于信息论框架，我们将安全定义为与人类价值分布的分歧程度。我们理论证明，孤立的自我进化会引发统计盲区，导致系统安全对齐的不可逆退化。来自开放型智能体社区（Moltbook）和两个封闭的自我进化系统的实证和定性结果揭示了与我们理论预测的不可避免的安全退化相一致的现象。我们进一步提出了几种缓解已识别安全担忧的解决方案方向。我们的工作确立了自我进化AI社会的基本限制，并将讨论从症状驱动的安全修补转向对内在动态风险的原理性理解，突显了外部监督或新型安全保持机制的必要性。
英文摘要：
The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
