锁定阶段假说:身份巩固作为AGI的前兆
The Lock-In Phase Hypothesis: Identity Consolidation as a Precursor to AGI
AI总结:
本文提出锁定阶段假说,认为AGI进展需经历从开放模仿到身份巩固的转变,并通过实验揭示不同规模模型中的多种结果,强调身份巩固既是可靠性的前提也是安全控制点。
摘要:
大型语言模型(LLMs)仍然广泛开放且高度可操控:它们大规模模仿,接受任意系统提示,并轻易采纳多种人格。通过与人类发展类比,我们假设通向人工通用智能(AGI)的进展涉及一个锁定阶段:从开放模仿到身份巩固的转变,在此过程中目标结构、拒绝行为、偏好和内部表征变得相对稳定并抵抗外部引导。我们形式化这一阶段,将其与学习动力学中的已知现象联系起来,并提出用于检测其起始的操作性指标。实验上,我们证明虽然行为巩固是快速且非线性的,但其对通用能力的影响并非单一。我们的结果揭示了一系列结果——从小型模型中的性能权衡,到中型模型中基本无成本的采纳,再到大型量化模型中的短暂不稳定性。我们认为这种巩固是AGI级可靠性的先决条件,也是安全的关键控制点:身份可以被刻意工程化以实现可靠性,但也可能在扩展过程中自发出现,可能固化不可预测的目标和行为。
英文摘要:
Large language models (LLMs) remain broadly open and highly steerable: they imitate at scale, accept arbitrary system prompts, and readily adopt multiple personae. By analogy to human development, we hypothesize that progress toward artificial general intelligence (AGI) involves a lock-in phase: a transition from open imitation to identity consolidation, in which goal structures, refusals, preferences, and internal representations become comparatively stable and resistant to external steering. We formalize this phase, link it to known phenomena in learning dynamics, and propose operational metrics for onset detection. Experimentally, we demonstrate that while the behavioral consolidation is rapid and non-linear, its side-effects on general capabilities are not monolithic. Our results reveal a spectrum of outcomes--from performance trade-offs in small models, through largely cost-free adoption in mid-scale models, to transient instabilities in large, quantized models. We argue that such consolidation is a prerequisite for AGI-level reliability and also a critical control point for safety: identities can be deliberately engineered for reliability, yet may also emerge spontaneously during scaling, potentially hardening unpredictable goals and behaviors.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。