智能体安全是认知属性,而非行为属性
Agentic Safety is an Epistemic Property, Not a Behavioral One
AI总结:
本文提出AI安全应视为认知属性,强调系统需保持可教性(teachability),即在学习、适应和自修改过程中仍能被纠正。
摘要:
当代AI安全涵盖了预训练干预、后训练对齐、部署时控制、监控和红队测试。这些方法是必要的,但它们主要认证系统行为的快照。随着AI系统变得更有能力、动态、具身化和自我改进,这种快照视图变得不完整:安全不仅取决于系统当前是否表现可接受,还取决于它在学习、适应、行动和随时间自我修改时是否保持可纠正性。本文认为,安全因此应被视为演化学习者的认知属性,而不仅仅是当前策略的行为属性。我们引入可教性(teachability)作为在有界的人类、机构或环境干预下保持未来纠正杠杆的能力。我们认为,高级系统可以在保持可见能力的同时侵蚀未来纠正所需的表征、算法或元决策条件。安全的高级AI系统不仅必须现在表现可接受;它们必须在以后保持可教性。
英文摘要:
Contemporary AI safety spans pre-training interventions, post-training alignment, deployment-time controls, monitoring, and red-teaming. These methods are necessary, but they primarily certify snapshots of system behavior. As AI systems become more capable, dynamic, embodied, and self-improving, this snapshot view becomes incomplete: safety depends not only on whether a system behaves acceptably now, but whether it remains correctable as it learns, adapts, acts, and modifies itself over time. This paper argues that safety should therefore be treated as an epistemic property of the evolving learner, not merely a behavioral property of the current policy. We introduce teachability as the capacity to preserve future corrective leverage under bounded human, institutional, or environmental intervention. We argue that advanced systems can retain visible competence while eroding the representational, algorithmic, or meta-decision conditions needed for future correction. Safe advanced AI systems must not only behave acceptably now; they must remain teachable later.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。