arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05164cs.LGcs.AI
收藏

位置:能力控制应是与对齐分开的目标

Position: Capability Control Should be a Separate Goal From Alignment

University of Cambridge(剑桥大学);Google DeepMind(谷歌DeepMind);University of Montreal(蒙特利尔大学);The Alan Turing Institute(艾伦·图灵研究所)

Shoaib Ahmed Siddiqui, Eleni Triantafillou, David Krueger, Adrian Weller

AI总结:

本文主张将能力控制视为与对齐不同的目标,提出分层的控制机制以应对模型的潜在滥用和失败。

摘要:

基础模型是在广泛的数据分布上进行训练,从而获得通用能力,这使许多下游应用成为可能,但也扩大了潜在滥用和失败的空间。本文主张将能力控制——对模型行为施加限制——视为与对齐不同的目标。虽然对齐通常由上下文和偏好驱动,但能力控制旨在对可接受的行为施加硬性操作限制,包括在对抗性引导下的情况。我们将能力控制机制按模型生命周期分为三层:(i)基于数据的训练分布控制,(ii)基于学习的控制通过权重或表征层面的干预,(iii)基于系统的控制通过部署后的防护措施对输入、输出和行动进行限制。因为每层在单独使用时都有其特有的失效模式,我们倡导一种纵深防御方法,即在全栈上组合互补的控制措施。我们进一步概述了实现此类控制的关键开放挑战,包括知识的双重用途和组合泛化。

英文摘要:

Foundation models are trained on broad data distributions, yielding generalist capabilities that enable many downstream applications but also expand the space of potential misuse and failures. This position paper argues that capability control -- imposing restrictions on permissible model behavior -- should be treated as a distinct goal from alignment. While alignment is often context and preference-driven, capability control aims to impose hard operational limits on permissible behaviors, including under adversarial elicitation. We organize capability control mechanisms across the model lifecycle into three layers: (i) data-based control of the training distribution, (ii) learning-based control via weight- or representation-level interventions, and (iii) system-based control via post-deployment guardrails over inputs, outputs, and actions. Because each layer has characteristic failure modes when used in isolation, we advocate for a defense-in-depth approach that composes complementary controls across the full stack. We further outline key open challenges in achieving such control, including the dual-use nature of knowledge and compositional generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑