arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.16210cs.AI
收藏

MOSAIC:通过模块化控制令牌实现可组合的安全对齐

MOSAIC: Composable Safety Alignment with Modular Control Tokens

University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Baidu Inc(百度公司);Minzu University of China(民族大学)

Jingyu Peng, Hongyu Chen, Jiancheng Dong, Maolin Wang, Wenxi Li, Yuchen Li, Kai Zhang, Xiangyu Zhao

AI总结:

MOSAIC通过可学习的控制令牌实现可组合的安全对齐,解决传统方法在动态安全规则下的局限性,实验显示其在降低过拒绝率的同时保持模型效用。

摘要:

在大型语言模型(LLMs)中,安全对齐通常实现为嵌入在模型参数中的单一静态策略。然而,现实部署往往需要根据上下文变化的安全规则,这些规则会因用户、地区和应用而异。现有方法难以提供此类条件控制:参数级对齐将安全行为与通用能力纠缠,而基于提示的方法依赖于自然语言指令,提供弱执行。我们提出了MOSAIC,一种模块化框架,通过在冻结主干模型上优化的可学习控制令牌实现可组合的安全对齐。每个令牌代表一个安全约束,并可在推理时灵活激活和组合。为了高效训练可组合令牌,我们引入了基于顺序的任务采样和分布级对齐目标,以缓解过拒绝问题。实验表明,MOSAIC在大幅降低过拒绝率的同时,仍能保持模型效用,展现出强大的防御性能。

英文摘要:

Safety alignment in large language models (LLMs) is commonly implemented as a single static policy embedded in model parameters. However, real-world deployments often require context-dependent safety rules that vary across users, regions, and applications. Existing approaches struggle to provide such conditional control: parameter-level alignment entangles safety behaviors with general capabilities, while prompt-based methods rely on natural language instructions that provide weak enforcement. We propose MOSAIC, a modular framework that enables compositional safety alignment through learnable control tokens optimized over a frozen backbone model. Each token represents a safety constraint and can be flexibly activated and composed at inference time. To train compositional tokens efficiently, we introduce order-based task sampling and a distribution-level alignment objective that mitigates over-refusal. Experiments show that MOSAIC achieves strong defense performance with substantially lower over-refusal while preserving model utility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑