Ready2Blend：从自然语言指令到可组合的对齐提示
Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts
浏览论文内容
中文总结 AI 辅助
提出Ready2Blend，结合自然语言灵活性与学习对齐，通过AlignFormer和可组合性正则化实现冻结骨干下的持续对齐，匹配训练后调整方法性能，减少训练时间并支持个性化组合。
中文摘要 AI 辅助
持续对齐要求大语言模型在适应新需求的同时不遗忘先前习得的行为。自然语言指令灵活且可组合，但仅提供间接控制，而训练后调整虽能提供更强的适应性，却以重复的参数更新为代价。我们提出Ready2Blend，它结合了自然语言的灵活性与学习到的对齐。AlignFormer将每个需求映射为存储在模块化提示库中的固定长度对齐提示，同时保持骨干网络和先前提示冻结。可组合性正则化将文本需求的语义几何结构迁移到提示空间，从而实现推理时的混合与重新加权。在两种实际的持续对齐设置中，Ready2Blend是唯一能匹配基于训练后调整的对齐方法的冻结骨干方法，达到了联合训练参考的$93.1$-$98.5\%$，并具有竞争力的保持性能，同时仅需少量提示标记和最多$4.3$倍的训练时间减少。其模块化设计还支持加权个性化和无需重训练的无序组合。代码将在录用后发布。
英文摘要
Continual alignment requires LLMs to adapt to new requirements without forgetting previously acquired behaviors. Natural-language instructions are flexible and composable but offer only indirect control, whereas post-training provides stronger adaptation at the cost of repeated parameter updates. We introduce Ready2Blend, which combines the flexibility of natural language with learned alignment. AlignFormer maps each requirement to a fixed-length alignment prompt stored in a modular prompt bank, while the backbone and prior prompts remain frozen. Composability regularization transfers the semantic geometry of textual requirements into prompt space, enabling inference-time blending and reweighting. Across two practical continual alignment settings, Ready2Blend is the only frozen-backbone method that matches post-training-based alignment methods, reaching $93.1$-$98.5\%$ of a joint-training reference with competitive retention, while requiring only a few prompt tokens and up to $4.3\times$ less training time. Its modular design further enables weighted personalization and order-free composition without retraining. Code will be released upon acceptance.
发表机构
- Korea Advanced Institute of Science Technology（韩国科学技术院）
- Samsung SDS（三星SDS）
机构由 AI 辅助整理，请以论文原文为准。
