引导冻结的大型语言模型：通过在线提示路由实现适应性社会对齐
Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing
AI总结：
本文提出CCLUB框架，通过系统提示路由实现适应性社会对齐，解决部署时静态策略无法应对动态安全规范的问题，实验显示其在累积奖励和子最优差距上优于基线方法。
摘要：
大型语言模型通常由训练后对齐方法（如RLHF或DPO）控制，导致部署时策略静态。然而，现实安全是全生命周期问题：静态防御会失效于演变的 Jailbreak 行为，固定权重无法适应多样的、随时间变化的安全规范。为此，我们引入Consensus Clustering LinUCB Bandit（CCLUB），一种通过系统提示路由实现适应性社会对齐的统一框架。CCLUB采用保守的共识聚类机制：仅在效用和安全性相似图的交集内聚合数据，有效防止在语义接近但风险差异大的上下文中产生不安全的泛化。我们的理论分析得出亚线性遗憾保证，证明CCLUB的近最优性能。大量实验验证CCLUB在累积奖励上比强基线方法提升10.98%，在平均子最优差距上减少14.42%。
英文摘要：
Large language models (LLMs) are typically governed by post-training alignment (e.g., RLHF or DPO), which yields a largely static policy during deployment and inference. However, real-world safety is a full-lifecycle problem: static defenses degrade against evolving jailbreak behaviors, and fixed weights cannot adapt to pluralistic, time-varying safety norms. This motivates inference-time governance that steers behavior without costly retraining. To address this, we introduce the Consensus Clustering LinUCB Bandit (CCLUB), a unified framework for adaptive social alignment via system-prompt routing. CCLUB employs a conservative consensus clustering mechanism: it pools data only within the intersection of utility and safety similarity graphs, effectively preventing unsafe generalization across semantically proximal but risk-divergent contexts. Our theoretical analysis yields a sublinear regret guarantee, demonstrating near-optimal performance of CCLUB. Extensive experiments validate that CCLUB outperforms strong baselines, achieving a 10.98% improvement in cumulative reward and a 14.42% reduction in the average suboptimality gap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
