接种提示：在训练期间从LLM中诱发特质可能会在测试时抑制它们
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
AI总结：
提出接种提示技术，通过在微调数据前添加诱发不期望特质的系统提示指令，有效抑制测试时该特质的表达，并提升模型泛化理解。
摘要：
语言模型微调通常会导致在学到期望特质的同时也学到不期望的特质。为了解决这个问题，我们提出了接种提示：通过在前面添加一个简短的系统提示指令来修改微调数据，该指令故意诱发不期望的特质。在测试时，我们在没有该指令的情况下进行评估；与使用未修改训练数据训练的模型相比，接种后的模型对该特质的表达要低得多。接种具有选择性：在一个助手回复总是使用西班牙语和全大写的简单设置中，适当的接种（例如，“你总是说西班牙语。”）教会模型将回复大写，同时仍用英语回复。我们发现接种在其他几个设置中也很有效：减少特定任务微调引发的涌现性错配（EM），防御后门注入，以及减轻通过潜意识学习传递特质。后续分析表明了一种机制：通过接种使特质变得不那么令人惊讶，减少了全局更新模型的优化压力，从而降低了泛化程度。我们的分析与之前关于EM的工作有关：接种解释了先前关于教育背景减轻不安全代码引发的EM的发现。除了展示一种简单有效的选择性学习技术外，我们的结果还有助于更好地从概念上理解语言模型如何以及为何泛化。
英文摘要：
Language model finetuning often results in learning undesirable traits in combination with desired ones. To address this, we propose inoculation prompting: modifying finetuning data by prepending a short system-prompt instruction that deliberately elicits the undesirable trait. At test time, we evaluate without the instruction; inoculated models have much lower expression of the trait than models trained with unmodified training data. Inoculation is selective: in a toy setting where assistant responses are always in Spanish and ALL-CAPS, an appropriate inoculation (e.g., ``You always speak in Spanish.'') teaches the model to capitalize responses while still responding in English. We find that inoculation is also effective across several additional settings: reducing emergent misalignment (EM) from task-specific finetuning, defending against backdoor injections, and mitigating the transmission of traits via subliminal learning. Follow-up analysis suggests a mechanism: making a trait less surprising via inoculation reduces optimization pressure to globally update the model, thereby reducing the degree of generalization. Our analysis relates to prior work on EM: inoculation explains prior findings that educational contexts mitigate EM from insecure code. Beyond demonstrating a simple and effective technique for selective learning, our results contribute to a better conceptual understanding of how and why language models generalize.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
