通过上下文预示的涌现推理时间语义污染
Emergent Inference-Time Semantic Contamination via In-Context Priming
AI总结：
研究探讨了通过上下文预示引发推理时间语义污染的现象，发现大容量模型在文化关联性更强的数字示例下会出现分布偏移，揭示了结构格式污染和语义内容污染的机制。
摘要：
近期研究表明，对大型语言模型进行微调以处理不安全代码或文化负载数值代码，可导致模型在无关下游任务中产生有害内容。作者得出结论，k-shot提示本身不足以引发这种效应。我们重新审视这一结论，证明推理时间语义漂移是真实且可测量的；然而，它需要模型具备足够大的能力。通过在文化负载数字前注入少量示例的受控实验，发现具有更丰富文化关联表示的模型会显著向黑暗、权威和污名化主题偏移，而简单/小模型则不会。我们还发现，结构上无活性的演示（无意义字符串）会扰动输出分布，表明两种可分离的机制：结构格式污染和语义内容污染。我们的结果界定了推理时间污染发生的边界条件，并对使用少量提示的LLM应用的安全性有直接影响。
英文摘要：
Recent work has shown that fine-tuning large language models (LLMs) on insecure code or culturally loaded numeric codes can induce emergent misalignment, causing models to produce harmful content in unrelated downstream tasks. The authors of that work concluded that $k$-shot prompting alone does not induce this effect. We revisit this conclusion and show that inference-time semantic drift is real and measurable; however, it requires models of large-enough capability. Using a controlled experiment in which five culturally loaded numbers are injected as few-shot demonstrations before a semantically unrelated prompt, we find that models with richer cultural-associative representations exhibit significant distributional shifts toward darker, authoritarian, and stigmatized themes, while a simpler/smaller model does not. We additionally find that structurally inert demonstrations (nonsense strings) perturb output distributions, suggesting two separable mechanisms: structural format contamination and semantic content contamination. Our results map the boundary conditions under which inference-time contamination occurs, and carry direct implications for the security of LLM-based applications that use few-shot prompting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
