上下文-参数反转：为什么指令微调会恶化上下文依赖
Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
AI总结：
研究发现指令微调中知识冲突下的上下文依赖先增后减，称为上下文-参数反转，源于数据中上下文与参数知识一致，并提出初步缓解策略。
摘要：
使用大型语言模型时，用户通常会补充一个包含新信息的输入上下文，以供模型处理。然而，模型难以可靠地遵循输入上下文，尤其是在其与预训练得到的参数知识冲突时。原则上，人们会期望模型在指令微调后能更好地适应用户上下文，特别是在处理知识冲突时。然而，我们观察到一个令人惊讶的失败模式：在指令微调期间，知识冲突下的上下文依赖起初如预期般增加，但随后随着指令微调进程逐渐下降。与此同时，标准基准上的性能在该下降之后仍持续提升。我们将此现象称为上下文-参数反转，并在多个通用指令微调数据集（如TULU、Alpaca和Ultrachat）以及不同模型家族（如Llama、Mistral和Pythia）中观察到这一现象。我们进行了多项受控研究和理论分析，表明上下文-参数反转是由于指令微调数据中的某些示例，其输入上下文提供的信息与模型的参数知识一致。我们的分析提出了一些自然缓解策略，虽收益有限但具有启发性，并为解决指令微调中的这一缺陷提供了有用的起点。
英文摘要：
A standard practice when using large language models is for users to supplement their instruction with an input context containing new information for the model to process. However, models struggle to reliably follow the input context, especially when it conflicts with their parametric knowledge from pretraining. In-principle, one would expect models to adapt to the user context better after instruction finetuning, particularly when handling knowledge conflicts. However, we observe a surprising failure mode: during instruction tuning, the context reliance under knowledge conflicts initially increases as expected, but then gradually decreases as instruction finetuning progresses. This happens while the performance on standard benchmarks keeps on increasing far after this drop. We call this phenomenon context-parametric inversion and observe it across multiple general purpose instruction tuning datasets such as TULU, Alpaca and Ultrachat, across different model families like Llama, Mistral, and Pythia. We perform various controlled studies and theoretical analysis to show that context-parametric inversion occurs due to examples in the instruction finetuning data where the input context provides information that aligns with model's parametric knowledge. Our analysis suggests some natural mitigation strategies with limited but insightful gains, and serves as a useful starting point in addressing this deficiency in instruction finetuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
