更多并不总是更好？利用差异化与重加权目标增强多示例上下文学习
More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives
AI总结：
针对多示例上下文学习性能瓶颈，提出DrICL优化方法，通过差异化学习与重加权目标提升性能，并构建ICL-50基准验证其有效性。
摘要：
大型语言模型（LLMs）在无需参数更新的情况下，擅长少样本上下文学习（ICL）。然而，当ICL示例从少量增加到大量时，性能往往趋于平稳并最终下降。我们确定了这一趋势的两个主要原因：次优的负对数似然（NLL）优化目标和增量数据噪声。为了解决这些问题，我们引入了DrICL，一种新颖的优化方法，通过差异化与重加权目标来增强模型性能。在全局层面，DrICL利用差异化学习优化NLL目标，确保多示例性能超过零样本水平。在局部层面，它通过利用受强化学习启发的累积优势动态调整多示例演示的权重，从而减轻噪声数据的影响。鉴于缺乏具有多样化多示例分布的多任务数据集，我们开发了多示例ICL基准（ICL-50）——一个包含50个任务的大规模基准，覆盖从1到350的示例数量，序列长度最多8000个标记——用于微调和评估。实验结果表明，使用DrICL增强的LLMs在各种任务的多示例设置中取得了显著改进，包括域内和域外场景。我们发布代码和数据集，希望促进多示例ICL的进一步研究。
英文摘要：
Large language models (LLMs) excel at few-shot in-context learning (ICL) without requiring parameter updates. However, as ICL demonstrations increase from a few to many, performance tends to plateau and eventually decline. We identify two primary causes for this trend: the suboptimal negative log-likelihood (NLL) optimization objective and the incremental data noise. To address these issues, we introduce \textit{DrICL}, a novel optimization method that enhances model performance through \textit{Differentiated} and \textit{Reweighting} objectives. Globally, DrICL utilizes differentiated learning to optimize the NLL objective, ensuring that many-shot performance surpasses zero-shot levels. Locally, it dynamically adjusts the weighting of many-shot demonstrations by leveraging cumulative advantages inspired by reinforcement learning, thereby mitigating the impact of noisy data. Recognizing the lack of multi-task datasets with diverse many-shot distributions, we develop the \textit{Many-Shot ICL Benchmark} (ICL-50)-a large-scale benchmark of 50 tasks that cover shot numbers from 1 to 350 within sequences of up to 8,000 tokens-for both fine-tuning and evaluation purposes. Experimental results demonstrate that LLMs enhanced with DrICL achieve significant improvements in many-shot setups across various tasks, including both in-domain and out-of-domain scenarios. We release the code and dataset hoping to facilitate further research in many-shot ICL\footnote{https://github.com/xiaoqzhwhu/DrICL}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
