NoisyICL：模型参数中的少量噪声校准了In-context Learning
NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning
AI总结：
针对ICL性能不佳和欠校准问题，提出NoisyICL方法，通过随机噪声扰动模型参数，在多个数据集上有效提升了预测准确度、公平性和置信度可靠性。
摘要：
In-Context Learning (ICL) 由于高先验偏差和不可靠的置信度，正面临性能不佳和欠校准的问题。一些先前的工作通过海量数据集和计算成本对语言模型进行微调以提升 ICL 性能。在本文中，我们提出 NoisyICL，通过随机噪声扰动模型参数来追求更好的性能和校准。我们在两个模型和12个下游数据集上的实验表明，NoisyICL 能帮助 ICL 产生更准确的预测。进一步分析表明，NoisyICL 使模型能提供更公平的预测，并具有更可靠的置信度。因此，我们相信 NoisyICL 是对 ICL 的有效校准。我们的实验代码已上传至 Github。
英文摘要：
In-Context Learning (ICL) is suffering from unsatisfactory performance and under-calibration due to high prior bias and unfaithful confidence. Some previous works fine-tuned language models for better ICL performance with enormous datasets and computing costs. In this paper, we propose NoisyICL, simply perturbing the model parameters by random noises to strive for better performance and calibration. Our experiments on two models and 12 downstream datasets show that NoisyICL can help ICL produce more accurate predictions. Our further analysis indicates that NoisyICL enables the model to provide more fair predictions, and also with more faithful confidence. Therefore, we believe that NoisyICL is an effective calibration of ICL. Our experimental code is uploaded to Github.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
