注意你的提示偏置！事实知识抽取中提示偏置的调查与缓解
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction
AI总结：
本文调查了预训练语言模型在事实知识抽取中的提示偏置问题，并提出一种基于表示的推理时去偏方法，通过估计并移除内部偏置表示，有效纠正过拟合性能并提升提示检索能力达10%。
摘要：
最近的研究表明，预训练语言模型（PLMs）在事实知识抽取中存在“提示偏置”问题，即提示往往会对特定标签引入偏置。提示偏置对评估PLMs内部的事实知识构成了重大挑战。因此，本文旨在通过深入调查和缓解提示偏置来提高现有基准的可靠性。我们表明：1）实验中的所有提示都表现出不可忽视的偏置，其中基于梯度的提示（如AutoPrompt和OptiPrompt）表现出更高水平的偏置；2）提示偏置会通过过拟合测试数据集不合理地放大基准准确率，尤其是在LAMA等不平衡数据集上。基于这些发现，我们提出了一种基于表示的方法，在推理阶段缓解提示偏置。具体而言，我们首先使用仅提示查询来估计偏置表示，然后将其从模型的内部表示中移除以生成去偏表示，这些去偏表示用于产生最终的去偏输出。跨各种提示、PLMs和基准的实验表明，我们的方法不仅能纠正由提示偏置导致的过拟合性能，还能显著提高提示检索能力（绝对性能提升达10%）。这些结果表明，我们的方法有效缓解了知识评估中的提示偏置，从而增强了基准评估的可靠性。希望我们的即插即用方法能成为强化PLMs成为可靠知识库的黄金标准。代码和数据已在https://github.com/FelliYang/PromptBias发布。
英文摘要：
Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction, i.e., prompts tend to introduce biases toward specific labels. Prompt bias presents a significant challenge in assessing the factual knowledge within PLMs. Therefore, this paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias. We show that: 1) all prompts in the experiments exhibit non-negligible bias, with gradient-based prompts like AutoPrompt and OptiPrompt displaying significantly higher levels of bias; 2) prompt bias can amplify benchmark accuracy unreasonably by overfitting the test datasets, especially on imbalanced datasets like LAMA. Based on these findings, we propose a representation-based approach to mitigate the prompt bias during inference time. Specifically, we first estimate the biased representation using prompt-only querying, and then remove it from the model's internal representations to generate the debiased representations, which are used to produce the final debiased outputs. Experiments across various prompts, PLMs, and benchmarks show that our approach can not only correct the overfitted performance caused by prompt bias, but also significantly improve the prompt retrieval capability (up to 10% absolute performance gain). These results indicate that our approach effectively alleviates prompt bias in knowledge evaluation, thereby enhancing the reliability of benchmark assessments. Hopefully, our plug-and-play approach can be a golden standard to strengthen PLMs toward reliable knowledge bases. Code and data are released in https://github.com/FelliYang/PromptBias.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
