arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.11482cs.CLcs.IR
收藏

语言模型中的事实知识追溯到训练数据

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

Google Research(谷歌研究院);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)

Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu

摘要:

语言模型(LM)已经被证明能够记忆包含在其训练数据中的大量事实知识。但是当LM生成断言时,通常很难确定它是从哪里得知这些信息的,以及这些信息是否正确。在本文中,我们提出了事实跟踪的问题:确定哪些训练样本教LM生成一个特定的事实断言。先前关于训练数据属性的工作可为确定这类被称为“支持者”的实例提供有效的工具。我们提出了第一个定量基准来评估这一点。我们比较了两种流行的TDA方法-基于梯度和基于嵌入-并发现仍然存在很大的净空。例如,这两种方法都具有比根本不能访问LM的信息检索基线(BM 25)更低的支持者检索精度。我们确定了可能需要进一步改进的关键挑战,例如克服梯度饱和问题,并展示了现有神经TDA方法的几个细微的实现细节如何显着提高整体事实跟踪性能。

英文摘要:

Language models (LMs) have been shown to memorize a great deal of factual knowledge contained in their training data. But when an LM generates an assertion, it is often difficult to determine where it learned this information and whether it is true. In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion. Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". We present the first quantitative benchmark to evaluate this. We compare two popular families of TDA methods -- gradient-based and embedding-based -- and find that much headroom remains. For example, both methods have lower proponent-retrieval precision than an information retrieval baseline (BM25) that does not have access to the LM at all. We identify key challenges that may be necessary for further improvement such as overcoming the problem of gradient saturation, and also show how several nuanced implementation details of existing neural TDA methods can significantly improve overall fact tracing performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑