Language models (LMs) have been shown to memorize a great deal of factual knowledge contained in their training data. But when an LM generates an assertion, it is often difficult to determine where it learned this information and whether it is true. In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion. Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". We present the first quantitative benchmark to evaluate this. We compare two popular families of TDA methods -- gradient-based and embedding-based -- and find that much headroom remains. For example, both methods have lower proponent-retrieval precision than an information retrieval baseline (BM25) that does not have access to the LM at all. We identify key challenges that may be necessary for further improvement such as overcoming the problem of gradient saturation, and also show how several nuanced implementation details of existing neural TDA methods can significantly improve overall fact tracing performance.
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Center for Information and Language Processing, LMU Munich(信息与语言处理中心,慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心(MCML));Bosch Center for Artificial Intelligence(博世人工智能中心);Sorbonne Université, CNRS, ISIR, France(索邦大学,法国国家科学研究中心,ISIR)
Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
Shanxi University(山西大学);Nanjing University of Science and Technology(南京理工大学);University of Manchester(曼彻斯特大学);Singapore University of Technology and Design(新加坡科技设计大学);University of Edinburgh(爱丁堡大学)
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Center for Information and Language Processing (CIS)(信息与语言处理中心);Ubiquitous Knowledge Processing Lab (UKP)(无所不在的知识处理实验室);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);SynthLabs;Carnegie Mellon University(卡内基梅隆大学)
Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively
University of Washington(华盛顿大学);Allen Institute for AI(艾伦人工智能研究所);LMU Munich(慕尼黑大学);UC Berkeley(加州大学伯克利分校);Stanford University(斯坦福大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Jarvis Research Center(贾维斯研究中心);Tencent YouTu Lab(腾讯优图实验室);Peking University(北京大学)
TrackList: Tracing Back Query Linguistic Diversity for Head and Tail Knowledge in Open Large Language Models
ATILF, University of Lorraine - CNRS, France(ATILF,洛林大学 - CNRS,法国);Center for Data Science in Humanities, Chosun University, South Korea(人文数据科学中心,全州大学,韩国)
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Center for Language and Cognition, University of Groningen(格罗宁根大学语言与认知中心);Institute for Logic, Language and Computation, University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所)
Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluation
National Biomarker Centre, CRUK-MI, Univ. of Manchester(英国曼彻斯特大学CRUK-MI国家生物标志物中心);Idiap Research Institute(Idiap研究所);Inflamalps SA(因弗马尔普斯公司);Department of Computer Science, Univ. of Manchester(英国曼彻斯特大学计算机科学系)