Petroni et al. (2019) demonstrated that it is possible to retrieve world facts from a pre-trained language model by expressing them as cloze-style prompts and interpret the model's prediction accuracy as a lower bound on the amount of factual information it encodes. Subsequent work has attempted to tighten the estimate by searching for better prompts, using a disjoint set of facts as training data. In this work, we make two complementary contributions to better understand these factual probing techniques. First, we propose OptiPrompt, a novel and efficient method which directly optimizes in continuous embedding space. We find this simple method is able to predict an additional 6.4% of facts in the LAMA benchmark. Second, we raise a more important question: Can we really interpret these probing results as a lower bound? Is it possible that these prompt-search methods learn from the training data too? We find, somewhat surprisingly, that the training data used by these methods contains certain regularities of the underlying fact distribution, and all the existing prompt methods, including ours, are able to exploit them for better fact prediction. We conduct a set of control experiments to disentangle "learning" from "learning to recall", providing a more detailed picture of what different prompts can reveal about pre-trained language models.
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Wenge Technology Co., Ltd(北京文格科技有限公司)
ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall
HKUST(GZ) Deep Interdisciplinary Intelligence Lab(香港科技大学(广州)深 interdisciplinary 智能实验室);BUAA(北京航空航天大学);HKUST(GZ) Institute of Deep Perception Technology, JITRI Deep Interdisciplinary Intelligence Lab(香港科技大学(广州)深度感知技术研究所,JITRI 深 interdisciplinary 智能实验室)
Tracing Multilingual Factual Knowledge Acquisition in Pretraining
Center for Information and Language Processing, LMU Munich(信息与语言处理中心,慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心(MCML));Bosch Center for Artificial Intelligence(博世人工智能中心);Sorbonne Université, CNRS, ISIR, France(索邦大学,法国国家科学研究中心,ISIR)
Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency
International Development Research Centre Canada(加拿大国际发展研究中心);University of Cape Town(开普敦大学);Global Center on AI Governance(人工智能治理全球中心);SnT, University of Luxembourg(卢森堡大学SnT分校);CITADEL AI Centre of Excellence, Burkina Faso(布基纳法索CITADEL人工智能卓越中心)
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Peking University(北京大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可验证负责任人工智能与数据分析(PRADA)实验室);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);South China University of Technology(华南理工大学)
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
在Petroni等人(2019)的基础上,我们提出了两个新的探测任务,分析存储在预训练语言模型(PLM)中的事实知识。(1)否定。我们发现,PLM没有区分否定型(“Birds cannot [MASK]”)和非否定型(“Birds can [MASK]”)完形填空题。(2)启动错误。受人类心理学中启动方法的启发,我们在完形填空题中加入了“误启动”(“谈话?鸟类可以[面具]”)。我们发现,PLM很容易被误启动分心。这些结果表明,PLM仍然有很长的路要走,充分学习人类一样的事实知识。
TiEBe: Tracking Language Model Recall of Notable Worldwide Events Through Time
Institute of Computing (IC) State University of Campinas(计算研究所(IC)坎皮纳斯州立大学);Maritaca AI(马里塔卡人工智能);State University of Campinas(坎皮纳斯州立大学);Campinas, SP, Brazil(巴西坎皮纳斯州)
StratMem-Bench: Evaluating Strategic Memory Use in Virtual Character Conversation Beyond Factual Recall
School of Computer Science and Engineering, Southeast University, China(东南大学计算机科学与工程学院,中国);Key Laboratory of New Generation Artificial Intelligence Technology and its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其交叉应用国家重点实验室(东南大学),中华人民共和国教育部,中国);College of Design and Innovation, Tongji University, China(同济大学设计与创新学院,中国)
Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
Quality and Usability Lab, Technische Universität Berlin(柏林技术大学质量与可用性实验室);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Saarland Informatics Campus(萨尔州信息学院);LMU Munich(慕尼黑大学);Bosch Center for Artificial Intelligence (BCAI)(博世人工智能中心);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Centre for European Research in Trusted AI (CERTAIN)(可信人工智能欧洲研究中心);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院);Technical University of Munich(慕尼黑技术大学)
Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling
Ubiquitous Knowledge Processing Lab (UKP Lab)(通用知识处理实验室);Department of Computer Science(计算机科学系);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Technical University of Darmstadt(达姆施塔特技术大学)
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Cyberspace Security, Southeast University(东南大学网络空间安全学院)
Center for Information and Language Processing (CIS), LMU Munich(慕尼黑大学语言与信息处理中心(CIS))
最近的研究调查了存储在大型预训练语言模型(PLM)中的事实知识。代替结构化知识库(KB)查询,使用诸如“Paris is the capital of [MASK]”之类的掩码语句作为探测。这种分析任务的良好表现被解释为PLM成为事实知识的潜在存储库。在10种不同语言的实验中,我们研究了静态嵌入中包含的知识。我们表明,当限制输出空间的候选集,简单的最近邻匹配使用静态嵌入比PLM的表现更好。例如,在一个示例中,静态嵌入的性能比BERT高1.6%,而仅使用0.3%的能量进行训练。它们良好的比较性能的一个重要因素是,静态嵌入对于大词汇表是标准的。相比之下,BERT利用其更复杂,但昂贵的能力,从一个小得多的子词词汇组成有意义的表示。
Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);University of Electronic Science and Technology of China(电子科技大学)