Current large language models (LLMs) often perform poorly on simple fact retrieval tasks. Here we investigate if coupling a dynamically adaptable external memory to a LLM can alleviate this problem. For this purpose, we test Larimar, a recently proposed language model architecture which uses an external associative memory, on long-context recall tasks including passkey and needle-in-the-haystack tests. We demonstrate that the external memory of Larimar, which allows fast write and read of an episode of text samples, can be used at test time to handle contexts much longer than those seen during training. We further show that the latent readouts from the memory (to which long contexts are written) control the decoder towards generating correct outputs, with the memory stored off of the GPU. Compared to existing transformer-based LLM architectures for long-context recall tasks that use larger parameter counts or modified attention mechanisms, a relatively smaller size Larimar is able to maintain strong performance without any task-specific training or training on longer contexts.
Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find
Center for Alzheimer’s Disease and Related Dementias, NIA, NIH（阿尔茨海默病及相关痴呆症研究中心，国家老龄化研究所，国家卫生研究院）；DataTecnica LLC（DataTecnica公司）；Johns Hopkins University（约翰霍普金斯大学）；Laboratory of Neurogenetics, NIA, NIH（神经遗传学实验室，国家老龄化研究所，国家卫生研究院）
Fudan University（复旦大学）；OpenGVLab（开放视觉实验室）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；Nanjing University（南京大学）；SenseTime Research（商汤科技研究院）
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
Laboratoire de Physique de l’Ecole normale supérieure ENS（巴黎高等师范学院物理实验室）；Université PSL（巴黎文理研究大学）；CNRS（法国国家科学研究中心）；Sorbonne Université（索邦大学）；Université Paris-Diderot（巴黎第七大学）；Sorbonne Paris Cité（巴黎索邦西岱大学）；Facebook AI Research（脸书人工智能研究院）；Facebook（脸书）；Courant Institute of Mathematical Sciences and Center for Data Science（库朗数学科学研究所与数据科学中心）；New York University（纽约大学）
NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
NeMo: Needle in a Montage for Video-Language Understanding
The Chinese University of Hong Kong（香港中文大学）；Phoenix TV（凤凰电视台）；Stanford University（斯坦福大学）；University of Liverpool（利物浦大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation
Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico（高等技术学院）；LUMLIS (Lisbon ELLIS Unit)（LUMLIS（里斯本ELLIS单元））；University of Edinburgh（爱丁堡大学）；Unbabel（Unbabel公司）
Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization
New York University（纽约大学）；ADAPT Centre, DCU（都柏林城市大学ADAPT中心）；IBM Research（IBM研究院）；Google（谷歌）；Google Research（谷歌研究院）；Yale University（耶鲁大学）；trivago N.V.（途易集团）；University of Edinburgh（爱丁堡大学）；Allen Institute for AI（艾伦人工智能研究所）
U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（南京航空航天大学人工智能学院）；College of Design and Innovation, Tongji University（同济大学设计与创新学院）
LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
A Controllable Examination for Long-Context Language Models
University of Edinburgh（爱丁堡大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Nanjing University（南京大学）；Alibaba Group（阿里巴巴集团）；University of Amsterdam（阿姆斯特丹大学）
SR-TTT Does Not Learn Retrieval: A Correction and Mechanistic Post-Mortem of Surprisal-Aware Residual Test-Time Training
Swamynathan V P Department of Computer Science and Engineering Amrita School of Computing Amrita Vishwa Vidyapeetham Chennai, Tamil Nadu, India（斯瓦米纳坦·V·P 计算机科学与工程系 阿米塔学校 阿米塔大学 芬治, 印度泰米尔纳德邦）
SEGMENT+: Long Text Processing with Short-Context Language Models
Fudan University（复旦大学）；Columbia University（哥伦比亚大学）；MYbank, Ant Group（蚂蚁集团MYbank）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Data Science, Fudan University（复旦大学数据科学学院）
