Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts, limiting their ability to maintain coherence and accuracy over long sequences. In contrast, the human brain excels at organising and retrieving episodic experiences across vast temporal scales, spanning a lifetime. In this work, we introduce EM-LLM, a novel approach that integrates key aspects of human episodic memory and event cognition into LLMs with no fine-tuning, enabling them to handle practically infinite context lengths while maintaining computational efficiency. EM-LLM organises sequences of tokens into coherent episodic events using a combination of Bayesian surprise and graph-theoretic boundary refinement in an online fashion. When needed, these events are retrieved through a two-stage memory process, combining similarity-based and temporally contiguous retrieval for efficient, human-inspired access to relevant information. Experiments on the LongBench and $\infty$-Bench benchmarks demonstrate EM-LLM's superior performance, consistently outperforming the state-of-the-art retrieval model InfLLM across various baseline LLMs. In addition, EM-LLM outperforms its popular counterpart, RAG, in a wide range of tasks, while requiring similar resources. Notably, EM-LLM's performance even surpasses full-context models in most tasks, while successfully performing retrieval across 10 million tokens -- a scale computationally infeasible for such models. Finally, our analysis reveals strong correlations between EM-LLM's event segmentation and human-perceived events, suggesting parallels between this artificial system and its biological counterpart, thereby offering a novel computational framework for exploring human memory mechanisms.
Echo: A Large Language Model with Temporal Episodic Memory
Institute of AI Education, East China Normal University(华东师范大学人工智能教育学院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks
Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);Intel Labs(英特尔实验室);University of Texas at Austin(德克萨斯大学奥斯汀分校);Princeton University(普林斯顿大学);Princeton Neuroscience Institute(普林斯顿神经科学研究所);New York University(纽约大学)
Role-aware Heuristic Episodic Attention for Conversational LLMs
National Key Laboratory of Parallel and Distributed Computing(平行与分布式计算国家重点实验室);College of Computer Science and Technology(计算机科学与技术学院);National University of Defense Technology(国防科学技术大学)
Role-aware Heuristic Episodic Attention for Conversational LLMs
National Key Laboratory of Parallel and Distributed Computing(并行与分布式计算国家重点实验室);College of Computer Science and Technology(计算机科学与技术学院);National University of Defense Technology(国防科技大学)
Video-EM: Event-Centric Episodic Memory for Long-Form Video Understanding
City University of Hong Kong(香港城市大学);University of Science and Technology of China(中国科学技术大学);Tianjin University(天津大学);Nanjing University(南京大学);Zhejiang University(浙江大学);The Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究所(TeleAI))
If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs
University of Electronic Science and Technology of China(电子科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统认知与决策智能重点实验室);Nanyang Technological University(南洋理工大学);Peking University(北京大学);Spin Matrix, China(中国Spin Matrix公司)
Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AI
GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(整合视觉与语言实验室,电气工程学院,韩国科学技术院(KAIST));Visual Intelligence Research Section, Superintelligence Creative Research Laboratory, Electronics and Telecommunications Research Institute (ETRI)(视觉智能研究部,超智能创意研究实验室,电子电信研究院)
UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Meituan(美团);AI Centre, Department of Computer Science, University College London(伦敦大学学院计算机科学系人工智能中心)
SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation
School of Computing, University of Georgia(佐治亚大学计算机学院);Department of Biosystems Engineering and Soil Science, University of Tennessee(田纳西大学生物系统工程与土壤科学系);Department of Biomedical Informatics, University of Colorado School of Medicine(科罗拉多医学院生物医学信息学系);Department of Biomedical Engineering, New Jersey Institute of Technology(新泽西理工学院生物医学工程系);Department of Physics and Astronomy, The University of Georgia(佐治亚大学物理与天文学系)
An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs
State Key Laboratory of AI Safety(人工智能安全国家重点实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models
School of Automation, Northwestern Polytechnical University, Xi’an, China(自动化学院,西北工业大学,西安,中国);School of Computing, University of Georgia, Athens, USA(计算机学院,佐治亚大学,亚特兰大,美国);School of Physics and Information Technology, Shaanxi Normal University, Xi’an, China(物理与信息技术学院,陕西师范大学,西安,中国)
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models
Department of Informatics, Universität Hamburg(汉堡大学信息学院);The University of Sydney(悉尼大学);Alibaba International Digital Commerce(阿里巴巴国际数字商业);Institute of Psychology, Chinese Academy of Sciences(中国科学院心理研究所)