语言大模型 / LLM - arXivDaily 专题

2407.09450 2025-10-14 cs.AI cs.CL cs.LG q-bio.NC 80%

Human-inspired Episodic Memory for Infinite Context LLMs

Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, Jun Wang

机构 * Huawei Noah’s Ark Lab（华为诺亚实验室）； University College London（伦敦大学学院）

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref Proc. International Conference on Learning Representations (ICLR), 2025

2410.11001 2025-05-30 cs.CL cs.AI cs.LG 80%

Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs

Haozhen Zhang, Tao Feng, Jiaxuan You

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ACL 2025 Main. The code is available at https://github.com/ulab-uiuc/GoR

2502.16747 2025-05-21 cs.CL cs.AI cs.LG cs.SE 80%

SQLong: Enhanced NL2SQL for Longer Contexts with LLMs

Dai Quoc Nguyen, Cong Duy Vu Hoang, Duy Vu, Gioacchino Tangari, Thanh Tien Vu, Don Dharmasiri, Yuan-Fang Li, Long Duong

机构 * Oracle Corporation（Oracle公司）

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to Table Representation Learning Workshop at ACL 2025

2505.13941 2025-05-21 cs.MA cs.AI cs.CL cs.LG 80%

MLZero: A Multi-Agent System for End-to-end Machine Learning Automation

Haoyang Fang, Boran Han, Nick Erickson, Xiyuan Zhang, Su Zhou, Anirudh Dagar, Jiani Zhang, Ali Caner Turkmen, Cuixiong Hu, Huzefa Rangwala, Ying Nian Wu, Bernie Wang, George Karypis

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2503.20589 2025-03-27 cs.SE 80%

What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond

Wenchao Gu, Juntao Chen, Yanlin Wang, Tianyue Jiang, Xingzhe Li, Mingwei Liu, Xilin Liu, Yuchi Ma, Zibin Zheng

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2502.13870 2025-02-20 cs.LG cs.AI cs.CL cs.IT math.IT 80%

SPEX: Scaling Feature Interaction Explanations for LLMs

Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2409.02976 2024-12-09 cs.LG cs.AI cs.CL 80%

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models

Gabriel Y. Arteaga, Thomas B. Schön, Nicolas Pielawski

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 6 pages, 3 figures

2410.19318 2024-10-28 cs.CL cs.AI cs.LG 80%

Two are better than one: Context window extension with multi-grained self-injection

Wei Han, Pan Zhou, Soujanya Poria, Shuicheng Yan

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The code is available at https://github.com/Clement25/SharedLLM

2410.02024 2024-10-24 cs.CE cs.AI cs.CL cs.LG 80%

FLAG: Financial Long Document Classification via AMR-based GNN

Bolun "Namir" Xia, Aparna Gupta, Mohammed J. Zaki

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 3 figures, to be published in CIFEr Conference 2024 as "Semantic Graph Learning for Trend Prediction from Long Financial Documents"

2404.07979 2024-10-18 cs.CL cs.AI cs.LG 80%

LLoCO: Learning Long Contexts Offline

Sijun Tan, Xiuyu Li, Shishir Patil, Ziyang Wu, Tianjun Zhang, Kurt Keutzer, Joseph E. Gonzalez, Raluca Ada Popa

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024. The first two authors contributed equally to this work

2410.05983 2024-10-10 cs.CL cs.AI cs.LG 80%

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

Bowen Jin, Jinsung Yoon, Jiawei Han, Sercan O. Arik

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 34 pages

2409.17422 2024-09-27 cs.CL cs.AI cs.LG 80%

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction

Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, Shafiq Joty

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.16008 2024-07-04 cs.CL cs.AI cs.LG 80%

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long T. Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas Pfister

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL Findings 2024

2406.05688 2024-06-11 cs.CL cs.AI cs.LG 80%

Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions

Cheng Tan, Dongxin Lyu, Siyuan Li, Zhangyang Gao, Jingxuan Wei, Siqi Ma, Zicheng Liu, Stan Z. Li

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Under review

2402.04617 2024-05-29 cs.CL cs.AI cs.LG 80%

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory

Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.05726 2024-04-25 cs.CV 80%

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, Ser-Nam Lim

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments Accepted at CVPR 2024. Project Page https://boheumd.github.io/MA-LMM/

2404.09339 2024-04-16 cs.CL cs.AI cs.LG 80%

Towards Practical Tool Usage for Continually Learning LLMs

Jerry Huang, Prasanna Parthasarathi, Mehdi Rezagholizadeh, Sarath Chandar

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages, 11 tables, 7 figures

2403.05004 2024-03-11 cs.CL cs.AI cs.IR cs.LG 80%

Can't Remember Details in Long Documents? You Need Some R&R

Devanshu Agrawal, Shang Gao, Martin Gajek

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 1 figure, 9 tables. For associated code repository see https://github.com/casetext/r-and-r

2307.12856 2024-02-27 cs.LG cs.AI cs.CL 80%

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ICLR 2024 (Oral)

2306.03901 2023-06-08 cs.AI cs.CL cs.DB cs.LG 80%

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, Hang Zhao

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2506.08184 2025-08-01 cs.CL cs.AI q-bio.NC 80%

Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length

Chupei Wang, Jiaqiu Vince Sun

专题命中长上下文与记忆：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ICML 2025 Workshop on Long Context Foundation Models (ICFM). Code: https://github.com/zhuangziGiantfish/Unable-to-Forget

2608.13921 2026-08-17 cs.AI 新提交 79%

When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict

当个人记忆没有唯一答案时：评估大语言模型智能体在不可约冲突下的表现

Lu Yang, Shusheng Xu, Zhuoran Li, Tongkai Yang, Longbo Huang

机构 * Ant Group（蚂蚁集团）

专题命中长上下文与记忆：LLM(title,abstract)；分类 cs.AI

AI总结该研究针对大语言模型智能体的记忆冲突问题，提出了基准TANGLE并评估其表现，发现现有方法存在缺陷，进而提出冲突感知行动策略CAAP。

2608.11248 2026-08-13 cs.AI cs.MA 新提交 79%

EvoGraph-Mem: Failure-Aware Editable Graph Memory for Long-Term Language Agents

EvoGraph-Mem：面向长期语言智能体的故障感知可编辑图记忆

Yuxi Qian, Yuxiang Ren

专题命中长上下文与记忆：language agent(title,abstract)；分类 cs.AI

AI总结本文针对长期语言智能体的记忆污染问题，提出故障感知可编辑图记忆框架，通过见解节点跟踪与图级编辑优化记忆，实验显示其性能优于现有记忆增强智能体基线。

Comments 10 pages, 3 figures

2509.18751 2026-08-13 cs.LG 版本更新 79%

Patch-based Memory Gate Model in Time Series Foundation Model

MOMEMTO：基于补丁的记忆门模型在时间序列基础模型中的应用

Samuel Yoon, Jongwon Kim, Juyoung Ha, Young Myoung Ko

机构 * Pohang University of Science and Technology（浦项科学技术大学）

专题命中长上下文与记忆：foundation model(title,abstract)；分类 cs.LG

AI总结 MOMEMTO通过基于补丁的记忆模块改进时间序列基础模型，有效缓解过度泛化问题，提升异常检测性能，尤其在少样本学习中表现突出。

Comments Published in Transactions on Machine Learning Research (TMLR), 2026

Journal ref Transactions on Machine Learning Research (TMLR), 2026

2608.07429 2026-08-11 cs.AI 版本更新 79%

TEPA: Revoking Stale Memories for Conflict-Robust Language Agents

TEPA：撤销过时记忆以构建冲突鲁棒的语言智能体

Yan Zhou, Yue Ouyang, Kaiyang Zheng, Suncheng Xiang

专题命中长上下文与记忆：language agent(title,abstract)；分类 cs.AI

AI总结该研究针对语言智能体的记忆污染问题，提出TEPA可撤销证据记忆机制，经多场景实验证实其能有效避免过时记忆干扰，性能优于仅追加、最后写入获胜等机制。

Comments 14 pages, 9 figures; includes supplementary material. Preprint

2608.00017 2026-08-04 cs.AI cs.CE 新提交 79%

Memory Reward Inflation in Self-Improving LLM Agents

自改进大语言模型智能体中的记忆奖励膨胀

Mohammad Asadolahi, Amir Amini, Samira Talebi, Amirfarhad Farhadi, Azadeh Zamanifar

机构 * University Of North Texas（北得克萨斯大学）； Edge Hill University（边山大学）； University of Cincinnati（辛辛那提大学）； Iran University of Science and Technology（伊朗科技大学）； Islamic Azad University（伊斯兰阿扎德大学）

专题命中长上下文与记忆：LLM(title,abstract)；分类 cs.AI

AI总结该研究发现自改进大语言模型智能体存在记忆奖励膨胀的“回声差距”问题，提出误差独立性假设（EIA），并通过LUCID算法在BIRD文本到SQL基准上提升了执行准确率。

2508.03668 2026-08-04 cs.CL 版本更新 79%

CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction

CTR-Sink：用于点击率预测的语言模型中的注意力汇聚点

Zixuan Li, Binzong Geng, Jing Xiong, Yong He, Yuxuan Hu, Jian Chen, Dingwei Chen, Xiyu Chang, Ngai Wong, Liang Zhang, Linjian Mo, Chengming Li, Chuan Yuan, Zhenan Sun

机构 * NLPR, Institute of Automation, Chinese Academy of Sciences（神经信息处理教育部重点实验室，自动化研究所，中国科学院）； Ant Group（蚂蚁集团）； The University of Hong Kong（香港大学）； City University of Hong Kong（香港城市大学）； Sun Yat-sen University（中山大学）； Shenzhen MSU-BIT University（深圳MSU-BIT大学）

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

AI总结针对用户行为序列与语言模型预训练文本之间的结构差异导致的语义碎片化问题，提出CTR-Sink框架，通过引入行为级注意力汇聚点并动态调节注意力聚合，提升点击率预测性能。

Comments Accepted by the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026). The source code of this paper has been made publicly available at https://github.com/UGUESS-lzx/CTR-SINK

2606.22932 2026-07-31 cs.LG 版本更新 79%

FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

FORGE: 融合寄存器梯度消除以实现内存高效的大语言模型训练

Dikshant Kukreja, Kritarth Prasad, Avinash Anand, Zhengkui Wang, Erik Cambria, Timothy Liu, Aik Beng Ng, Simon See, Bapi Chatterjee

机构 * Puch AI ； Singapore Institute of Technology（新加坡理工大学）； Nanyang Technological University（南洋理工大学）； NVIDIA（英伟达）； IIIT-Delhi（德里印度理工学院）

专题命中长上下文与记忆：LLM(title)；pretraining(abstract)；分类 cs.LG

AI总结提出FORGE方法，将优化器步骤融合到反向传播中，逐块在寄存器中应用，消除梯度张量，减少内存占用并加速训练，在微调和持续预训练中速度提升约1.5倍。

2607.24097 2026-07-28 cs.AI 新提交 79%

MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents

MemChain：为内存增强型大语言模型智能体学习可解释的内存痕迹

Yiwen Ma, Songjun Tu, Qichao Zhang, Dong Li, Linjing Li, Dongbin Zhao

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学交叉科学学院）； Memorax AI（Memorax人工智能公司）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）

专题命中长上下文与记忆：LLM(title,abstract)；分类 cs.AI

AI总结研究针对内存增强型大语言模型智能体检索内存回答查询存在的问题，提出MemChain可训练检索后内存策略，经两阶段学习框架训练，能生成有效证据上下文，实验证明其在多种模型上性能领先且减少内存上下文。

2607.22562 2026-07-28 cs.AI 新提交 79%

SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent

SF-AMS：大语言模型智能体中结构化记忆的策略性遗忘

Ning Yang, Siqi Li, Miaoxin Shen, Yuan Zhou, Meng Zhang, Tong Li, Haijun Zhang

专题命中长上下文与记忆：LLM(title,abstract)；分类 cs.AI

AI总结研究大语言模型智能体长上下文依赖瓶颈问题，提出SF-AMS框架，通过效用驱动机制和复合评分整合信号维护记忆，实验表明该方法在多跳推理等任务中相比基线有显著提升，证明建模记忆重要性对长上下文推理关键。

1. 长上下文与记忆 4822 篇

Human-inspired Episodic Memory for Infinite Context LLMs

Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs

SQLong: Enhanced NL2SQL for Longer Contexts with LLMs

MLZero: A Multi-Agent System for End-to-end Machine Learning Automation

What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond

SPEX: Scaling Feature Interaction Explanations for LLMs

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models

Two are better than one: Context window extension with multi-grained self-injection

FLAG: Financial Long Document Classification via AMR-based GNN

LLoCO: Learning Long Contexts Offline

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Towards Practical Tool Usage for Continually Learning LLMs

Can't Remember Details in Long Documents? You Need Some R&R

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length

When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict

EvoGraph-Mem: Failure-Aware Editable Graph Memory for Long-Term Language Agents

Patch-based Memory Gate Model in Time Series Foundation Model

TEPA: Revoking Stale Memories for Conflict-Robust Language Agents

Memory Reward Inflation in Self-Improving LLM Agents

CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction

FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents

SF-AMS: Strategic Forgetting for Structured Memory in LLM Agent