语言大模型 / LLM

Ascend-CC: Confidential Computing on Heterogeneous NPU for Emerging Generative AI Workloads

Aritra Dhar, Clément Thorens, Lara Magdalena Lazier, Lukas Cavigelli

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments ECCV-24; Project page: videoagent.github.io; First two authors contributed equally

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs

Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, Yu-Gang Jiang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project Page: https://deepstack-vl.github.io/

LOLAMEME: Logic, Language, Memory, Mechanistic Framework

Jay Desai, Xiaobo Guo, Srinivasan H. Sengamedu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://openreview.net/pdf?id=73dhbcXxtV

MileBench: Benchmarking MLLMs in Long Context

Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, Benyou Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages, 13 figures, 14 tables; We add results of GPT-4o in this version

TransformerFAM: Feedback attention is working memory

Dongseong Hwang, Weiran Wang, Zhuoyuan Huo, Khe Chai Sim, Pedro Moreno Mengibar

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages, 12 figures, 14 tables

Traffic Performance GPT (TP-GPT): Real-Time Data Informed Intelligent ChatBot for Transportation Surveillance and Management

Bingzhang Wang, Zhiyu Cai, Muhammad Monjurul Karim, Chenxi Liu, Yinhai Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 8 pages, 5 figures, submitted to 27th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2024)

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, Gaoang Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Unlocking Memories with AI: Exploring the Role of AI-Generated Cues in Personal Reminiscing

Jun Li Jeung, Janet Yi-Ching Huang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Rooms

Diandian Guo, Manxi Lin, Jialun Pei, He Tang, Yueming Jin, Pheng-Ann Heng

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 4 figures, 3 tables

A RAG Method for Source Code Inquiry Tailored to Long-Context LLMs

Toshihiro Kamiya

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 6 pages, 2 columns, English translation of the manuscript originally presented in Japanese at a domestic workshop

Enabling Memory Safety of C Programs using LLMs

Nausheen Mohammed, Akash Lal, Aseem Rastogi, Subhajit Roy, Rahul Sharma

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, Gaoang Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments CVPR 2024. First three authors contribute equally to this work. Project Website https://rese1f.github.io/MovieChat/

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending

Shiyi Zhu, Jing Ye, Wei Jiang, Siqiao Xue, Qi Zhang, Yifan Wu, Jianguo Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 7 figures

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE

Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei, Zhangyue Yin, Yunhua Zhou, Linyang Li, Tianxiang Sun, Hang Yan, Dahua Lin, Xipeng Qiu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents

James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan O Arik, Yan Liu, Tomas Pfister

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Focused Transformer: Contrastive Training for Context Scaling

Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek, Yuhuai Wu, Henryk Michalewski, Piotr Miłoś

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at 37th Conference on Neural Information Processing Systems (NeurIPS 2023). 28 pages, 10 figures, 11 tables

TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance

Yang Li, Yangyang Yu, Haohang Li, Zhi Chen, Khaldoun Khashanah

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agents

Ziheng Huang, Sebastian Gutierrez, Hemanth Kamana, Stephen MacNeil

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Segment Everything Everywhere All at Once

Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, Yong Jae Lee

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；prompting(abstract)

KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

KVEraser: 学习操控KV缓存以实现高效的局部上下文擦除

Mufei Li, Shikun Liu, Dongqi Fu, Haoyu Wang, Yinglong Xia, Hong Li, Hong Yan, Pan Li

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Meta 纠错

专题命中 长上下文与记忆 ：LLM(abstract,abstract_cn)；分类 cs.CL、cs.LG；foundation model(comments)

AI总结 提出KVEraser方法，通过学习操控KV缓存实现局部上下文擦除，避免全局重计算，在长上下文任务中接近全重算性能且延迟仅增加24%。

Comments Oral at the ICML 2026 Workshop on the Impact of Memorization on Trustworthy Foundation Models; Code available at https://github.com/Graph-COM/KVEraser

Position: Modular Memory is the Key to Continual Learning Agents

Position: 模块化记忆是持续学习智能体的关键

Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov, Lucas Caccia, Antonio Carta, Laurent Charlin, Barbara Hammer, Tyler L. Hayes, Timm Hess, Christopher Kanan, Dhireesha Kudithipudi, Xialei Liu, Vincenzo Lomonaco, Jorge Mendez-Mendez, Darshan Patil, Ameya Prabhu, Elisa Ricci, Tinne Tuytelaars, Gido M. van de Ven, Liyuan Wang, Joost van de Weijer, Jonghyun Choi, Martin Mundt, Rahaf Aljundi

机构 * University of Bremen（不莱梅大学） ； Seoul National University（首尔国立大学） ； Computer Vision Center Barcelona（巴塞罗那计算机视觉中心） ； University of Florence（佛罗伦萨大学） ； Microsoft Research（微软研究院） ； HEC Montreal, Mila--Quebec AI Institute, Canada CIFAR AI Chair（蒙特利尔HEC学院、魁北克人工智能研究所、加拿大CIFAR人工智能 chair） ； Bielefeld University（比勒海姆大学） ； Georgia Institute of Technology（佐治亚理工学院） ； University of Rochester（罗切斯特大学） ； University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校） ； Nankai University（南开大学） ； LUISS University（卢西亚诺大学） ； Stony Brook University（石溪大学） ； University of Tübingen（图宾根大学） ； University of Trento, FBK（特伦托大学，FBK） ； Tsinghua University（清华大学） ； University of Groningen（Groningen大学） 纠错

专题命中 长上下文与记忆 ：foundation model(abstract,comments)；pretraining(abstract)；分类 cs.AI、cs.LG

AI总结 本文提出通过模块化记忆结合权重内学习与上下文学习，解决持续学习中的灾难性遗忘问题，实现大规模持续适应。

Comments ICML 2026 Position Track Spotlight. This work stems from discussions held at the Dagstuhl seminar on Continual Learning in the Era of Foundation Models (October 2025)

Pay Attention to What You Need

Yifei Gao, Shaohong Chen, Lei Wang, Ruiting Dai, Ziyun Zhang, Kerui Ren, Jiaji Wu, Jun Cheng

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI；LLM(comments)

Comments LLM for long context comprehension, attention mechanism

Inadvertent Context Leakage in Language Models

语言模型中的无意上下文泄漏

Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg, Saeed Mahloujifar

机构 * FAIR, Meta Superintelligence Labs（Meta超级智能实验室FAIR） ； University of California, Berkeley（加利福尼亚大学伯克利分校） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 长上下文与记忆 ：language model(title)；分类 cs.LG

AI总结 该研究发现语言模型的上下文窗口会无意泄漏敏感用户信息，提出自适应攻击方法，在8个专有模型上验证了2、4位数秘密的重建准确率，还展示了两种实际攻击方式，指出泄漏是模型能力的副产品。

Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference

分形KV缓存存档：用于长上下文语言模型推理的带原地检索的无损符号存储

Vladimir Gusev

机构 * Independent Researcher（独立研究者） 纠错

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.LG

AI总结 研究长上下文语言模型推理中KV缓存的存储问题，提出用分形KV缓存存档方法，该方法无损、线性时间运行且支持随机访问与追加，经实验验证能大幅减少存档缓存，还兼具搜索索引功能，发布代码可重现结果。

Comments 10 pages, 3 figures. Code: https://github.com/eighteight/fractal-kv

Hierarchical Graph Memory for LLM Agents with Path-level Localization and Rewrite

面向具备路径级定位与重写能力的大语言模型智能体的分层图记忆框架HiGram

Xiawei Yue, Boran Wang, Xiaoqing Zhang, Shuxin Zheng, Ziwei Zhang

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.AI

AI总结 本文提出具备路径级定位与重写能力的分层图记忆框架HiGram，解决扁平图记忆的无关信息多、重写效率低问题，在相关基准上提升了答案质量、令牌效率及冲突场景下的表现。

Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

基于循环语言模型的线性时间与常数内存文本嵌入

Tobias Grantner, Emanuel Sallinger, Martin Flechl

机构 * Dynatrace Research（DynaTrace研究） 纠错

专题命中 长上下文与记忆 ：language model(title)；分类 cs.CL

AI总结 本文提出基于循环架构的高效文本嵌入方法，通过垂直分块推理策略实现线性时间复杂度和常数内存使用，验证了Mamba2等模型在文本嵌入任务中的有效性。

Journal ref Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026, pages 41459-41481

Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

大语言模型家族中的跨模型KV缓存迁移：用于预填充复用的闭式线性映射

Taekyung Heo, Rasoul Shafipour, Ritchie Zhao, Maximilian Golub, Mohammad Mahdi Kamani, Ritika Borkar, Makesh Tarun Chandran, Pantea Zardoshti, Bita Darvish Rouhani

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.LG

AI总结 针对大语言模型家族切换时需重新预填充的问题，提出跨模型KV缓存迁移方法，通过闭式线性映射器复用源模型KV缓存，可提升预填充速度，保留大部分准确率，具备实用性。

From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps

从认知架构到语言智能体：谱系、融合与迁移差距的机制层面综述

Haodi Fan, Zucong Lan

专题命中 长上下文与记忆 ：language agent(title)；分类 cs.AI

AI总结 该综述连接多个认知架构与语言智能体系统，重构机制并编码相关关系，指出现代智能体部分功能已实现，最强机会在机制耦合，存在五个剩余组合，贡献了机制目录、证据深度框架及可证伪议程。

Comments 28 pages, 9 figures, 14 tables. Review article

Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

时间上下文恢复驱动长上下文语言模型中的情景式顺序记忆

Mathis Pink, Vy Ai Vo, Qinyuan Wu, Jianing Mu, Javier Turek, Uri Hasson, Kenneth A. Norman, Sebastian Michelmann, Alexander Huth, Mariya Toneva

专题命中 长上下文与记忆 ：language model(title)；分类 cs.AI

AI总结 研究长上下文语言模型中情景式顺序记忆的机制，通过时间顺序记忆任务及新数据集，发现模型呈现与人类相同的距离效应，且性能依赖一维时间编码，由单个注意力头在检索时恢复，支持时间上下文恢复是重要机制。