语言大模型 / LLM

One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space

Raghav Addanki, Chenyang Li, Zhao Song, Chiwun Yang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

LoMA: Lossless Compressed Memory Attention

Yumeng Wang, Zhenyang Xiao

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Working Memory Capacity of ChatGPT: An Empirical Study

Dongyu Gong, Xingchen Wan, Dingmin Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at the 38th AAAI Conference on Artificial Intelligence (AAAI-24)

In-Context Exemplars as Clues to Retrieving from Large Associative Memory

Jiachen Zhao

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Presented at Neural Conversational AI @ ICML 2023 and Associative Memory & Hopfield Networks @ NeurIPS 2023

LDM$^2$: A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement

Xingjin Wang, Linjing Li, Daniel Zeng

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Findings of the Association for Computational Linguistics: EMNLP 2023

SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion

Yuxiang Guo

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments work in progress

In-Context Learning for Text Classification with Many Labels

Aristides Milios, Siva Reddy, Dzmitry Bahdanau

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 4 figures

HyperAttention: Long-context Attention in Near-Linear Time

Insu Han, Rajesh Jayaram, Amin Karbasi, Vahab Mirrokni, David P. Woodruff, Amir Zandieh

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Drilling Down into the Discourse Structure with LLMs for Long Document Question Answering

Inderjeet Nair, Shwetha Somasundaram, Apoorv Saxena, Koustava Goswami

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the Findings of EMNLP 2023

Birth of a Transformer: A Memory Viewpoint

Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, Leon Bottou

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2023

Improving CLIP Training with Language Rewrites

Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, Yonglong Tian

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2023

Giraffe: Adventures in Expanding Context Lengths in LLMs

Arka Pal, Deep Karkhanis, Manley Roberts, Samuel Dooley, Arvind Sundararajan, Siddartha Naidu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Mass-Editing Memory in a Transformer

Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, David Bau

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 11 figures. Code and data at https://memit.baulab.info

Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section

Hongyi Zheng, Yixin Zhu, Lavender Yao Jiang, Kyunghyun Cho, Eric Karl Oermann

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Our code is publicly available on GitHub (https://github.com/nyuolab/EfficientTransformer)

Journal ref Association for Computational Linguistics - Student Research Workshop, 2023, pages 104-108

RepoFusion: Training Code Models to Understand Your Repository

Disha Shrivastava, Denis Kocetkov, Harm de Vries, Dzmitry Bahdanau, Torsten Scholak

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer

Yubin Kim, Dong Won Lee, Paul Pu Liang, Sharifa Algohwinem, Cynthia Breazeal, Hae Won Park

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Global memory transformer for processing long documents

Arij Al Adel

专题命中 长上下文与记忆 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 10 pages, 4 figures, 5 tables. Published in neuroinformatics 2022

Imagined versus Remembered Stories: Quantifying Differences in Narrative Flow

Maarten Sap, Anna Jafarpour, Yejin Choi, Noah A. Smith, James W. Pennebaker, Eric Horvitz

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Equal contribution from Sap and Jafarpour; in review; version 2

Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback

Niket Tandon, Aman Madaan, Peter Clark, Yiming Yang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2022 (Findings)

Luna: Linear Unified Nested Attention

Xuezhe Ma, Xiang Kong, Sinong Wang, Chunting Zhou, Jonathan May, Hao Ma, Luke Zettlemoyer

专题命中 长上下文与记忆 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Camera-Ready version in NeurIPS 2021. 2 figures, 9 tables

A review of on-device fully neural end-to-end automatic speech recognition algorithms

Chanwoo Kim, Dhananjaya Gowda, Dongsoo Lee, Jiyeon Kim, Ankur Kumar, Sungsoo Kim, Abhinav Garg, Changwoo Han

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted as an invited paper to the Asilomar Conference on Signals, Systems, and Computers 2020. Figures are slightly updated in Aug. 2021

Single-Pass Document Scanning for Question Answering

Weili Cao, Jianyou Wang, Youze Zheng, Longtian Bao, Qirui Zheng, Taylor Berg-Kirkpatrick, Ramamohan Paturi, Leon Bergen

机构 * Laboratory for Emerging Intelligence（新兴智能实验室） ； University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 长上下文与记忆 ：language model(abstract,comments)；large language model(abstract)；分类 cs.CL

Comments Published at Conference on Language Modeling (COLM), 2025

Single Conversation Methodology: A Human-Centered Protocol for AI-Assisted Software Development

Salvador D. Escobedo

机构 * Stefanini（斯蒂法尼） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI；LLM(comments)

Comments Style reviewed by a LLM for improving clarity and English syntax

Synergistic Modality-and-Slice Memory Framework for Cross-Modal 3D Brain Tumor Segmentation

用于跨模态3D脑肿瘤分割的协同模态与切片记忆框架

Yuxiang Luo, Qing Xu, Hai Huang, Yuqi Ouyang, Xiangjian He, Zhen Chen, Wenting Duan, Jiebo Luo

机构 * Graduate School of Information, Production and Systems, Waseda University, Japan（信息、生产与系统研究生院，早稻田大学，日本） ； School of Computer Science, University of Lincoln, UK（林肯大学计算机科学学院，英国） ； University of Nottingham, UK（诺丁汉大学，英国） ； University of Nottingham Ningbo China, China（宁波诺丁汉大学，中国） ； College of Electrical Engineering and Information, Northeast Agricultural University, Harbin, China（电气工程与信息学院，东北农业大学，中国） ； College of Computer Science, Sichuan University, Chengdu, China（计算机科学学院，四川大学，中国） ； Yale University, New Haven, CT 06510, USA（耶鲁大学，美国） ； School of Engineering and Physical Science, University of Lincoln, Lincoln LN6 7TS, UK（工程与物理科学学院，林肯大学，英国） 纠错

专题命中 长上下文与记忆 ：prompting(title)

AI总结 针对现有跨模态3D脑肿瘤分割方法忽略跨模态相关性、依赖特定类别提示的问题，提出MSM-Seg框架，通过MSMA、MCP-Encoder和MF-Decoder实现精准分割，性能优于现有方法。

Comments Accepted by TMM

RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

RoboStream: 用记忆融合时空推理提升视觉语言模型在机器人中的应用

Yuzhi Huang, Jie Wu, Weijue Bu, Ziyi Xiong, Gaoyang Jiang, Ye Li, Kangye Ji, Shuzhao Xie, Yue Huang, Chenglei Wu, Jingyan Jiang, Zhi Wang

机构 * Shenzhen International Graduate School, Tsinghua University（深圳国际研究生院，清华大学） ； YXGN Robotics（YXGN机器人） ； China University of Mining and Technology（中国矿业大学） ； Shenzhen Technology University（深圳技术大学） ； Huazhong University of Science and Technology（华中科技大学） ； Xiamen University（厦门大学） 纠错

专题命中 长上下文与记忆 ：language model(title)

AI总结 RoboStream通过时空融合令牌和因果时空图实现持久记忆，解决机器人长时间任务中的几何锚定和状态追踪问题，提升长期任务表现。

Comments Accepted by ECCV 2026

Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models

通过fMRI基础模型进行全脑分析解码视觉体验并映射语义

Yanchen Wang, Adam Turnbull, Tiange Xiang, Yunlong Xu, Sa Zhou, Adnan Masoud, Shekoofeh Azizi, Feng Vankee Lin, Ehsan Adeli

机构 * Department of Psychiatry and Behavioral Sciences, Stanford University（精神病学与行为科学系，斯坦福大学） ； Department of Computer Science, Stanford University（计算机科学系，斯坦福大学） ； Department of Neurobiology, University of Chicago（神经生物学系，芝加哥大学） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 长上下文与记忆 ：foundation model(title)

AI总结 本文提出基于fMRI基础模型的全脑分析方法，通过解码视觉体验提升对视觉处理的理解，实现超越传统视觉皮层的解码能力。

3D Scene Prompting for Scene-Consistent Camera-Controllable Video Generation

用于场景一致的摄像机可控视频生成的3D场景提示

JoungBin Lee, Jaewoo Jung, Jisang Han, Takuya Narihira, Kazumi Fukuda, Junyoung Seo, Sunghwan Hong, Yuki Mitsufuji, Seungryong Kim

机构 * KAIST AI（韩国科学技术院人工智能研究所） ； Sony AI（索尼人工智能研究所） ； ETH Zürich（苏黎世联邦理工学院） ； Sony Group Corporation（索尼集团） 纠错

专题命中 长上下文与记忆 ：prompting(title)

AI总结 3DScenePrompt通过双时空条件化和3D场景记忆实现场景一致且可控的视频生成，提升生成质量与摄像机控制精度。

Comments Project page : https://cvlab-kaist.github.io/3DScenePrompt/

MLC-Agent: Cognitive Model based on Memory-Learning Collaboration in LLM Empowered Agent Simulation Environment

Ming Zhang, Yiling Xuan, Qun Ma, Yuwei Guo

专题命中 长上下文与记忆 ：LLM(title)

Language model integration based on memory control for sequence to sequence speech recognition

Jaejin Cho, Shinji Watanabe, Takaaki Hori, Murali Karthick Baskar, Hirofumi Inaguma, Jesus Villalba, Najim Dehak

专题命中 长上下文与记忆 ：language model(title)

Comments 4 pages, 1 figure, 5 tables, ICASSP 2019, A notice added to the previous version

Whose fault is it anyway? SILC: Safe Integration of LLM-Generated Code

Peisen Lin, Yuntong Zhang, Andreea Costea, Abhik Roychoudhury

专题命中 长上下文与记忆 ：LLM(title)