语言大模型 / LLM

Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?

Jonathan Roberts, Kai Han, Samuel Albanie

机构 * University of Cambridge（剑桥大学） ； The University of Hong Kong（香港大学） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at ICLR 2025

Self-evolving Agents with reflective and memory-augmented abilities

Xuechen Liang, Yangfan He, Yinghui Xia, Xinyuan Song, Jianhui Wang, Meiling Tao, Li Sun, Xinhang Yuan, Jiayi Su, Keqin Li, Jiaqi Chen, Jinsong Yang, Siyuan Chen, Tianyu Shi

机构 * East China Jiao Tong University（东中国交通大学） ； University of Minnesota - Twin Cities（明尼苏达大学-双城分校） ； Emory University（埃默里大学） ； University of Electronic Science and Technology of China（电子科技大学） ； Guangdong University of Technology（广东工业大学） ； Amazon（亚马逊） ； Washington University（华盛顿大学） ； Xiamen University（厦门大学） ； University of Bristol（布里斯托尔大学） ； University of Toronto（多伦多大学） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

MARS: Memory-Enhanced Agents with Reflective Self-improvement

Xuechen Liang, Meiling Tao, Yinghui Xia, Jianhui Wang, Kun Li, Yijin Wang, Jingsong Yang, Tianyu Shi, Yuantao Wang, Miao Zhang, Xueqian Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments We are withdrawing this version because it duplicates our previous submission (arXiv:2409.00872)

GazeLLM: Multimodal LLMs incorporating Human Visual Attention

Jun Rekimoto

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Journal ref Augmented Humans 2025

VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning

Yongshuo Zong, Ondrej Bohdal, Timothy Hospedales

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ICLR 2025

FReM: A Flexible Reasoning Mechanism for Balancing Quick and Slow Thinking in Long-Context Question Answering

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Bin Liang, Binyang Li, Kam-Fai Wong

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

DEMENTIA-PLAN: An Agent-Based Framework for Multi-Knowledge Graph Retrieval-Augmented Generation in Dementia Care

Yutong Song, Chenhan Lyu, Pengfei Zhang, Sabine Brunswicker, Nikil Dutt, Amir Rahmani

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted by AAAI 2025 Workshop on Knowledge Graphs for Personalized Public Health

Bootstrap Your Own Context Length

Liang Wang, Nan Yang, Xingxing Zhang, Xiaolong Huang, Furu Wei

专题命中 长上下文与记忆 ：language model(abstract)；instruction tuning(abstract)；分类 cs.CL

Comments 19 pages

DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels

Zhe Xu, Jiasheng Ye, Xiaoran Liu, Xiangyang Liu, Tianxiang Sun, Zhigeng Liu, Qipeng Guo, Linlin Li, Qun Liu, Xuanjing Huang, Xipeng Qiu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Encrypted Vector Similarity Computations Using Partially Homomorphic Encryption: Applications and Performance Analysis

Sefik Serengil, Alper Ozpinar

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.LG

NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains

Wonje Choi, Jinwoo Park, Sanghyun Ahn, Daehee Lee, Honguk Woo

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted at ICLR 2025. Project site with code: https://pjw971022.github.io/nesyc/

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling

Zhenghua Wang, Yiran Ding, Changze Lv, Zhibo Xu, Tianlong Li, Tianyuan Shi, Xiaoqing Zheng, Xuanjing Huang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graph

Teng Lin, Yizhang Zhu, Yuyu Luo, Nan Tang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

OkraLong: A Flexible Retrieval-Augmented Framework for Long-Text Query Processing

Yulong Hui, Yihao Liu, Yao Lu, Huanchen Zhang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Nguyen Xuan Thanh, Anh Duc Le, Quyen Tran, Thanh-Thien Le, Linh Ngo Van, Thien Huu Nguyen

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to AAAI 2025

LongAttn: Selecting Long-context Training Data via Token-level Attention

Longyun Wu, Dawei Zhu, Guangxiang Zhao, Zhuocheng Yu, Junfeng Ran, Xiangyu Wong, Lin Sun, Sujian Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 17 pages, 5 figures

EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance

Yingxin Li, Ye Li, Yuan Meng, Xinzhu Ma, Zihan Geng, Shutao Xia, Zhi Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Logic Haystacks: Probing LLMs Long-Context Logical Reasoning (Without Easily Identifiable Unrelated Padding)

Damien Sileo

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression

Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 13pages,9 figures

MuDAF: Long-Context Multi-Document Attention Focusing through Contrastive Learning on Attention Heads

Weihao Liu, Ning Wu, Shiping Yang, Wenbiao Ding, Shining Liang, Ming Gong, Dongmei Zhang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 18 pages

Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptor

Barys Liskavets, Shuvendu Roy, Maxim Ushakov, Mark Klibanov, Ali Etemad, Shane Luke

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMs

Joon Park, Kyohei Atarashi, Koh Takeuchi, Hisashi Kashima

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 11 pages, 2 figures

Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs

Ryan Synk, Monte Hoover, John Kirchenbauer, Neel Jain, Alex Stein, Manli Shu, Josue Melendez Sanchez, Ramani Duraiswami, Tom Goldstein

专题命中 长上下文与记忆 ：LLM(abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages, 9 figures, 2 tables in main body

Improving Adaptive Moment Optimization via Preconditioner Diagonalization

Son Nguyen, Bo Liu, Lizhang Chen, Qiang Liu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 19 pages, 13 figures

VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos

Xubin Ren, Lingrui Xu, Long Xia, Shuaiqiang Wang, Dawei Yin, Chao Huang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Cogito, ergo sum: A Neurobiologically-Inspired Cognition-Memory-Growth System for Code Generation

Yanlong Li, Jindong Li, Qi Wang, Menglin Yang, He Kong, Shengsheng Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads

Xingyang He, Jie Liu, Shaowei Chen

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Information Entropy Invariance: Enhancing Length Extrapolation in Attention Mechanisms

Kewei Li, Yanwen Kong, Yiping Xu, Jianlin Su, Lan Huang, Ruochi Zhang, Fengfeng Zhou

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted to ICLR 2025. Code is available at https://github.com/hmxiong/StreamChat

LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs

Yuhao Wu, Ming Shan Hee, Zhiqing Hu, Roy Ka-Wei Lee

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ICLR 2025; Github: https://github.com/mozhu621/LongGenBench/