语言大模型 / LLM - arXivDaily 专题

2311.15879 2024-04-09 cs.CV 67%

EVCap: Retrieval-Augmented Image Captioning with External Visual-Name Memory for Open-World Comprehension

Jiaxuan Li, Duc Minh Vo, Akihiro Sugimoto, Hideki Nakayama

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)

Comments CVPR 2024

2402.01714 2024-02-07 cs.CL cs.AI cs.LG 67%

TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copy

Vibhav Agarwal, Sourav Ghosh, Harichandana BSS, Himanshu Arora, Barath Raj Kandur Raja

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in the IEEE/ACM Transactions on Audio, Speech, and Language Processing. (Sourav Ghosh and Vibhav Agarwal contributed equally to this work.)

Journal ref IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 1173-1184, 2024

2304.11062 2024-02-07 cs.CL cs.AI cs.LG 67%

Scaling Transformer to 1M tokens and beyond with RMT

Aydar Bulatov, Yuri Kuratov, Yermek Kapushev, Mikhail S. Burtsev

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2401.12412 2024-01-24 cs.SE 67%

Program Decomposition and Translation with Static Analysis

Ali Reza Ibrahimzada

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)

Comments Published in ICSE SRC 2024

2401.00849 2024-01-02 cs.CV 67%

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

Alex Jinpeng Wang, Linjie Li, Kevin Qinghong Lin, Jianfeng Wang, Kevin Lin, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)

Comments 16 pages; Website: http://fingerrec.github.io/cosmo

2305.04782 2023-12-05 cs.CL cs.AI cs.LG 67%

HistAlign: Improving Context Dependency in Language Generation by Aligning with History

David Wan, Shiyue Zhang, Mohit Bansal

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2023 (20 pages)

2306.09312 2023-10-30 cs.LG cs.AI cs.CL stat.ML 67%

Semantic HELM: A Human-Readable Memory for Reinforcement Learning

Fabian Paischer, Thomas Adler, Markus Hofmarcher, Sepp Hochreiter

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To appear at NeurIPS 2023, 10 pages (+ references and appendix), Code: https://github.com/ml-jku/helm

2309.11568 2023-09-22 cs.AI cs.CL cs.LG 67%

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model

Nolan Dey, Daria Soboleva, Faisal Al-Khateeb, Bowen Yang, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming, Chen, Robert Myers, Jacob Robert Steeves, Natalia Vassilieva, Marvin Tom, Joel Hestness

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2309.08715 2023-09-19 cs.FL 67%

Formalizing BPE Tokenization

Martin Berglund, Brink van der Merwe

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)

Comments In Proceedings NCMA 2023, arXiv:2309.07333

Journal ref EPTCS 388, 2023, pp. 16-27

2306.10231 2023-06-21 cs.CL cs.AI cs.LG 67%

GLIMMER: generalized late-interaction memory reranker

Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Sumit Sanghai, William W. Cohen, Joshua Ainslie

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2301.10448 2023-06-06 cs.CL cs.AI cs.LG 67%

Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute

Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William Cohen

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2023

2210.11807 2022-10-24 cs.CL cs.AI cs.LG 67%

Is Encoder-Decoder Redundant for Neural Machine Translation?

Yingbo Gao, Christian Herold, Zijian Yang, Hermann Ney

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted at AACL2022

2206.14969 2022-07-01 cs.CL cs.AI cs.LG 67%

Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?

Xiang Zhou, Shiyue Zhang, Mohit Bansal

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2022 (16 pages)

2202.10447 2022-06-28 cs.LG cs.AI cs.CL cs.NE 67%

Transformer Quality in Linear Time

Weizhe Hua, Zihang Dai, Hanxiao Liu, Quoc V. Le

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the 39th International Conference on Machine Learning (ICML'22)

2205.08343 2022-06-24 cs.IR 67%

Moving Stuff Around: A study on efficiency of moving documents into memory for Neural IR models

Arthur Câmara, Claudia Hauff

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)

Comments 7 pages, 2 figures. Accepted to the ReNeuIR workshop at SIGIR 2022

1808.08720 2022-03-30 cs.LG cs.AI cs.CL stat.ML 67%

Predefined Sparseness in Recurrent Sequence Models

Thomas Demeester, Johannes Deleu, Fréderic Godin, Chris Develder

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments the SIGNLL Conference on Computational Natural Language Learning (CoNLL, 2018)

2203.08913 2022-03-18 cs.LG cs.AI cs.CL 67%

Memorizing Transformers

Yuhuai Wu, Markus N. Rabe, DeLesley Hutchins, Christian Szegedy

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2022 (spotlight)

2105.07510 2021-10-12 cs.CL cs.AI cs.LG 67%

Doc2Dict: Information Extraction as Text Generation

Benjamin Townsend, Eamon Ito-Fisher, Lily Zhang, Madison May

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2110.02220 2021-10-08 eess.AS cs.AI cs.CL cs.LG cs.NE 67%

Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition

Tsendsuren Munkhdalai, Khe Chai Sim, Angad Chandorkar, Fan Gao, Mason Chua, Trevor Strohman, Françoise Beaufays

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 5 pages, 3 figures, 3 tables

2012.08695 2020-12-17 cs.CL cs.AI cs.IR cs.LG 67%

DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition

Weizhou Shen, Junqing Chen, Xiaojun Quan, Zhixian Xie

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by AAAI 2021 main conference

2007.00849 2020-07-03 cs.CL cs.AI cs.LG 67%

Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Pat Verga, Haitian Sun, Livio Baldini Soares, William W. Cohen

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

1908.02367 2019-08-08 cs.CL cs.AI cs.LG 67%

Semantic Role Labeling with Associated Memory Network

Chaoyu Guan, Yuhao Cheng, Hai Zhao

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at NAACL 2019; This is camera Ready version; Code is available at https://github.com/Frozenmad/AMN_SRL

1810.06682 2019-03-13 cs.LG cs.AI cs.CL stat.ML 67%

Trellis Networks for Sequence Modeling

Shaojie Bai, J. Zico Kolter, Vladlen Koltun

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICLR 2019

1709.05027 2018-02-13 cs.LG cs.AI cs.CL cs.NE 67%

Learning Intrinsic Sparse Structures within Long Short-Term Memory

Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in ICLR 2018 ( the Sixth International Conference on Learning Representations)

2512.23852 2026-01-01 cs.LG cs.CL 66%

Trellis: Learning to Compress Key-Value Memory in Attention Models

Trellis: 在注意力模型中学习压缩键值记忆

Mahdi Karami, Ali Behrouz, Praneeth Kacham, Vahab Mirrokni

机构 * Google Research（谷歌研究）

专题命中长上下文与记忆：language model(abstract,comments)；分类 cs.CL、cs.LG

AI总结 Trellis通过动态压缩键值内存提升注意力模型的效率与长上下文处理能力

Comments In Second Conference on Language Modeling (COLM) (2025)

2104.08401 2021-10-08 cs.CL cs.AI 66%

Enriching a Model's Notion of Belief using a Persistent Memory

Nora Kassner, Oyvind Tafjord, Hinrich Schutze, Peter Clark

专题命中长上下文与记忆：language model(abstract,comments)；分类 cs.CL、cs.AI

Comments This is an old and now obsolete draft. See arXiv:2109.14723 ("BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief") for the final paper

2608.20427 2026-08-24 cs.LG cs.AI 新提交 62%

BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers

BF1：一种用于高效长上下文Transformer的因果二元稀疏注意力改造方案

Hina Dixit

专题命中长上下文与记忆：language model(abstract)；分类 cs.AI、cs.LG

AI总结本文提出BF1，一种结合局部邻域、全局首块与对数间隔历史块的稀疏注意力方案，改造Qwen3-0.6B部分注意力层后，在长上下文场景下显著降低模型延迟，且语言建模性能优于基线方法。

2510.09930 2026-08-24 cs.LG cs.AI 版本更新 62%

Perseus: Interactive Time Series Segmentation with Sparse Supervision via Stateful Memory

Perseus：基于状态记忆的稀疏监督交互式时间序列分割

Ching Chang, Ming-Chih Lo, Chiao-Tung Chan, Wen-Chih Peng, Tien-Fu Chen

机构 * Computer Science National Yang Ming Chiao Tung University（计算机科学国立阳明交通大学）； Control Engineering National Yang Ming Chiao Tung University（控制工程国立阳明交通大学）； National Yang Ming Chiao Tung University（国立阳明交通大学）

专题命中长上下文与记忆：prompting(abstract)；分类 cs.AI、cs.LG

AI总结针对现有交互式时间序列分割方法无法保留用户交互历史的问题，提出 Perseus 框架，通过写-读架构实现异步状态管理，在多粒度设置中准确率提升最高达 85%。

Comments Accepted at IEEE ICDM 2026

2608.19013 2026-08-20 cs.LG cs.AI 新提交 62%

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

利用工具链持续学习：超越模型参数的持续适应

Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao

机构 * University of Wollongong（卧龙岗大学）； Nanjing University（南京大学）

专题命中长上下文与记忆：foundation model(abstract)；分类 cs.AI、cs.LG

AI总结该研究提出工具链持续学习（HCL）范式，围绕冻结基础模型演化工具链，通过受保护演化机制缓解工具链级遗忘，在多任务实验中相对基准增益超10%，可调整稳定性-可塑性权衡。

2608.12597 2026-08-14 cs.LG cs.AI 新提交 62%

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks

预测随机低维重参数化何时能训练神经网络

Andrew Cheng, Ali Eslamian, Jie Cheng, Mehdi Zargham, Qiang Cheng

机构 * Tsinghua University（清华大学）； The University of Manchester（曼彻斯特大学）； University of Kentucky（肯塔基大学）； Miami University（迈阿密大学）； University of Dayton（代顿大学）； Institute for Biomedical Informatics, University of Kentucky（肯塔基大学生物医学信息学研究所）

专题命中长上下文与记忆：language model(abstract)；分类 cs.AI、cs.LG

AI总结该研究针对随机低维重参数化训练神经网络的潜在空间规模问题，提出取向分辨二次主公式，引入RaMaN模型，实现内存优化，实验验证其预测与转换跟踪性能优于取向无关近似。

1. 长上下文与记忆 4797 篇

EVCap: Retrieval-Augmented Image Captioning with External Visual-Name Memory for Open-World Comprehension

TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copy

Scaling Transformer to 1M tokens and beyond with RMT

Program Decomposition and Translation with Static Analysis

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

HistAlign: Improving Context Dependency in Language Generation by Aligning with History

Semantic HELM: A Human-Readable Memory for Reinforcement Learning

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model

Formalizing BPE Tokenization

GLIMMER: generalized late-interaction memory reranker

Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute

Is Encoder-Decoder Redundant for Neural Machine Translation?

Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?

Transformer Quality in Linear Time

Moving Stuff Around: A study on efficiency of moving documents into memory for Neural IR models

Predefined Sparseness in Recurrent Sequence Models

Memorizing Transformers

Doc2Dict: Information Extraction as Text Generation

Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition

DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition

Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Semantic Role Labeling with Associated Memory Network

Trellis Networks for Sequence Modeling

Learning Intrinsic Sparse Structures within Long Short-Term Memory

Trellis: Learning to Compress Key-Value Memory in Attention Models

Enriching a Model's Notion of Belief using a Persistent Memory

BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers

Perseus: Interactive Time Series Segmentation with Sparse Supervision via Stateful Memory

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks