LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction
Georgia Institute of Technology(佐治亚理工学院);Peking University(北京大学);Shandong University(山东大学);Huazhong University of Science and Technology(华中科技大学);UT Southwestern Medical Center(西南医学中心)
Yuxing Lu, J. Ben Tamo, Weichen Zhao, Nan Sun, Yishan Zhong, Wenqi Shi, Jinzhuo Wang, May D. Wang
Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation.
CNRS(法国国家科学研究中心);ENS Paris(巴黎高等师范学校);Université Sorbonne Nouvelle - Paris 3(新索邦大学(巴黎第三大学));PSL Research University(巴黎文理研究大学);USPC (Université Sorbonne Paris Cité)(索邦巴黎西岱大学)
RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Key Laboratory for Information System Security, Ministry of Education(教育部信息系统安全重点实验室);Huawei Technologies Co., Ltd.(华为技术有限公司)
Recurrent Neural Networks as Weighted Language Recognizers
Dartmouth College(达特茅斯学院);ILCC(爱丁堡大学语言、认知与计算研究所);University of Edinburgh(爱丁堡大学);Institute of Computer Science(计算机科学研究所);Universität Leipzig(莱比锡大学);Information Sciences Institute(信息科学研究所);University of Southern California(南加利福尼亚大学)
Improving Recurrent Neural Networks For Sequence Labelling
LaTTiCe (UMR 8094)(LaTTiCe实验室(UMR 8094));CNRS(法国国家科学研究中心);ENS Paris(巴黎高等师范学院);Université Sorbonne Nouvelle - Paris 3(索邦新巴黎第三大学);PSL Research University(PSL研究大学);USPC (Université Sorbonne Paris Cité)(索邦巴黎西岱大学)
Going Wider: Recurrent Neural Network With Parallel Cells
Department of Computer Science, Nanjing University(南京大学计算机科学与技术系)
循环神经网络(Recurrent Neural Network,RNN)已广泛应用于序列建模。在RNN中,当前步骤中的隐藏状态与前一步骤中的隐藏状态完全连接,因此前一步骤中相关性较低的特征的影响可能会潜在地降低模型的学习能力。我们提出了一种简单的技术,称为并行细胞(PC),以提高循环神经网络(RNN)的学习能力。在每一层中,我们运行多个小的RNN单元,而不是一个大的单元。在本文中,我们评估PC上的2个任务。在PTB(Penn Tree Bank)的语言建模任务中,我们的模型通过将困惑度从78.6降低到75.3而优于最先进的模型。在汉英翻译任务中,我们的模型比基线模型提高了BLEU分数0.39分。