The Transformer architecture has led to significant gains in machine translation. However, most studies focus on only sentence-level translation without considering the context dependency within documents, leading to the inadequacy of document-level coherence. Some recent research tried to mitigate this issue by introducing an additional context encoder or translating with multiple sentences or even the entire document. Such methods may lose the information on the target side or have an increasing computational complexity as documents get longer. To address such problems, we introduce a recurrent memory unit to the vanilla Transformer, which supports the information exchange between the sentence and previous context. The memory unit is recurrently updated by acquiring information from sentences, and passing the aggregated knowledge back to subsequent sentence states. We follow a two-stage training strategy, in which the model is first trained at the sentence level and then finetuned for document-level translation. We conduct experiments on three popular datasets for document-level machine translation and our model has an average improvement of 0.91 s-BLEU over the sentence-level baseline. We also achieve state-of-the-art results on TED and News, outperforming the previous work by 0.36 s-BLEU and 1.49 d-BLEU on average.
ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation
University of Edinburgh（爱丁堡大学）；Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院直觉机器人实验室）；NVIDIA（英伟达）；Robotics Institute Germany (RIG)（德国机器人研究所）
Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improve
Technical University of Munich（慕尼黑技术大学）；TUM University Hospital（TUM大学医院）；LMU Munich（慕尼黑大学）；National University of Singapore（新加坡国立大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Imperial College London（伦敦帝国理工学院）
LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replay
A neural network that maintains and retrieves memories based on context
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；City University of Hong Kong（香港城市大学）；University of Chicago（芝加哥大学）
