Sparse expert execution does not remove the stored states needed to adapt a mixture-of-experts language model. We present DuplexTune, a framework for replaying coupled hidden states through pretrained attention and expert modules. Trainable width bridges connect two residual streams to the backbone. Bridge warm-up precedes backbone adaptation, with router weights kept frozen. We analyze the implicit state-recovery equation and identify the convergence and replay conditions needed for reconstruction. On Qwen1.5-MoE-A2.7B, the single-H800 evaluation reports 38.8 GB peak memory, versus 64.8 GB for supervised fine-tuning with checkpointing, and throughput of 25.1 versus 19.9 samples/s. DuplexTune achieves 67.2% MMLU, 75.6% GSM8K, and 7.69 MT-Bench, with a weaker multilingual score than the checkpointed baseline. The results characterize a configuration-dependent tradeoff among state storage, throughput, and adaptation quality.
Replay on Demand: An Emergent Curriculum for Balancing Adaptation and Forgetting in Continued Pretraining
University of Tübingen(图宾根大学);Tübingen AI Center(图宾根人工智能中心);Helmholtz Munich(亥姆霍兹慕尼黑中心);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Thomson Reuters Foundational Research(汤森路透基础研究);Imperial College London(伦敦帝国理工学院);Technical University of Munich(慕尼黑工业大学)
Replay Buffer with Local Forgetting for Adapting to Local Environment Changes in Deep Model-Based Reinforcement Learning
Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Microsoft Research(微软研究院);École Polytechnique de Montréal(蒙特利尔高等理工学院);Canada CIFAR AI Chair(加拿大CIFAR人工智能讲席)
Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
Imagia;Montreal Institute for Learning Algorithms (MILA), University of Montreal(蒙特利尔学习算法研究所(MILA),蒙特利尔大学);Dalhousie University(达尔豪西大学);Canadian Institute for Advanced Research (CIFAR)(加拿大高级研究所(CIFAR))
STAMP: Outlier-Aware Test-Time Adaptation with Stable Memory Replay
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);University of Science and Technology of China(中国科学技术大学)
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
Department of Computer Engineering, University of Texas at San Antonio, USA(德克萨斯大学圣安东尼奥分校计算机工程系);Department of Computer Science, University of Rochester, USA(罗切斯特大学计算机科学系);Department of Psychology, University of Texas at San Antonio, USA(德克萨斯大学圣安东尼奥分校心理学系)
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);Beijing Zhongguancun Academy(北京中关村学院);StepFun(阶跃星辰);Huazhong University of Science and Technology(华中科技大学);IAAR(智能自动化研究院)
Towards Lifelong Aerial Autonomy: Geometric Memory Management for Continual Visual Place Recognition in Dynamic Environments
Department of Precision Instrument, Tsinghua University(清华大学精密仪器系);College of Instrument Science and Opto-electronics Engineering, Beijing Information Science and Technology University(北京信息科技大学仪器科学与光电工程学院);Key Laboratory of Complex System Intelligent Control and Decision Making, Beijing Institute of Technology(北京理工大学复杂系统智能控制与决策重点实验室);School of Aerospace Engineering, Beijing Institute of Technology(北京理工大学宇航学院)
Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning
The University of Hong Kong(香港大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Huawei Cloud Computing Technologies(华为云计算技术);Ola Dimensions;HKU Shanghai Intelligent Computing Research Center(香港大学上海智能计算研究中心)