通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2605.21951cs.LG
动态潜在记忆混合用于自演化智能体
Dynamic Mixture of Latent Memories for Self-Evolving Agents
The Chinese University of Hong Kong（香港中文大学）；University of Oxford（牛津大学）；Nanyang Technological University（南洋理工大学）；University of Edinburgh（爱丁堡大学）；Shanghai Jiao Tong University（上海交通大学）
Dianzhi Yu, Vireo Zhang, Hongru Wang, Yanyu Chen, Minda Hu, Wanghan Xu, Siki Chen, Philip Torr, Zhenfei Yin, Irwin King
Achieving self-evolution in intelligent agents requires the continual accumulation of new knowledge across changing task sequences without forgetting previously acquired abilities. Existing approaches either internalize knowledge by updating model parameters, which induces catastrophic forgetting, or rely on external memory, which fails to genuinely enhance the model's intrinsic capabilities. We propose MoLEM, a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE). We treat multiple experts as independent carriers to generate memory. A router selects and weights experts through key-query matching, and the aggregated latent memory is injected into the reasoning process. The base model for reasoning remains entirely frozen, with all experiential knowledge internalized into the additional modules, avoiding catastrophic forgetting. For continual learning, each training stage is paired with a lightweight autoencoder that selects the appropriate routing group at inference, and inputs that match no stage fall back to the pretrained model. Experiments train the framework on continual-learning sequences spanning math, science, and code domains. After training, we evaluate the framework on the corresponding test sets to measure task learning and competence preservation across continual adaptation stages. After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline, while none of the competing methods consistently exceed this baseline across different training orders.
MoM: Linear Sequence Modeling with Mixture-of-Memories
Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；Fudan University（复旦大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）
Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment
Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所FIT）；University of Cologne（科隆大学）；University of Stuttgart（斯图加特大学）；University of Southampton（南安普顿大学）；Soochow University（苏州大学）；University Hospital of Cologne（科隆大学医院）
VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
Hong Kong University of Science and Technology (HKUST)（香港科学与技术大学）；Huawei Technologies Ltd.（华为技术有限公司）；Zhejiang University（浙江大学）；Shanghai AI Laboratory（上海人工智能实验室）
MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
Rochester Institute of Technology（罗切斯特理工学院）；University of Rochester（罗切斯特大学）；Meta AI；Rice University（莱斯大学）；Case Western Reserve University（凯斯西储大学）；U.S. DEVCOM Army Research Laboratory（美国陆军作战能力发展司令部陆军研究实验室）；University of Missouri - Kansas City（密苏里大学堪萨斯城分校）；UC Davis（加州大学戴维斯分校）
Towards a Statistical Understanding of Mixture-of-Experts
Qiuzhen College, Tsinghua University（清华大学求真书院）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）；East China Normal University（华东师范大学）；Beijing Institute of Mathematical Sciences and Applications (BIMSA)（北京数学科学与应用研究院）
MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents
Nanyang Technological University（南洋理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Tsinghua University（清华大学）
In this work, we propose to learn a generative model using both learned features (through a latent space) and memories (through neighbors). Although human learning makes seamless use of both learned perceptual features and instance recall, current generative learning paradigms only make use of one of these two components. Take, for instance, flow models, which learn a latent space of invertible features that follow a simple distribution. Conversely, kernel density techniques use instances to shift a simple distribution into an aggregate mixture model. Here we propose multiple methods to enhanc…
Duke University（杜克大学）；UC Berkeley（加州大学伯克利分校）；Princeton University（普林斯顿大学）
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection between gradient-based meta-learning and hierarchical Bayes to propose a Dirichlet process mixture of hierarchical Bayesian models over the parameters of an arbitrary parametric model such as a neural network. In contrast to consolidating inductive biases into a single set of hyperparameters, our approac…
GenMem: Generative Symbolic Memory for Self-Evolving Harness
National Engineering Research Center of Software Engineering, Peking University（北京大学软件工程国家工程研究中心）；School of Computer Science, Peking University（北京大学计算机学院）；Key Laboratory of High Confidence Software Technologies, Ministry of Education（教育部高可信软件技术重点实验室）；Center on Frontiers of Computing Studies, Peking University（北京大学计算前沿研究中心）
Evolving Graph Learning for Out-of-Distribution Generalization in Non-stationary Environments
Beijing Advanced Innovation Center for Big Data and Brain Computing, School of Computer Science and Engineering, Beihang University（北京大数据与脑计算先进创新中心，计算机科学与工程学院，北京航空航天大学）；Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University（教育区块链与智能技术重点实验室，教育部，广西师范大学）；Department of Computer Science, University of Illinois at Chicago（计算机科学系，伊利诺伊大学芝加哥分校）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Amazon（亚马逊）；IBM Research NY（IBM 纽约研究院）
We consider a covariate shift problem where one has access to several different training datasets for the same learning problem and a small validation set which possibly differs from all the individual training distributions. This covariate shift is caused, in part, due to unobserved features in the datasets. The objective, then, is to find the best mixture distribution over the training datasets (with only observed features) such that training a learning algorithm using this mixture has the best validation performance. Our proposed algorithm, ${\sf Mix\&Match}$, combines stochastic gradient d…
MIT（麻省理工学院）；Duke University（杜克大学）；Ohio State University（俄亥俄州立大学）
This paper presents a novel algorithm, based upon the dependent Dirichlet process mixture model (DDPMM), for clustering batch-sequential data containing an unknown number of evolving clusters. The algorithm is derived via a low-variance asymptotic analysis of the Gibbs sampling algorithm for the DDPMM, and provides a hard clustering with convergence guarantees similar to those of the k-means algorithm. Empirical results from a synthetic test with moving Gaussian clusters and a test with real ADS-B aircraft trajectory data demonstrate that the algorithm requires orders of magnitude less computa…
Singapore University of Technology and Design（新加坡科技设计大学）；University of Washington（华盛顿大学）
The predominant paradigm in evolutionary game theory and more generally online learning in games is based on a clear distinction between a population of dynamic agents that interact given a fixed, static game. In this paper, we move away from the artificial divide between dynamic agents and static games, to introduce and analyze a large class of competitive settings where both the agents and the games they play evolve strategically over time. We focus on arguably the most archetypal game-theoretic setting -- zero-sum games (as well as network generalizations) -- and the most studied evolutiona…
Sequences of events including infectious disease outbreaks, social network activities, and crimes are ubiquitous and the data on such events carry essential information about the underlying diffusion processes between communities (e.g., regions, online user groups). Modeling diffusion processes and predicting future events are crucial in many applications including epidemic control, viral marketing, and predictive policing. Hawkes processes offer a central tool for modeling the diffusion processes, in which the influence from the past events is described by the triggering kernel. However, the …
