提出 Composite Learning Units (CLUs) 框架，通过动态知识库和强化反馈循环，使 LLMs 无需参数更新即可实现持续学习与自适应推理，在密码学推理任务中显著优于常规模型。
摘要：
人类学习之所以卓有成效，在于能从错误中学习、通过反馈适应并完善理解——这些过程往往是静态机器学习模型所缺失的。在本工作中，我们引入了 Composite Learning Units (CLUs)，旨在将 Large Language Models (LLMs) 等推理器转化为具备泛化、持续学习能力的 learner，无需常规参数更新，同时通过持续交互和反馈增强其推理能力。CLUs 构建于一种允许推理模型维护并演化动态知识库的架构之上：包含用于广泛、可复用洞察的 General Knowledge Space 和用于任务特定学习的 Prompt-Specific Knowledge Space。通过目标驱动的交互，CLUs 迭代地精炼这些知识空间，使系统能够动态适应复杂任务、提取细微洞察并自主积累过往经验。我们通过一项密码学推理任务验证了 CLUs 的有效性，在此任务中，它们通过反馈持续演化理解以揭示隐藏的变换规则。当常规模型难以把握底层逻辑时，CLUs 凭借参与迭代、目标导向的过程而表现出色。专门处理知识检索、提示生成和反馈分析的组件在强化反馈循环中协同工作。该方法使 CLUs 能够保留对过往成败的记忆、自主适应并有效应用复杂推理，在从错误中持续学习的同时也基于突破不断构建。
英文摘要：
Human learning thrives on the ability to learn from mistakes, adapt through feedback, and refine understanding-processes often missing in static machine learning models. In this work, we introduce Composite Learning Units (CLUs) designed to transform reasoners, such as Large Language Models (LLMs), into learners capable of generalized, continuous learning without conventional parameter updates while enhancing their reasoning abilities through continual interaction and feedback. CLUs are built on an architecture that allows a reasoning model to maintain and evolve a dynamic knowledge repository: a General Knowledge Space for broad, reusable insights and a Prompt-Specific Knowledge Space for task-specific learning. Through goal-driven interactions, CLUs iteratively refine these knowledge spaces, enabling the system to adapt dynamically to complex tasks, extract nuanced insights, and build upon past experiences autonomously. We demonstrate CLUs' effectiveness through a cryptographic reasoning task, where they continuously evolve their understanding through feedback to uncover hidden transformation rules. While conventional models struggle to grasp underlying logic, CLUs excel by engaging in an iterative, goal-oriented process. Specialized components-handling knowledge retrieval, prompt generation, and feedback analysis-work together within a reinforcing feedback loop. This approach allows CLUs to retain the memory of past failures and successes, adapt autonomously, and apply sophisticated reasoning effectively, continually learning from mistakes while also building on breakthroughs.
Technische Universität Berlin（柏林工业大学）；GT-ARC Gemeinnützige GmbH（GT-ARC公益有限责任公司）
This paper proposes a new family of algorithms for the online optimisation of composite objectives. The algorithms can be interpreted as the combination of the exponentiated gradient and $p$-norm algorithm. Combined with algorithmic ideas of adaptivity and optimism, the proposed algorithms achieve a sequence-dependent regret upper bound, matching the best-known bounds for sparse target decision variables. Furthermore, the algorithms have efficient implementations for popular composite objectives and constraints and can be converted to stochastic optimisation algorithms with the optimal acceler…
Technische Universität Berlin（柏林工业大学）；GT-ARC Gemeinnützige GmbH（GT-ARC公益有限责任公司）
In this paper, we propose and analyse a family of generalised stochastic composite mirror descent algorithms. With adaptive step sizes, the proposed algorithms converge without requiring prior knowledge of the problem. Combined with an entropy-like update-generating function, these algorithms perform gradient descent in the space equipped with the maximum norm, which allows us to exploit the low-dimensional structure of the decision sets for high-dimensional problems. Together with a sampling method based on the Rademacher distribution and variance reduction techniques, the proposed algorithms…
Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoning
Beijing Institute of Mathematical Sciences and Applications（北京数学科学与应用研究院）；Tsinghua University（清华大学）；Peking University（北京大学）；The University of Hong Kong（香港大学）
Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration
School of Computer Science and Engineering, Suzhou University of Technology（苏州科技大学计算机科学与工程学院）；School of Mathematics, Southeast University（东南大学数学学院）；School of Cyber Science and Engineering, Southeast University（东南大学网络科学与工程学院）
