Multi-objective optimization involves diverse preferences over conflicting objectives. Rather than returning a finite set of trade-off solutions, preference-conditioned Pareto set learning seeks to map a specified preference directly to a corresponding Pareto optimal solution. Yet in many real-world scenarios, optimization must proceed offline without further objective evaluations, leaving only a limited static dataset that sparsely covers the decision and objective spaces. Existing methods either optimize separately for each preference or train a problem-specific Pareto solution model, requiring repeated search across preferences or repeated training across problems. We introduce in-context preference-conditioned Pareto set learning (ICPSL), a framework that amortizes both preference-wise solution search and problem-specific model construction. ICPSL pretrains a foundation-style hypernetwork on a large and diverse set of synthetic prior data. Given the offline observations of an unseen problem, the pretrained hypernetwork learns to generate a problem-specific, preference-conditioned Pareto solution model in a single forward pass. The generated model can then be reused to predict solutions for arbitrary preferences without problem-specific training or preference-wise search. Experiments on synthetic and real-world problems show that ICPSL produces high-quality Pareto solutions while providing substantial speedups in problem-specific solution-model construction.
Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off
City University of Hong Kong(香港城市大学);HK Institute of Science & Innovation, Chinese Academy of Sciences(中国科学院香港创新研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);City University of Hong Kong Shenzhen Research Institute(香港城市大学深圳研究院);University of Science and Technology of China(中国科学技术大学)
Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
Tsinghua University(清华大学);University of Cambridge(剑桥大学);Eindhoven University of Technology(埃因霍温理工大学);King’s College London(伦敦国王学院);Tencent Robotics X(腾讯机器人实验室)
OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机技术研究所)
GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models
University of Warwick(华威大学);Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学);University of Cambridge(剑桥大学);University of Liverpool(利物浦大学)
Self-Evolutionary Optimization for Pareto Front Learning
Seoul National University(首尔国立大学)
多任务学习(MTL)旨在通过同时学习多个任务来提高性能,由于多个目标,它固有地提出了优化挑战。因此,多目标优化(MOO)方法已被提出用于多任务问题。最近的MOO方法用一个统一的模型来近似多个最优解(Pareto front),这被统称为Pareto front learning(PFL)。在本文中,我们表明,PFL可以重新制定成另一个多目标的MOO问题,每个目标对应于不同的偏好权重的任务。我们利用进化算法(EA)提出了一种方法,PFL称为自进化优化(SEO)直接最大化的超体积。通过使用SEO,神经网络学习以多个极大地影响超体积的超参数为条件来近似帕累托前沿。然后,通过简单地通过推断网络来生成近似的群体,可以通过EA来优化网络的超参数。利用PFL的SEO,我们还引入了自进化帕累托网络(SEPNet),使统一模型能够近似整个帕累托前集,最大化超体积。大量的实验结果证实,SEPNet可以找到比当前最先进的方法更好的Pareto前沿,同时最大限度地减少模型大小和训练成本的增加。
Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL
Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea(首尔国立大学电气与计算机工程系);Department of Automotive Engineering, Ajou University, Gyeonggi-do, South Korea(全州大学汽车工程系)
Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning
Department of Electrical and Computer Engineering (ECE), Seoul National University(电子与计算机工程系,首尔国立大学);Interdisciplinary Program in Artificial Intelligence (IPAI), Seoul National University(人工智能跨学科项目,首尔国立大学);ASRI / INMC, Seoul National University(ASRI/INMC,首尔国立大学)
Department of Electrical Engineering and Computer Science, Pennsylvania State University, PA, USA(电气工程与计算机科学系,宾夕法尼亚州立大学);College of Information Sciences and Technology, Pennsylvania State University, PA, USA(信息科学与技术学院,宾夕法尼亚州立大学)
Preference-Conditioned Multi-Objective Reinforcement Learning for Tactical Decision Making in Heavy-Duty Vehicles
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg(计算机科学与工程系,查尔姆斯理工大学和哥德堡大学);Department of Mechanics and Maritime Sciences, Chalmers University of Technology(机械与海洋科学系,查尔姆斯理工大学)
Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);Beijing Forestry University(北京林业大学);Peking University(北京大学);Microsoft Research(微软研究院)