Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximization
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；Leiden University Medical Center（莱顿大学医学中心）；Delft University of Technology（代尔夫特理工大学）
Timo M. Deist, Monika Grewal, Frank J. W. M. Dankers, Tanja Alderliesten, Peter A. N. Bosman
备注：
T.M.D. and M.G. contributed equally. Changes in new version (v2): improved method, added comparison to method described in old manuscript (v1), added experiments, added appendix, revised text
Real-world problems are often multi-objective with decision-makers unable to specify a priori which trade-off between the conflicting objectives is preferable. Intuitively, building machine learning solutions in such cases would entail providing multiple predictions that span and uniformly cover the Pareto front of all optimal trade-off solutions. We propose a novel approach for multi-objective training of neural networks to approximate the Pareto front during inference. In our approach, the neural networks are trained multi-objectively using a dynamic loss function, wherein each network's losses (corresponding to multiple objectives) are weighted by their hypervolume maximizing gradients. We discuss and illustrate why training processes to approximate Pareto fronts need to optimize on fronts of individual training samples instead of on only the front of average losses. Experiments on three multi-objective problems show that our approach returns outputs that are well-spread across different trade-offs on the approximated Pareto front without requiring the trade-off vectors to be specified a priori. Further, results of comparisons with the state-of-the-art approaches highlight the added value of our proposed approach, especially in asymmetric Pareto fronts.
This paper introduces the hypervolume maximization with a single solution as an alternative to the mean loss minimization. The relationship between the two problems is proved through bounds on the cost function when an optimal solution to one of the problems is evaluated on the other, with a hyperparameter to control the similarity between the two problems. This same hyperparameter allows higher weight to be placed on samples with higher loss when computing the hypervolume's gradient, whose normalized version can range from the mean loss to the max loss. An experiment on MNIST with a neural ne…
Single-objective black box optimization (also known as zeroth-order optimization) is the process of minimizing a scalar objective $f(x)$, given evaluations at adaptively chosen inputs $x$. In this paper, we consider multi-objective optimization, where $f(x)$ outputs a vector of possibly competing objectives and the goal is to converge to the Pareto frontier. Quantitatively, we wish to maximize the standard hypervolume indicator metric, which measures the dominated hypervolume of the entire set of chosen inputs. In this paper, we introduce a novel scalarization function, which we term the hyper…
Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning
Vrije Universiteit Brussel（布鲁塞尔自由大学）；Université de Montréal（蒙特利尔大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；University of Galway（戈尔韦大学）；City of Amsterdam（阿姆斯特丹市）；Utrecht University（乌得勒支大学）
INRS-EMT, Université du Québec（魁北克大学INRS-EMT分校）；Desautels Faculty of Management, McGill University（麦吉尔大学德索泰尔管理学院）；Quebec Artificial Intelligence Institute, Université de Montréal（蒙特利尔大学魁北克人工智能研究所）
Recent literature has demonstrated promising results for training Generative Adversarial Networks by employing a set of discriminators, in contrast to the traditional game involving one generator against a single adversary. Such methods perform single-objective optimization on some simple consolidation of the losses, e.g. an arithmetic average. In this work, we revisit the multiple-discriminator setting by framing the simultaneous minimization of losses provided by different models as a multi-objective optimization problem. Specifically, we evaluate the performance of multiple gradient descent…
Optimizing multiple competing black-box objectives is a challenging problem in many fields, including science, engineering, and machine learning. Multi-objective Bayesian optimization (MOBO) is a sample-efficient approach for identifying the optimal trade-offs between the objectives. However, many existing methods perform poorly when the observations are corrupted by noise. We propose a novel acquisition function, NEHVI, that overcomes this important practical limitation by applying a Bayesian treatment to the popular expected hypervolume improvement (EHVI) criterion and integrating over this …
Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition
Department of Computer Science, Aalto University, Espoo, Finland（阿尔托大学计算机科学系，芬兰 Espoo）；ELLIS Institute Finland（芬兰 ELLIS 机构）；Department of Computer Science, University of Manchester, Manchester, United Kingdom（曼彻斯特大学计算机科学系，英国 Manchester）
We present a multi-objective Bayesian optimisation algorithm that allows the user to express preference-order constraints on the objectives of the type "objective A is more important than objective B". These preferences are defined based on the stability of the obtained solutions with respect to preferred objective functions. Rather than attempting to find a representative subset of the complete Pareto front, our algorithm selects those Pareto-optimal points that satisfy these constraints. We formulate a new acquisition function based on expected improvement in dominated hypervolume (EHI) to e…
Sample-Efficient Pareto Front Modeling for Energy-Aware Reinforcement Learning Using Bayesian Optimization
Josef Ressel Centre for Intelligent and Secure Industrial Automation, Salzburg University of Applied Sciences（萨尔茨堡应用科学大学智能与安全工业自动化约瑟夫·雷塞尔中心）；Department of Artificial Intelligence and Human Interfaces, Paris Lodron University of Salzburg（萨尔茨堡巴黎洛德龙大学人工智能与人类交互系）
Improving Pareto Front Learning via Multi-Sample Hypernetworks
College of Engineering and Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）；School of Applied Mathematics and Informatics, Hanoi University of Science and Technology（河内科技大学应用数学与信息学院）
In multi-objective Bayesian optimization and surrogate-based evolutionary algorithms, Expected HyperVolume Improvement (EHVI) is widely used as the acquisition function to guide the search approaching the Pareto front. This paper focuses on the exact calculation of EHVI given a nondominated set, for which the existing exact algorithms are complex and can be inefficient for problems with more than three objectives. Integrating with different decomposition algorithms, we propose a new method for calculating the integral in each decomposed high-dimensional box in constant time. We develop three n…
GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning
University of California, Los Angeles（加州大学洛杉矶分校）；USC Institute for Creative Technologies（美国南加州大学创意技术研究所）；U.S. Army DEVCOM Army Research Laboratory（美国陆军 DEVCOM 军事研究实验室）
A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
International Research Center for Neurointelligence, The University of Tokyo（国际神经智能研究中心，东京大学）；ATR Neural Information Analysis Laboratories（ATR神经信息分析实验室）；Department of Information Science, Kyoto University（京都大学信息科学系）
East China Normal University（华东师范大学）；Southern University of Science and Technology（南方科技大学）
Multi-objective combinatorial optimization (MOCO) problems are prevalent in various real-world applications. Most existing neural MOCO methods rely on problem decomposition to transform an MOCO problem into a series of singe-objective combinatorial optimization (SOCO) problems and train attention models based on a single-step and deterministic greedy rollout. However, inappropriate decomposition and undesirable short-sighted behaviors of previous methods tend to induce a decline in diversity. To address the above limitation, we design a Context-aware Diversity Enhancement algorithm named CDE, …
TNO（荷兰国家应用科学研究院）；University of Exeter（埃克塞特大学）；Swansea University（斯旺西大学）；Delft University of Technology（代尔夫特理工大学）
Performing multi-objective Bayesian optimisation by scalarising the objectives avoids the computation of expensive multi-dimensional integral-based acquisition functions, instead of allowing one-dimensional standard acquisition functions\textemdash such as Expected Improvement\textemdash to be applied. Here, two infill criteria based on hypervolume improvement\textemdash one recently introduced and one novel\textemdash are compared with the multi-surrogate Expected Hypervolume Improvement. The reasons for the disparities in these methods' effectiveness in maximising the hypervolume of the acqu…
Diversity-Driven Offline Multi-Objective Optimization via Nested Pareto Set Learning
Shanghai Institute of AI for Education（上海人工智能教育研究院）；School of Computer Science（计算机科学学院）；East China Normal University（华东师范大学）；Shanghai Innovation Institute（上海创新研究院）
University of South Carolina（南卡罗来纳大学）；University of Wyoming（怀俄明大学）
The design of machine learning systems often requires trading off different objectives, for example, prediction error and energy consumption for deep neural networks (DNNs). Typically, no single design performs well in all objectives; therefore, finding Pareto-optimal designs is of interest. The search for Pareto-optimal designs involves evaluating designs in an iterative process, and the measurements are used to evaluate an acquisition function that guides the search process. However, measuring different objectives incurs different costs. For example, the cost of measuring the prediction erro…
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Shenzhen CyberAray Network Technology Co., Ltd, China（深圳 CyberAray 网络技术有限公司）；Information and Communication Branch of State Grid Hebei Electric Power Co., Ltd, China（国网河北电力信息通信分部）；Zhejiang Lab as well as Zhejiang University（浙江实验室以及浙江大学）
Federated learning (FL) in multi-service provider (SP) ecosystems is fundamentally hampered by non-cooperative dynamics, where privacy constraints and competing interests preclude the centralized optimization of multi-SP communication and computation resources. In this paper, we introduce PAC-MCoFL, a game-theoretic multi-agent reinforcement learning (MARL) framework where SPs act as agents to jointly optimize client assignment, adaptive quantization, and resource allocation. Within the framework, we integrate Pareto Actor-Critic (PAC) principles with expectile regression, enabling agents to c…
A Two-Phase Adaptive Balanced Penalty Method for Controllable Pareto Front Learning under Split Feasibility Conditions
Faculty of Applied Mathematics and Informatics, Hanoi University of Science and Technology（应用数学与信息技术学院，河内科学技术大学）；College of Engineering and Computer Science, VinUniversity（工程与计算机科学学院，Vin大学）
The Offline-Frontier Shift: Diagnosing Distributional Limits in Generative Multi-Objective Optimization
LIT AI Lab and Institute for Machine Learning JKU Linz, Austria（LIT人工智能实验室和机器学习研究所，约翰·肯普夫大学（JKU）林茨分校，奥地利）；School of Computing, Engineering and Technology RGU, Aberdeen, Scotland（计算、工程与技术学院 RGU，阿伯丁，苏格兰）；Linz Center of Mechatronics GmbH Linz, Austria（林茨机电研究中心 GmbH，林茨，奥地利）
Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；University of Oxford（牛津大学）；School of Computer Science, Beijing Institute of Technology（北京理工大学计算机学院）；Nanyang Technological University（南洋理工大学）
Groupe PSA（标致雪铁龙集团）；École Nationale Supérieure des Mines de Saint-Étienne（圣艾蒂安国立高等矿业学校）；CNRS LIMOS（法国国家科学研究中心LIMOS实验室）；INRA Toulouse（法国国家农业科学研究院图卢兹分院）
Multi-objective optimization aims at finding trade-off solutions to conflicting objectives. These constitute the Pareto optimal set. In the context of expensive-to-evaluate functions, it is impossible and often non-informative to look for the entire set. As an end-user would typically prefer a certain part of the objective space, we modify the Bayesian multi-objective optimization algorithm which uses Gaussian Processes to maximize the Expected Hypervolume Improvement, to focus the search in the preferred region. The cumulated effects of the Gaussian Processes and the targeting strategy lead t…
Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment
Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所FIT）；University of Cologne（科隆大学）；University of Stuttgart（斯图加特大学）；University of Southampton（南安普顿大学）；Soochow University（苏州大学）；University Hospital of Cologne（科隆大学医院）
Autonomous AI Agents for Real-Time Affordable Housing Site Selection: Multi-Objective Reinforcement Learning Under Regulatory Constraints
Department of Applied Mathematics and Computer Science (DTU Compute), Technical University of Denmark（应用数学与计算机科学系（DTU Compute），丹麦技术大学）；Department of Mathematics, Trakya University（数学系，特拉基亚大学）；Department of Engineering, Eskisehir Technical University（工程系，埃斯基谢普尔技术大学）；Department of Computer Engineering, Afyon Kocatepe University（计算机工程系，阿菲昂卡奥卡佩大学）；Department of Computer Systems, Riga Technical University（计算机系统系，里加技术大学）
C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front
University of Oxford（牛津大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Queen Mary University of London（伦敦大学玛丽女王学院）；Microsoft Research Asia（微软亚洲研究院）；Peking University（北京大学）；University of Alberta（阿尔伯塔大学）
University of Notre Dame（圣母大学）；Snap Inc.（色拉布公司）；Brandeis University（布兰迪斯大学）
Self-supervised learning (SSL) for graph neural networks (GNNs) has attracted increasing attention from the graph machine learning community in recent years, owing to its capability to learn performant node embeddings without costly label information. One weakness of conventional SSL frameworks for GNNs is that they learn through a single philosophy, such as mutual information maximization or generative reconstruction. When applied to various downstream tasks, these frameworks rarely perform equally well for every task, because one philosophy may not span the extensive knowledge required for a…
Simulated Annealing-based Candidate Optimization for Batch Acquisition Functions
a Department of Materials Science；Engineering, Texas A\&M University, College Station, TX, USA 77843；b J. Mike Walker '66 Department of Mechanical Engineering, Texas A\&M University, College Station, TX, USA 77843；c Wm Michael Barnes '64 Department of Industrial；Systems Engineering, Texas A\&M University, College Station, TX, USA 77843
