arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.01480cs.LGmath.OC
收藏

在机器学习中使用多目标优化时需避免的常见陷阱

Common pitfalls to avoid while using multiobjective optimization in machine learning

TU Dortmund(多特蒙德大学);University of Paderborn(帕德博恩大学)

Junaid Akhter, Paul David Fährmann, Konstantin Sonntag, Sebastian Peitz, Daniel Schwietert

AI总结:

本文为机器学习从业者提供多目标优化(MOO)的入门指南,通过物理信息神经网络(PINNs)等示例对比WS、MGDA和NSGA-II方法,指出了应用MOO时需避免的常见陷阱与挑战。

摘要:

近年来,多目标优化(MOO)在机器学习(ML)中的应用日益受到关注。这种兴趣源于众多现实场景中必须同时优化多个目标。MOO的一个关键特征是存在Pareto集,而非单一的最优解,该集代表了不同目标之间的最优权衡。尽管其具有潜力,但明显缺乏令人满意的文献作为旨在有效应用MOO的ML从业者的入门指南。在本文中,我们的目标是提供这样一份资源,并强调需要避免的陷阱。我们首先为MOO奠定基础,重点关注诸如加权和(WS)方法等广为人知的方法,以及像多目标梯度下降算法(MGDA)等更高级的技术。我们批判性地回顾了应用了MOO的各个ML领域的现有研究,并识别出可能导致错误解释的挑战。这些领域之一是物理信息神经网络(PINNs),我们将其作为指导性示例,仔细构建实验以说明这些陷阱。通过将WS和MGDA与最常见的进化算法之一NSGA-II进行比较,我们证明无论使用哪种特定的MOO方法,都可能出现困难。我们强调理解特定问题、目标空间和所选MOO方法的重要性,同时指出忽略收敛准则等因素会导致误导性的实验。

英文摘要:

Recently, there has been an increasing interest in the application of multiobjective optimization (MOO) in machine learning (ML). This interest is driven by the numerous real-life situations where multiple objectives must be optimized simultaneously. A key aspect of MOO is the existence of a Pareto set, rather than a single optimal solution, which represents the optimal trade-offs between different objectives. Despite its potential, there is a noticeable lack of satisfactory literature serving as an entry-level guide for ML practitioners aiming to apply MOO effectively. In this paper, our goal is to provide such a resource and highlight pitfalls to avoid. We begin by establishing the groundwork for MOO, focusing on well-known approaches such as the weighted sum (WS) method, alongside more advanced techniques like the multiobjective gradient descent algorithm (MGDA). We critically review existing studies across various ML fields where MOO has been applied and identify challenges that can lead to incorrect interpretations. One of these fields is physics informed neural networks (PINNs), which we use as a guiding example to carefully construct experiments illustrating these pitfalls. By comparing WS and MGDA with one of the most common evolutionary algorithms, NSGA-II, we demonstrate that difficulties can arise regardless of the specific MOO method used. We emphasize the importance of understanding the specific problem, the objective space, and the selected MOO method, while also noting that neglecting factors such as convergence criteria can result in misleading experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑