基于有限内存MultiLRSGA的Nash-Bargaining HalpernSGD：一种用于多目标学习的两阶段优化器

中文摘要 AI 辅助

我们提出了基于LM-MultiLRSGA的NB-HalpernSGD，这是一种用于多目标优化的两阶段优化器。该过程首先进入竞争优化阶段，应用MultiLRSGA的有限内存变体LM-MultiLRSGA，该变体旨在在保留原始优化器旋转校正机制的同时降低内存占用。它通过考虑与原始多目标问题相关的竞争博弈来处理多目标任务，近似Nash均衡。该点定义了Nash均衡诱导的分歧点：我们构建与原始损失相关的Nash议价问题，并将其Nash乘积重写为对数最小化代理问题。该最小化问题使用HalpernSGD求解，求解时以计算出的竞争参考点为基准。因此，该方法将Nash均衡作为议价阶段的原则性参考点，然后向原始多目标问题的Pareto导向解移动。我们在受PINN启发的神经模型上验证了所提出的优化器，其性能优于已有的多目标优化器，包括PCGrad、MultiAdam和DualConeGD。最后，尽管HalpernSGD的收敛特性已得到广泛研究，我们讨论了所提出的LM-MultiLRSGA阶段的收敛性和稳定性特性。

英文摘要

We propose NB-HalpernSGD via LM-MultiLRSGA, a two-phase optimizer for multi-objective optimization. The process starts with a competitive optimization phase by applying a limited-memory variant of MultiLRSGA, named LM-MultiLRSGA, designed to reduce the memory footprint of the original optimizer while preserving its rotational correction mechanism. It addresses multi-objective tasks by considering the competitive game associated with the original multi-objective problem, approximating a Nash equilibrium. This point defines a Nash-equilibrium-induced disagreement point: we formulate the Nash bargaining problem associated with the original losses and rewrite its Nash product as a logarithmic minimization surrogate. This minimization problem is solved using HalpernSGD, anchored at the computed competitive reference point. Therefore, the method uses the Nash equilibrium as a principled reference point for the bargaining stage and then moves toward a Pareto-oriented solution of the original multi-objective problem. We validate the proposed optimizer on a PINN-inspired neural model, where it outperforms established multi-objective optimizers, including PCGrad, MultiAdam, and DualConeGD. Finally, while the convergence properties of HalpernSGD have been extensively studied, we discuss the convergence and stability properties of the proposed LM-MultiLRSGA phase.

发表机构

• University of Calabria（卡拉布里亚大学）
• Italian National Research Council（意大利国家研究委员会）

机构由 AI 辅助整理，请以论文原文为准。