单调变换不变的多任务学习
Monotonic Transformation Invariant Multi-task Learning
AI总结：
本文提出DiBS-MTL方法，通过单调变换不变性提升多任务学习的鲁棒性，实验证明其在非凸任务损失场景下具有优越性能。
摘要：
多任务学习（MTL）算法通常依赖于通过加权平均结合不同任务损失或其梯度的方案。这些方法旨在通过使用需要访问任务损失值、梯度或两者的启发式方法来寻找帕累托平稳点。在此过程中，一个核心挑战是任务损失可以相对于彼此任意缩放，导致某些任务主导训练并降低整体性能。最近在合作谈判理论中的进展，方向基谈判解法（DiBS），通过其对单调非线性任务损失变换的不变性，产生对任务主导免疫的帕累托平稳解。然而，DiBS在非凸MTL设置中的收敛行为目前尚不明确。为此，我们证明在标准假设下，当任务损失非凸时，DiBS迭代的子序列收敛于帕累托平稳点，并提出DiBS-MTL，即一种适应于MTL设置的DiBS变体，比以往的谈判启发式MTL方法更具计算效率。最后，我们实证显示，DiBS-MTL在标准基准上与领先的MTL方法具有竞争力，并在多个具有不良缩放任务损失的例子中大幅优于最先进的基线方法，突显了对损失景观非线性单调变换不变性的重要性。代码可在https://github.com/suryakmurthy/dibs-mtl获取。
英文摘要：
Multi-task learning (MTL) algorithms typically rely on schemes that combine different task losses or their gradients through weighted averaging. These methods aim to find Pareto stationary points by using heuristics that require access to task loss values, gradients, or both. In doing so, a central challenge arises because task losses can be arbitrarily scaled relative to one another, causing certain tasks to dominate training and degrade overall performance. A recent advance in cooperative bargaining theory, the Direction-based Bargaining Solution (DiBS), yields Pareto stationary solutions immune to task domination because of its invariance to monotonic nonaffine task loss transformations. However, the convergence behavior of DiBS in nonconvex MTL settings is currently not understood. To this end, we prove that under standard assumptions, a subsequence of DiBS iterates converges to a Pareto stationary point when task losses are nonconvex, and propose DiBS-MTL, an adaptation of DiBS to the MTL setting which is more computationally efficient that prior bargaining-inspired MTL approaches. Finally, we empirically show that DiBS-MTL is competitive with leading MTL methods on standard benchmarks, and it drastically outperforms state-of-the-art baselines in multiple examples with poorly-scaled task losses, highlighting the importance of invariance to nonaffine monotonic transformations of the loss landscape. Code available at https://github.com/suryakmurthy/dibs-mtl
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
