发表机构
Southern University of Science and Technology; City University of Hong Kong; Shanxi University（南方科技大学; 香港城市大学; 山西大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多任务学习中目标尺度差异导致优化失衡的问题，提出尺度不变的 merit 函数标量化方法（SIMS），通过对数变换实现尺度不变性，保持弱 Pareto 最优性，并在多任务基准上取得最先进性能。
AI 中文摘要
多任务学习（MTL）需要在相互竞争的目标之间处理不可避免的权衡。这一范式常被表述为多目标优化（MOO），其中标量化方法被青睐，以将 MOO 问题简化为单一目标。我们通过实验发现，现有的基于 merit 函数的标量化方法在实际 MTL 中对不同目标的相对尺度敏感，其中任务损失通常相差多个数量级。优化过程往往偏向于尺度较大的目标，尽管潜在的 Pareto 最优解对重新缩放（即，将目标乘以一个正常数）是不变的。为解决这一问题，我们提出了用于 MTL 的尺度不变 merit 函数标量化方法（SIMS）。具体而言，SIMS 采用一种变换诱导的 merit 函数，将 MTL 的 MOO 问题转化为一个单一目标，使得优化对损失的大小不敏感。理论上，我们证明了尺度不变性的要求唯一地确定了该变换为对数变换。我们进一步表明，这种通用的变换诱导 merit 函数保持了弱 Pareto 最优性，并允许具有可控逼近误差的平滑替代。在代表性多任务基准上的大量实验表明，SIMS 始终优于现有的标量化方法，并实现了最先进的性能。
英文摘要
Multi-task learning (MTL) requires navigating unavoidable trade-offs among competing objectives. This paradigm is frequently formulated as multi-objective optimization (MOO), where the scalarization is favored to reduce an MOO problem to a single objective. We empirically find that existing merit-function-based scalarization approaches are sensitive to the relative scales of different objectives in practical MTL, where task losses commonly differ by orders of magnitude. The optimization process often favors objectives with larger scales even though the underlying Pareto optimal solutions remains invariant to rescaling (i.e., multiplying an objective by a positive constant). To address this issue, we propose Scale-Invariant Merit-function-based Scalarization (SIMS) for MTL. Specifically, SIMS adopts a transformation-induced merit function to convert the MOO problem of MTL to a single objective that renders optimization invariant to the magnitudes of losses. Theoretically, we prove that the requirement for scale invariance uniquely determines this transformation to be logarithmic. We further show that this general transformation-induced merit function preserves weak Pareto optimality and admits a smooth surrogate with controllable approximation error. Extensive experiments on representative multi-task benchmarks demonstrate that SIMS consistently outperforms existing scalarization methods and achieves state-of-the-art performance.
CommentsAccepted by the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026)
Journal refProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD '26), pp. 520-531, 2026
