通过子空间正交投影进行梯度解冲突以用于多任务学习
Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning
AI总结：
针对多任务学习中任务间梯度冲突导致性能下降的问题，提出基于子空间正交投影的梯度解冲突方法GradOPS，有效消除任务冲突并搜索多样化权衡方案，在多个基准上取得了最先进的表现。
摘要：
尽管多任务学习（MTL）已成为首选方法并成功应用于许多现实场景，但 MTL 模型并不能保证在所有任务上都优于单任务模型，这主要是由于任务间冲突梯度的负面影响。在本文中，我们全面考察了冲突梯度的影响，并进一步强调了实现无冲突梯度的重要性与优势，这使得任务间能够采用简单但有效的权衡策略并保持稳定的性能。基于我们的发现，我们提出了通过由其他任务特定梯度张成的子空间正交投影进行梯度解冲突（GradOPS）。我们的方法不仅解决了任务间的所有冲突，还能有效地针对任务间不同的权衡偏好搜索多样化的解决方案。我们提供了关于收敛性的理论分析，并在各个领域的多个基准上充分验证了算法的性能。结果表明，我们的方法能够在多个数据集上有效地找到具有不同任务间权衡策略的多个最先进解决方案。
英文摘要：
Although multi-task learning (MTL) has been a preferred approach and successfully applied in many real-world scenarios, MTL models are not guaranteed to outperform single-task models on all tasks mainly due to the negative effects of conflicting gradients among the tasks. In this paper, we fully examine the influence of conflicting gradients and further emphasize the importance and advantages of achieving non-conflicting gradients which allows simple but effective trade-off strategies among the tasks with stable performance. Based on our findings, we propose the Gradient Deconfliction via Orthogonal Projections onto Subspaces (GradOPS) spanned by other task-specific gradients. Our method not only solves all conflicts among the tasks, but can also effectively search for diverse solutions towards different trade-off preferences among the tasks. Theoretical analysis on convergence is provided, and performance of our algorithm is fully testified on multiple benchmarks in various domains. Results demonstrate that our method can effectively find multiple state-of-the-art solutions with different trade-off strategies among the tasks on multiple datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
