检验多任务学习中的常见范式
Examining Common Paradigms in Multi-Task Learning
AI总结：
该研究结合单任务学习背景探究多任务学习范式，验证了Adam优化器的关键作用并推导其部分损失尺度不变性，指出梯度幅度差异而非角度对齐是多任务学习的特有问题，发现两类学习存在诸多相似性。
摘要：
尽管多任务学习（MTL）近年来受到广泛关注，但其底层机制仍未被充分理解。现有方法相较于单任务学习（STL）基线并未带来一致的性能提升，这凸显了深入洞察MTL特有挑战的重要性。本研究结合STL背景探究MTL中的范式：首先，优化器选择的影响在MTL中仅得到了轻度研究。我们通过多组实验，从经验层面证明了Adam优化器等常见STL工具在MTL中的关键作用。为进一步探究Adam的有效性，我们在温和假设下从理论上推导出了一种部分损失尺度不变性。其次，梯度冲突的概念常被表述为MTL中的特有问题。我们深入研究了梯度冲突在MTL中的作用，并将其与STL中的情况进行对比。对于角度梯度对齐，我们未发现证据表明这是MTL中的独有问题。我们强调梯度幅度的差异是主要区分因素。总体而言，我们发现STL与MTL之间存在令人惊讶的相似性，这表明应在更广泛的语境下考量两个领域的方法。
英文摘要：
While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we investigate paradigms in MTL in the context of STL: First, the impact of the choice of optimizer has only been mildly investigated in MTL. We show the pivotal role of common STL tools such as the Adam optimizer in MTL empirically in various experiments. To further investigate Adam's effectiveness, we theoretical derive a partial loss-scale invariance under mild assumptions. Second, the notion of gradient conflicts has often been phrased as a specific problem in MTL. We delve into the role of gradient conflicts in MTL and compare it to STL. For angular gradient alignment we find no evidence that this is a unique problem in MTL. We emphasize differences in gradient magnitude as the main distinguishing factor. Overall, we find surprising similarities between STL and MTL suggesting to consider methods from both fields in a broader context.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
