判断亚当：研究优化方法在ML4SE任务上的性能
Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks
摘要：
用深度学习模型解决问题，需要研究人员用一定的优化方法对损失函数进行优化。研究界已经开发了100多种不同的优化器，但是关于优化器在各种任务中的性能的数据很少。特别是，没有一个基准测试测试优化器在源代码相关问题上的性能。然而，现有的基准数据表明，某些优化器可能对特定领域更有效。在这项工作中，我们在源代码的深度学习模型上测试了各种优化器的性能，并发现优化器的选择可能对模型质量产生重大影响，一些性能相对较好的优化器之间的得分差异高达两倍。我们还发现，RDam优化器（及其修改与前瞻性包络）是最好的优化器，几乎总是表现良好的任务，我们考虑。我们的研究结果表明，需要对代码相关任务中的优化器进行更广泛的研究，并表明ML4SE社区应该考虑使用RDam而不是Adam作为代码相关深度学习任务的默认优化器。
英文摘要：
Solving a problem with a deep learning model requires researchers to optimize the loss function with a certain optimization method. The research community has developed more than a hundred different optimizers, yet there is scarce data on optimizer performance in various tasks. In particular, none of the benchmarks test the performance of optimizers on source code-related problems. However, existing benchmark data indicates that certain optimizers may be more efficient for particular domains. In this work, we test the performance of various optimizers on deep learning models for source code and find that the choice of an optimizer can have a significant impact on the model quality, with up to two-fold score differences between some of the relatively well-performing optimizers. We also find that RAdam optimizer (and its modification with the Lookahead envelope) is the best optimizer that almost always performs well on the tasks we consider. Our findings show a need for a more extensive study of the optimizers in code-related tasks, and indicate that the ML4SE community should consider using RAdam instead of Adam as the default optimizer for code-related deep learning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
