CodeTrans：通过自我监督深度学习和高性能计算破解硅代码语言
CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing
摘要：
目前，越来越多成熟的自然语言处理应用使人们的生活更加方便。这样的应用程序是由源代码构建的--源代码是软件工程中的语言。然而，理解源代码语言，以简化软件工程过程的应用研究不足。同时，Transformer模型，特别是它与迁移学习的结合，已被证明是自然语言处理任务的强大技术。这些突破为处理源代码和破解软件工程任务指明了一个有希望的方向。本文描述了CodeTrans --一种软件工程领域任务的编码器-解码器转换器Transformer模型，探讨了编码器-解码器转换器Transformer模型对6个软件工程任务（包括13个子任务）的有效性。此外，我们还研究了不同训练策略的效果，包括单任务学习、迁移学习、多任务学习和微调多任务学习。CodeTrans在所有任务上都优于最先进的模型。为了加快软件工程领域的未来工作，我们发布了我们的CodeTrans预训练模型。https://github.com/agemagician/CodeTrans
英文摘要：
Currently, a growing number of mature natural language processing applications make people's life more convenient. Such applications are built by source code - the language in software engineering. However, the applications for understanding source code language to ease the software engineering process are under-researched. Simultaneously, the transformer model, especially its combination with transfer learning, has been proven to be a powerful technique for natural language processing tasks. These breakthroughs point out a promising direction for process source code and crack software engineering tasks. This paper describes CodeTrans - an encoder-decoder transformer model for tasks in the software engineering domain, that explores the effectiveness of encoder-decoder transformer models for six software engineering tasks, including thirteen sub-tasks. Moreover, we have investigated the effect of different training strategies, including single-task learning, transfer learning, multi-task learning, and multi-task learning with fine-tuning. CodeTrans outperforms the state-of-the-art models on all the tasks. To expedite future works in the software engineering domain, we have published our pre-trained models of CodeTrans. https://github.com/agemagician/CodeTrans
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
