组装基础模型以实现自动代码摘要
Assemble Foundation Models for Automatic Code Summarization
摘要：
自动代码摘要有助于减少对手工编写的需求，有利于日常软件开发。当前，人工智能正在经历一场范式转变。基础模型在海量数据上进行预训练，并针对下游任务进行微调，超越了专门定制的模型。这种趋势促使我们考虑重用基础模型，而不是从头开始学习。因此，我们提出了一个灵活的和强大的自动代码摘要的方法，基于神经模型。我们将现有的基础模型，如CodeBERT和GPT-2，组装成一个名为AdaMo的神经模型。此外，我们利用高斯噪声作为上下文信息的模拟来优化潜在表示。此外，我们从知识转移的角度介绍了两种自适应方案，即连续预训练和中间微调，并设计了一般序列到序列学习的中间阶段任务。最后，我们通过将AdaMo与最先进的模型进行比较，来评估AdaMo对代码摘要的基准数据集。
英文摘要：
Automatic code summarization is beneficial to daily software development since it could help reduce the requirement of manual writing. Currently, artificial intelligence is undergoing a paradigm shift. The foundation models pretrained on massive data and finetuned to downstream tasks surpass specially customized models. This trend inspired us to consider reusing foundation models instead of learning from scratch. Thereby, we propose a flexible and robust approach for automatic code summarization, based on neural models. We assemble available foundation models, such as CodeBERT and GPT-2, into a single neural model named AdaMo. Moreover, we utilize Gaussian noise as the simulation of contextual information to optimize the latent representation. Furthermore, we introduce two adaptive schemes from the perspective of knowledge transfer, namely continuous pretraining and intermediate finetuning, and design intermediate stage tasks for general sequence-to-sequence learning. Finally, we evaluate AdaMo against a benchmark dataset for code summarization, by comparing it with state-of-the-art models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
