大代码！=大词汇表：Source Code的开放词汇表模型
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
摘要：
统计语言建模技术已成功应用于大型源代码语料库，产生了各种新的软件开发工具，如代码建议工具，提高可读性和API迁移。这些技术的一个主要问题是，随着新标识符名称的激增，代码以比自然语言高得多的速率引入新词汇表。词汇量大和词汇量不足的问题都会严重影响源代码的神经语言模型（NLM），降低其性能并使其无法扩展。在本文中，我们解决这个问题：1）研究各种建模选择如何影响13，362个项目的大规模语料库上的最终词汇; 2）提出一个开放的词汇源代码NLM，可以扩展到这样的语料库，比以前的工作大100倍;以及3）表明这样的模型在三种不同的代码语料库（Java、C、Python）上优于现有技术。据我们所知，这些是已报告的代码的最大NLM。在这项工作中使用的所有数据集，代码和训练模型都是公开的。
英文摘要：
Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with these techniques is that code introduces new vocabulary at a far higher rate than natural language, as new identifier names proliferate. Both large vocabularies and out-of-vocabulary issues severely affect Neural Language Models (NLMs) of source code, degrading their performance and rendering them unable to scale. In this paper, we address this issue by: 1) studying how various modelling choices impact the resulting vocabulary on a large-scale corpus of 13,362 projects; 2) presenting an open vocabulary source code NLM that can scale to such a corpus, 100 times larger than in previous work; and 3) showing that such models outperform the state of the art on three distinct code corpora (Java, C, Python). To our knowledge, these are the largest NLMs for code that have been reported. All datasets, code, and trained models used in this work are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
