标识符分裂能改善代码的开放词汇语言模型吗？
Can Identifier Splitting Improve Open-Vocabulary Language Model of Code?
摘要：
基于源代码的统计语言模型已经成功地辅助了软件工程任务。但是，开发人员可以在编写源代码时创建或选择任意标识符。自由选择的标识符会导致臭名昭著的词汇表外（OOV）问题，对模型性能产生负面影响。最近，Karampatsis等人表明，使用字节对编码（BPE）算法来解决OOV问题可以提高语言模型对源代码的预测性能。然而，BPE的一个缺点是它不能以保留有意义的语义的方式分割标识符。先前的研究人员还表明，将复合标识符拆分为反映语义的子词可以使软件开发工具受益。这两个事实促使我们探索是否可以利用标识符分裂技术来增强BPE算法，并提高Karampatsis等人考虑的开放词汇表语言模型的性能。的工作。本文提出了在构建词汇和处理模型输入过程中分割标识符，从而利用三种不同的设置将标识符分割应用于代码完成任务的语言模型。我们对比了这些设置下模型的性能，发现简单地将标识符拆分插入到管道中会损害模型性能，而将标识符拆分和BPE算法相结合的混合策略可以在预测标识符方面优于原始开放词汇模型，召回率为3.68%，平均倒数排名为6.32%。实验结果还表明，该混合策略可以使语言模型的熵提高2.02%。
英文摘要：
Statistical language models on source code have successfully assisted software engineering tasks. However, developers can create or pick arbitrary identifiers when writing source code. Freely chosen identifiers lead to the notorious out-of-vocabulary (OOV) problem that negatively affects model performance. Recently, Karampatsis et al. showed that using the Byte Pair Encoding (BPE) algorithm to address the OOV problem can improve the language models' predictive performance on source code. However, a drawback of BPE is that it cannot split the identifiers in a way that preserves the meaningful semantics. Prior researchers also show that splitting compound identifiers into sub-words that reflect the semantics can benefit software development tools. These two facts motivate us to explore whether identifier splitting techniques can be utilized to augment the BPE algorithm and boost the performance of open-vocabulary language models considered in Karampatsis et al.'s work. This paper proposes to split identifiers in both constructing vocabulary and processing model inputs procedures, thus exploiting three different settings of applying identifier splitting to language models for the code completion task. We contrast models' performance under these settings and find that simply inserting identifier splitting into the pipeline hurts the model performance, while a hybrid strategy combining identifier splitting and the BPE algorithm can outperform the original open-vocabulary models on predicting identifiers by 3.68% of recall and 6.32% of Mean Reciprocal Rank. The results also show that the hybrid strategy can improve the entropy of language models by 2.02%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
