arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.13214cs.SEcs.LG
收藏

SCELMo:从语言模型嵌入Source Code

SCELMo: Source Code Embeddings from Language Models

The University of Edinburgh(爱丁堡大学);Google Research(谷歌研究院)

Rafael - Michael Karampatsis, Charles Sutton

摘要:

在计算机程序中连续嵌入令牌已被用于支持各种软件开发工具,包括可读性、代码搜索和程序修复。上下文嵌入在自然语言处理中很常见,但以前没有在软件工程中应用。我们引入了一套新的深度上下文化的词表示的计算机程序的基础上语言模型。我们使用Peters et al(2018)的ELMo(embeddings from语言模型)框架训练了一组嵌入。我们调查这些嵌入是否是有效的,当微调的下游任务的错误检测。我们表明,即使是在相对较小的程序语料库上训练的低维嵌入也可以提高最先进的机器学习系统的缺陷检测能力。

英文摘要:

Continuous embeddings of tokens in computer programs have been used to support a variety of software development tools, including readability, code search, and program repair. Contextual embeddings are common in natural language processing but have not been previously applied in software engineering. We introduce a new set of deep contextualized word representations for computer programs based on language models. We train a set of embeddings using the ELMo (embeddings from language models) framework of Peters et al (2018). We investigate whether these embeddings are effective when fine-tuned for the downstream task of bug detection. We show that even a low-dimensional embedding trained on a relatively small corpus of programs can improve a state-of-the-art machine learning system for bug detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑