深度学习框架中非功能性错误的多方面分层报告识别
Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks
摘要:
非功能性错误(例如,深度学习(DL)框架中的性能或准确性相关的错误)可能会导致一些最具破坏性的后果。在GitHub等存储库上报告这些错误是修复它们的标准途径。然而,鉴于DL框架的新GitHub报告越来越多,开发人员很难区分那些揭示非功能性错误的报告,并将其分配给正确的贡献者进行及时调查。在本文中,我们提出了MHNurf -一个端到端的工具,用于自动识别DL框架中的非功能性错误相关报告。MHNurf的核心是一个多方面的层次注意力网络(MHAN),它解决了三个未解决的挑战:(1)学习语义知识,但这样做是通过(2)考虑层次结构(例如,句子/陈述中的词/记号)并关注重要部分(即,单词、令牌、句子和语句),而(3)从不同类型的特征中独立地提取信息,即,内容、注释、代码、命令和标签。为了评估MHNurf,我们利用来自五个DL框架的3,721个GitHub报告进行实验。结果表明,MHNurf在内容、注释和代码的组合下工作得最好,这大大优于仅使用内容的经典HAN。MHNurf还产生了比其他九个具有强统计显著性的最先进分类器更准确的结果,即,AUC改善高达71%,在四个框架上具有最佳Scott-Knott排名,而在其余一个框架上则排名第二。为了方便复制和促进未来的研究,我们已经将我们的数据集,代码和详细的补充结果公开在:https://github.com/ideas-labo/APSEC2022-MHNurf。
英文摘要:
Non-functional bugs (e.g., performance- or accuracy-related bugs) in Deep Learning (DL) frameworks can lead to some of the most devastating consequences. Reporting those bugs on a repository such as GitHub is a standard route to fix them. Yet, given the growing number of new GitHub reports for DL frameworks, it is intrinsically difficult for developers to distinguish those that reveal non-functional bugs among the others, and assign them to the right contributor for investigation in a timely manner. In this paper, we propose MHNurf - an end-to-end tool for automatically identifying non-functional bug related reports in DL frameworks. The core of MHNurf is a Multifaceted Hierarchical Attention Network (MHAN) that tackles three unaddressed challenges: (1) learning the semantic knowledge, but doing so by (2) considering the hierarchy (e.g., words/tokens in sentences/statements) and focusing on the important parts (i.e., words, tokens, sentences, and statements) of a GitHub report, while (3) independently extracting information from different types of features, i.e., content, comment, code, command, and label. To evaluate MHNurf, we leverage 3,721 GitHub reports from five DL frameworks for conducting experiments. The results show that MHNurf works the best with a combination of content, comment, and code, which considerably outperforms the classic HAN where only the content is used. MHNurf also produces significantly more accurate results than nine other state-of-the-art classifiers with strong statistical significance, i.e., up to 71% AUC improvement and has the best Scott-Knott rank on four frameworks while 2nd on the remaining one. To facilitate reproduction and promote future research, we have made our dataset, code, and detailed supplementary results publicly available at: https://github.com/ideas-labo/APSEC2022-MHNurf.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。