如何改进AI工具(通过添加SE知识):使用TimeLIME缺陷减少工具的实验
How to Improve AI Tools (by Adding in SE Knowledge): Experiments with the TimeLIME Defect Reduction Tool
摘要:
人工智能算法在SE研究和实践中的使用频率越来越高。这些算法通常使用来自SE域外部的数据进行委托和认证。我们是否可以假设这样的算法可以“现成”使用(即无需修改)?换句话说,SE问题的特殊特征是否表明使用AI工具的不同和更好的方法?为了回答这些问题,本文报告了TimeLIME的实验,这是KDD'16中LIME解释算法的一个变体。LIME可以提供关于如何更改静态代码属性的建议,以减少下一个软件版本中的缺陷数量。该版本的LIME使用内部加权工具来决定在这些建议中包括/排除哪些属性。TimeLIME使用以下SE知识改进了该加权方案:软件发布;对软件的不合理更改是在以前的版本中从未更改过的内容;因此最好使用合理的更改,即在以前的版本中有一些先例的更改。通过将建议限制在经常更改的属性上,TimeLIME可以(a)对导致缺陷的原因做出更好的解释,(b)对如何修复错误代码提出更好的建议。除了这些关于缺陷减少和TimeLIME的具体结果之外,本文更普遍的观点是,我们的社区应该在使用现成的AI工具时更加小心,而不是首先应用SE知识。正如这里所示,应用这些知识可能不是一件复杂的事情。此外,一旦应用了SE知识,就可以产生更好的系统。
英文摘要:
AI algorithms are being used with increased frequency in SE research and practice. Such algorithms are usually commissioned and certified using data from outside the SE domain. Can we assume that such algorithms can be used ''off-the-shelf'' (i.e. with no modifications)? To say that another way, are there special features of SE problems that suggest a different and better way to use AI tools? To answer these questions, this paper reports experiments with TimeLIME, a variant of the LIME explanation algorithm from KDD'16. LIME can offer recommendations on how to change static code attributes in order to reduce the number of defects in the next software release. That version of LIME used an internal weighting tool to decide what attributes to include/exclude in those recommendations. TimeLIME improves on that weighting scheme using the following SE knowledge: software comes in releases; an implausible change to software is something that has never been changed in prior releases; so it is better to use plausible changes, i.e. changes with some precedent in the prior releases. By restricting recommendations to just the frequently changed attributes, TimeLIME can produce (a)~dramatically better explanations of what causes defects and (b)~much better recommendations on how to fix buggy code. Apart from these specific results about defect reduction and TimeLIME, the more general point of this paper is that our community should be more careful about using off-the-shelf AI tools, without first applying SE knowledge. As shown here, it may not be a complex matter to apply that knowledge. Further, once that SE knowledge is applied, this can result in dramatically better systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。