FRUGAL:解锁SSL以实现软件分析
FRUGAL: Unlocking SSL for Software Analytics
摘要:
标准的软件分析通常涉及大量带有标签的数据,以便以可接受的性能委托模型。然而,之前的工作表明,此类要求可能成本高昂,需要数周时间来标记数千个提交,并且在穿越新的研究问题和领域时并不总是可用。无监督学习是一个很有前途的方向,学习隐藏在未标记数据中的模式,这只在缺陷预测中得到了广泛的研究。然而,无监督学习本身可能是无效的,并且尚未在其他领域(例如,静态分析和发行关闭时间)。出于这种文献空白和技术限制的动机,我们提出了FRUGAL,一种基于简单优化方案的调优半监督方法,该方法不需要复杂的(例如,深度学习器)和昂贵的(例如,100%手动标记数据)方法。FRUGAL优化了无监督学习器的配置(通过简单的网格搜索),同时验证了我们在预测前仅标记2.5%数据的设计决策。本文的实验结果表明,FRUGAL的性能优于最先进的可采用的静态代码警告识别器和问题关闭时间预测器,同时将标签成本降低了40倍(从100%降低到2.5%)。因此,我们断言,FRUGAL可以节省相当大的努力,在数据标记,特别是在验证以前的工作或研究新的问题。基于这项工作,我们建议复杂和昂贵的方法的支持者应该始终将这些方法与更简单,更便宜的替代品进行比较。例如,像FRUGAL这样的半监督学习器可以作为最先进的软件分析的基线。
英文摘要:
Standard software analytics often involves having a large amount of data with labels in order to commission models with acceptable performance. However, prior work has shown that such requirements can be expensive, taking several weeks to label thousands of commits, and not always available when traversing new research problems and domains. Unsupervised Learning is a promising direction to learn hidden patterns within unlabelled data, which has only been extensively studied in defect prediction. Nevertheless, unsupervised learning can be ineffective by itself and has not been explored in other domains (e.g., static analysis and issue close time). Motivated by this literature gap and technical limitations, we present FRUGAL, a tuned semi-supervised method that builds on a simple optimization scheme that does not require sophisticated (e.g., deep learners) and expensive (e.g., 100% manually labelled data) methods. FRUGAL optimizes the unsupervised learner's configurations (via a simple grid search) while validating our design decision of labelling just 2.5% of the data before prediction. As shown by the experiments of this paper FRUGAL outperforms the state-of-the-art adoptable static code warning recognizer and issue closed time predictor, while reducing the cost of labelling by a factor of 40 (from 100% to 2.5%). Hence we assert that FRUGAL can save considerable effort in data labelling especially in validating prior work or researching new problems. Based on this work, we suggest that proponents of complex and expensive methods should always baseline such methods against simpler and cheaper alternatives. For instance, a semi-supervised learner like FRUGAL can serve as a baseline to the state-of-the-art software analytics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。