结合检索与分类:在重复缺陷报告检测中平衡效率与准确率
Combining Retrieval and Classification: Balancing Efficiency and Accuracy in Duplicate Bug Report Detection
AI总结:
针对重复缺陷报告检测中准确率与时间效率难以兼顾的问题,提出一种结合检索模型(sentence BERT)与分类模型(RoBERTa)的混合transformer系统,在五个数据集上实现了准确率与效率的有效权衡。
摘要:
在重复缺陷报告检测(DBRD)领域,传统方法主要侧重于静态分析缺陷数据库,往往忽视了模型的运行时间。在此背景下,复杂模型尽管具有高准确率的潜力,但可能非常耗时,而更高效的模型则可能在准确率上妥协。为了解决这个问题,我们提出了一个基于transformer的系统,旨在在时间效率和准确率性能之间取得平衡。现有方法主要将其作为检索或分类任务来处理。然而,我们的混合方法利用了两种模型的优势。通过利用检索模型,我们可以执行初始排序以减少候选集,而分类模型则允许进行更精确和准确的分类。在我们对用于检索和分类任务的常用模型评估中,sentence BERT和RoBERTa分别在检索和分类中优于其他基线模型。为了提供性能和效率的全面评估,我们在五个公共数据集上进行了严格的实验。结果表明,我们的系统保持了与分类模型相当的准确率,在时间效率上显著优于它,且在时间上仅略逊于检索模型,从而在准确率和效率之间实现了有效的权衡。
英文摘要:
In the realm of Duplicate Bug Report Detection (DBRD), conventional methods primarily focus on statically analyzing bug databases, often disregarding the running time of the model. In this context, complex models, despite their high accuracy potential, can be time-consuming, while more efficient models may compromise on accuracy. To address this issue, we propose a transformer-based system designed to strike a balance between time efficiency and accuracy performance. The existing methods primarily address it as either a retrieval or classification task. However, our hybrid approach leverages the strengths of both models. By utilizing the retrieval model, we can perform initial sorting to reduce the candidate set, while the classification model allows for more precise and accurate classification. In our assessment of commonly used models for retrieval and classification tasks, sentence BERT and RoBERTa outperform other baseline models in retrieval and classification, respectively. To provide a comprehensive evaluation of performance and efficiency, we conduct rigorous experimentation on five public datasets. The results reveal that our system maintains accuracy comparable to a classification model, significantly outperforming it in time efficiency and only slightly behind a retrieval model in time, thereby achieving an effective trade-off between accuracy and efficiency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。