arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.06703cs.SEcs.AI
收藏

DeepLSH:用于快速高效近似重复崩溃报告检测的深度局部敏感哈希学习

DeepLSH: Deep Locality-Sensitive Hash Learning for Fast and Efficient Near-Duplicate Crash Report Detection

INSA Lyon(里昂国立应用科学学院);Infologic R&D(Infologic研发中心);Riot Games(拳头游戏)

Youcef Remil, Anes Bendimerad, Romain Mathonat, Chedy Raissi, Mehdi Kaytoue

AI总结:

本文提出DeepLSH,一种基于孪生深度神经网络和原始损失函数的局部敏感哈希方法,用于快速高效检测近似重复崩溃报告,即使对Jaccard和余弦度量也能近似局部敏感属性,并通过实验验证。

摘要:

自动崩溃分桶是软件开发过程中高效分类错误报告的关键阶段,通常通过聚类技术将相似报告分组。然而,面对实时流式错误收集,系统需要快速回答:哪些错误与新的错误最相似?即高效地查找近似重复项。因此,自然考虑使用最近邻搜索来解决此问题,尤其是众所周知的局部敏感哈希(LSH),因其亚线性性能和相似性搜索准确性的理论保证而适用于大规模数据集。令人惊讶的是,LSH尚未在崩溃分桶文献中被考虑。确实,为最先进的崩溃分桶度量推导满足所谓局部敏感属性的哈希函数并非易事。因此,本文研究如何利用LSH完成此任务。为了能够考虑文献中使用的最相关度量,我们引入了DeepLSH,一种具有原始损失函数的孪生深度神经网络架构,即使对于存在精确LSH解的Jaccard和余弦度量,也能完美近似局部敏感属性。我们通过一系列实验支持这一主张,实验使用我们提供的一个原始数据集。

英文摘要:

Automatic crash bucketing is a crucial phase in the software development process for efficiently triaging bug reports. It generally consists in grouping similar reports through clustering techniques. However, with real-time streaming bug collection, systems are needed to quickly answer the question: What are the most similar bugs to a new one?, that is, efficiently find near-duplicates. It is thus natural to consider nearest neighbors search to tackle this problem and especially the well-known locality-sensitive hashing (LSH) to deal with large datasets due to its sublinear performance and theoretical guarantees on the similarity search accuracy. Surprisingly, LSH has not been considered in the crash bucketing literature. It is indeed not trivial to derive hash functions that satisfy the so-called locality-sensitive property for the most advanced crash bucketing metrics. Consequently, we study in this paper how to leverage LSH for this task. To be able to consider the most relevant metrics used in the literature, we introduce DeepLSH, a Siamese DNN architecture with an original loss function, that perfectly approximates the locality-sensitivity property even for Jaccard and Cosine metrics for which exact LSH solutions exist. We support this claim with a series of experiments on an original dataset, which we make available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑