提升深度学习缺陷可复现性：一项实证研究
Towards Enhancing the Reproducibility of Deep Learning Bugs: An Empirical Study
AI总结：
本文通过构建668个深度学习缺陷数据集并复现其中165个缺陷，识别出十种编辑操作和五类有用信息，使开发者缺陷复现率提升22.92%、复现时间缩短24.35%，为提升深度学习缺陷可复现性提供了实证依据。
摘要：
背景：深度学习在各个领域取得了显著进展。然而，与任何软件系统一样，深度学习系统也存在缺陷，其中一些缺陷可能产生严重影响，自动驾驶汽车发生的碰撞事故即为明证。尽管深度学习技术取得了长足进步，但很少有研究关注深度学习缺陷的复现，而缺陷复现是解决缺陷的关键步骤。现有文献表明，只有3%的深度学习缺陷是可复现的，这凸显了进一步研究的必要性。\n目标：本文考察深度学习缺陷的可复现性。我们识别出能够提高深度学习缺陷可复现性的编辑操作和有用信息。\n方法：首先，我们从Stack Overflow和GitHub中构建了一个包含668个深度学习缺陷的数据集，这些缺陷涉及三个框架和22种架构。其次，在668个缺陷中，我们采用分层抽样方法选取了165个缺陷，并尝试确定其可复现性。在复现这些缺陷的过程中，我们识别出用于缺陷复现的编辑操作和有用信息。第三，我们使用Apriori算法识别复现特定类型缺陷所需的有用信息和编辑操作。最后，我们开展了一项涉及22名开发者的用户研究，以评估我们的发现在真实场景中的有效性。\n结果：在尝试复现的165个缺陷中，我们成功复现了148个。我们识别出十种编辑操作和五种有用的组件信息类型，它们能够帮助我们复现深度学习缺陷。借助我们的发现，开发者能够多复现22.92%的缺陷，并将复现时间缩短24.35%。\n结论：我们的研究解决了深度学习缺陷可复现性这一关键问题。从业者和研究人员可以利用我们的发现来提高深度学习缺陷的可复现性。
英文摘要：
Context: Deep learning has achieved remarkable progress in various domains. However, like any software system, deep learning systems contain bugs, some of which can have severe impacts, as evidenced by crashes involving autonomous vehicles. Despite substantial advancements in deep learning techniques, little research has focused on reproducing deep learning bugs, which is an essential step for their resolution. Existing literature suggests that only 3% of deep learning bugs are reproducible, underscoring the need for further research. Objective: This paper examines the reproducibility of deep learning bugs. We identify edit actions and useful information that could improve the reproducibility of deep learning bugs. Method: First, we construct a dataset of 668 deep-learning bugs from Stack Overflow and GitHub across three frameworks and 22 architectures. Second, out of the 668 bugs, we select 165 bugs using stratified sampling and attempt to determine their reproducibility. While reproducing these bugs, we identify edit actions and useful information for their reproduction. Third, we used the Apriori algorithm to identify useful information and edit actions required to reproduce specific types of bugs. Finally, we conducted a user study involving 22 developers to assess the effectiveness of our findings in real-life settings. Results: We successfully reproduced 148 out of 165 bugs attempted. We identified ten edit actions and five useful types of component information that can help us reproduce the deep learning bugs. With the help of our findings, the developers were able to reproduce 22.92% more bugs and reduce their reproduction time by 24.35%. Conclusions: Our research addresses the critical issue of deep learning bug reproducibility. Practitioners and researchers can leverage our findings to improve deep learning bug reproducibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
