关于使用基于大语言模型的预测模型解释代码变更风险的初步研究
A Preliminary Study on Explaining Risk of Code Changes using LLM-Based Prediction Models
AI总结:
研究利用基于大语言模型的Diff风险评分模型注意力权重突出代码变更中风险集中部分,聚合为可解释代码单元供开发者代码审查参考,评估显示能有效覆盖风险变更行且可扩展。
摘要:
机器学习和人工智能模型的预测若不伴有易懂解释往往遭怀疑。在即时缺陷预测中,突出软件变更中风险集中部分尚未充分研究。本文利用基于大语言模型的Diff风险评分(DRS)模型注意力权重突出预测风险时模型关注的变更部分,聚合为可解释代码单元供开发者代码审查参考并评估。
英文摘要:
Predictions by machine learning (ML) and artificial intelligence (AI) models are often received skeptically unless they are paired with intelligible explanations. In the context of just-in-time defect prediction, highlighting small portions of a software change (diff) -- beyond rule-based lints -- where risk may be concentrated has not yet been extensively investigated. In this work, we leverage attention weights from an LLM-based Diff Risk Score (DRS) model to highlight parts of a diff that the model focuses on when predicting risk. We aggregate token-level attention into interpretable code units (lines, hunks, and files), and present the top-K units to developers as a lightweight form of guidance during code review. We evaluate our approach using expert-labeled changes that have caused real outages. Results show that the highlighted snippets cover expert-labeled outage-causing change lines 53.85% of the time when highlighting the top-2 hunks, while requiring developers to review 26.28% of the changed lines on average. Because attention is produced during standard model inference, the approach is scalable for large development workflows and can be surfaced in the code review UI with low additional latency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。