ToxiSpanSE：代码评审评论中的可解释毒性检测
ToxiSpanSE: An Explainable Toxicity Detection in Code Review Comments
AI总结：
本文提出ToxiSpanSE，首个面向软件工程领域的可解释毒性片段检测工具，基于19,651条代码评审评论训练，微调RoBERTa取得F1值0.88的最佳效果，可自动高亮有毒内容以辅助人工审核。
摘要：
背景：开源平台中存在的有害对话会损害软件开发者之间的关系，并可能对软件产品质量产生负面影响。为了帮助缓解这一问题，已有一些初步工作致力于在软件工程（SE）领域检测有毒评论。目标：由于将整段文本自动分类为有毒或无毒并不能帮助人工审核者理解毒性的具体原因，我们致力于为SE领域开发一种可解释的毒性检测器。方法：我们的可解释毒性检测器能够从SE文本中检测出具体的有毒内容片段，并通过自动高亮这些片段来帮助人工审核者。该有毒片段检测模型ToxiSpanSE使用19,651条带有有毒片段标签的代码评审（CR）评论进行训练。我们的标注人员在3,757条有毒CR样本中标注了有毒片段。我们探索了多种类型的模型，包括一种基于词典的方法和五种不同的基于Transformer的编码器。结果：在对所有模型进行广泛评估后，我们发现微调后的RoBERTa模型取得了最佳成绩，有毒类别token的F1值为0.88，精确率为0.87，召回率为0.93，从而为SE领域提供了一个可解释的毒性分类器。结论：由于ToxiSpanSE是首个在SE领域检测有毒片段的工具，该工具将为在SE社区中对抗毒性铺平道路。
英文摘要：
Background: The existence of toxic conversations in open-source platforms can degrade relationships among software developers and may negatively impact software product quality. To help mitigate this, some initial work has been done to detect toxic comments in the Software Engineering (SE) domain. Aims: Since automatically classifying an entire text as toxic or non-toxic does not help human moderators to understand the specific reason(s) for toxicity, we worked to develop an explainable toxicity detector for the SE domain. Method: Our explainable toxicity detector can detect specific spans of toxic content from SE texts, which can help human moderators by automatically highlighting those spans. This toxic span detection model, ToxiSpanSE, is trained with the 19,651 code review (CR) comments with labeled toxic spans. Our annotators labeled the toxic spans within 3,757 toxic CR samples. We explored several types of models, including one lexicon-based approach and five different transformer-based encoders. Results: After an extensive evaluation of all models, we found that our fine-tuned RoBERTa model achieved the best score with 0.88 $F1$, 0.87 precision, and 0.93 recall for toxic class tokens, providing an explainable toxicity classifier for the SE domain. Conclusion: Since ToxiSpanSE is the first tool to detect toxic spans in the SE domain, this tool will pave a path to combat toxicity in the SE community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
