发现、解释、改进：一种用于自然语言处理的自动切片检测框架
Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing
AI总结：
本文提出NLP分类任务基准DEIM与切片检测模型Edisa，自动发现并解释表现不佳的数据点群体，在不调整模型参数的情况下直接提升模型性能。
摘要：
预训练自然语言处理（NLP）模型已取得很高的整体性能，但它们仍会产生系统性错误。针对自动识别表现不佳的数据点群体的切片检测模型（SDM）研究，无需人工错误分析，在计算机视觉领域已引起越来越多的关注，这既有助于理解模型行为，也为未来模型的训练和设计提供见解。然而，在NLP任务上，关于SDM及其有效性的定量评估研究却很少。本文通过为分类NLP任务提出一个名为“发现、解释、改进（DEIM）”的基准，以及一种新的SDM模型Edisa来填补这一空白。Edisa能够发现连贯且表现不佳的数据点群体；DEIM随后将它们统一在人类可理解的概念下，并提供全面的评估任务及相应的定量指标。DEIM中的评估表明，Edisa能够准确地选择容易出错的数据点，并提取具有信息量的语义特征来总结错误模式。直接检测困难数据点即可提升模型性能，而无需调整任何原始模型参数，这表明所发现的切片对用户具有可操作性。
英文摘要：
Pretrained natural language processing (NLP) models have achieved high overall performance, but they still make systematic errors. Instead of manual error analysis, research on slice detection models (SDM), which automatically identify underperforming groups of datapoints, has caught escalated attention in Computer Vision for both understanding model behaviors and providing insights for future model training and designing. However, little research on SDM and quantitative evaluation of their effectiveness have been conducted on NLP tasks. Our paper fills the gap by proposing a benchmark named "Discover, Explain, Improve (DEIM)" for classification NLP tasks along with a new SDM Edisa. Edisa discovers coherent and underperforming groups of datapoints; DEIM then unites them under human-understandable concepts and provides comprehensive evaluation tasks and corresponding quantitative metrics. The evaluation in DEIM shows that Edisa can accurately select error-prone datapoints with informative semantic features that summarize error patterns. Detecting difficult datapoints directly boosts model performance without tuning any original model parameters, showing that discovered slices are actionable for users.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
