CrossCheck：快速、可再现和可解释的模型评估
CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation
摘要：
除了综合性能指标（例如F1分数）之外的评估对于在机器学习模型中建立适当的信任水平和识别未来的模型改进至关重要。在本文中，我们展示了CrossCheck，一个交互式的可视化工具，用于快速的跨模型比较和可重复的误差分析。我们描述的工具，并讨论设计和实施细节。然后，我们提出了三个用例（命名实体识别，阅读理解和点击诱饵检测），显示使用模型评估工具的好处。CrossCheck允许数据科学家做出明智的决策，在多个模型之间进行选择，识别模型何时正确以及哪些示例正确，调查模型是否犯了与人类相同的错误，评估模型的可推广性并突出模型的局限性，优势和劣势。此外，CrossCheck是作为一个小部件实现的，它允许快速方便地集成到数据科学家的模型开发工作流中。
英文摘要：
Evaluation beyond aggregate performance metrics, e.g. F1-score, is crucial to both establish an appropriate level of trust in machine learning models and identify future model improvements. In this paper we demonstrate CrossCheck, an interactive visualization tool for rapid crossmodel comparison and reproducible error analysis. We describe the tool and discuss design and implementation details. We then present three use cases (named entity recognition, reading comprehension, and clickbait detection) that show the benefits of using the tool for model evaluation. CrossCheck allows data scientists to make informed decisions to choose between multiple models, identify when the models are correct and for which examples, investigate whether the models are making the same mistakes as humans, evaluate models' generalizability and highlight models' limitations, strengths and weaknesses. Furthermore, CrossCheck is implemented as a Jupyter widget, which allows rapid and convenient integration into data scientists' model development workflows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
