机器学习系统质量评估指南
Towards Guidelines for Assessing Qualities of Machine Learning Systems
摘要：
如今，包含基于机器学习（ML）方法的组件的系统正变得越来越普遍。为了确保软件系统的预期行为，有一些标准定义了系统及其组件的必要质量方面（例如ISO/IEC 25010）。由于ML的不同性质，我们必须调整质量方面或添加其他方面（例如可信度），并非常精确地确定哪个方面与哪个感兴趣的对象真正相关（例如训练数据的完整性），以及如何客观地评估对质量要求的遵守情况。在这篇文章中，我们提出了一个质量模型的构建（即，评估对象，质量方面和指标）的ML系统基于工业用例。这种质量模型使从业者能够客观地指定和评估这类ML系统的质量要求。在未来，我们希望了解术语质量在不同类型的ML系统之间的差异，并提出用于指定和评估ML系统质量的通用指南。
英文摘要：
Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary quality aspects of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to adjust quality aspects or add additional ones (such as trustworthiness) and be very precise about which aspect is really relevant for which object of interest (such as completeness of training data), and how to objectively assess adherence to quality requirements. In this article, we present the construction of a quality model (i.e., evaluation objects, quality aspects, and metrics) for an ML system based on an industrial use case. This quality model enables practitioners to specify and assess quality requirements for such kinds of ML systems objectively. In the future, we want to learn how the term quality differs between different types of ML systems and come up with general guidelines for specifying and assessing qualities of ML systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
