机器学习支持系统中失配的表征与检测
Characterizing and Detecting Mismatch in Machine-Learning-Enabled Systems
摘要：
机器学习（ML）框架和工具的可用性越来越高，以及它们改善数据驱动决策问题解决方案的承诺，导致了在软件系统中使用ML技术的普及。然而，支持ML的系统的端到端开发及其无缝部署和操作仍然是一个挑战。原因之一是支持ML的系统的开发和部署涉及三个不同的工作流、视角和角色，包括数据科学、软件工程和操作。这三个不同的视角，当由于不正确的假设而不一致时，会导致ML不匹配，从而导致系统失败。我们进行了一次访谈和调查研究，收集并验证了在支持ML的系统的端到端开发中发生的常见不匹配类型。我们的分析表明，每个角色如何优先考虑相关不匹配的重要性各不相同，可能导致这些不匹配的假设。此外，我们确定的不匹配类别可以指定为机器可读的描述符，有助于改进支持ML的系统开发。在本文中，我们报告了我们的研究结果及其对改进端到端ML启用系统开发的影响。
英文摘要：
Increasing availability of machine learning (ML) frameworks and tools, as well as their promise to improve solutions to data-driven decision problems, has resulted in popularity of using ML techniques in software systems. However, end-to-end development of ML-enabled systems, as well as their seamless deployment and operations, remain a challenge. One reason is that development and deployment of ML-enabled systems involves three distinct workflows, perspectives, and roles, which include data science, software engineering, and operations. These three distinct perspectives, when misaligned due to incorrect assumptions, cause ML mismatches which can result in failed systems. We conducted an interview and survey study where we collected and validated common types of mismatches that occur in end-to-end development of ML-enabled systems. Our analysis shows that how each role prioritizes the importance of relevant mismatches varies, potentially contributing to these mismatched assumptions. In addition, the mismatch categories we identified can be specified as machine readable descriptors contributing to improved ML-enabled system development. In this paper, we report our findings and their implications for improving end-to-end ML-enabled system development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
