揭秘黑暗艺术：理解现实世界的机器学习模型开发
Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development
摘要：
众所周知，开发机器学习（ML）工作流程的过程是一门黑暗艺术;即使是专家也很难找到一个最佳的工作流程，从而获得高精度的模型。用户目前依赖于经验试错法来获得自己的一套经过实战检验的指导方针，以告知他们的建模决策。在这项研究中，我们的目标是通过了解人们如何在实践中使用ML工作流程来揭开这种黑暗艺术的神秘面纱。我们在OpenML上分析了超过47.5万个用户生成的工作流，OpenML是一个用于跟踪和共享ML工作流的开源平台。我们发现，用户在迭代工作流时通常采用手动、自动或混合方法。我们观察到手动方法比自动方法导致更少的浪费迭代。然而，自动化方法通常涉及更多的预处理和探索的超参数选项，从而提高整体性能-这表明了人类在环机器学习系统的潜在好处，该系统适当地推荐了两种策略的巧妙组合。
英文摘要：
It is well-known that the process of developing machine learning (ML) workflows is a dark-art; even experts struggle to find an optimal workflow leading to a high accuracy model. Users currently rely on empirical trial-and-error to obtain their own set of battle-tested guidelines to inform their modeling decisions. In this study, we aim to demystify this dark art by understanding how people iterate on ML workflows in practice. We analyze over 475k user-generated workflows on OpenML, an open-source platform for tracking and sharing ML workflows. We find that users often adopt a manual, automated, or mixed approach when iterating on their workflows. We observe that manual approaches result in fewer wasted iterations compared to automated approaches. Yet, automated approaches often involve more preprocessing and hyperparameter options explored, resulting in higher performance overall--suggesting potential benefits for a human-in-the-loop ML system that appropriately recommends a clever combination of the two strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
