依赖地狱是大型软件项目开发中众所周知的痛点,机器学习(ML)代码库也不能幸免。事实上,ML应用程序还遭受着另一种形式的依赖地狱,即“数据源依赖地狱”。这个术语指的是数据及其独特的怪癖所扮演的核心角色,这些怪癖通常会导致ML模型的意外失败,而这些失败无法通过代码更改来解释。在本文中,我们提出了一个自动依赖映射框架,允许MLOps工程师在快节奏的工程环境中监视其模型的整个依赖映射,从而提前减轻任何数据源更改的后果(例如,重新训练模型、忽略数据、设置默认数据等)。我们的系统是基于一个统一的和通用的方法,采用静态分析技术,从数据源可以可靠地确定任何类型的依赖范围广泛的源语言和文物。依赖映射框架作为REST Web API公开,其中唯一的输入是托管代码库的Git存储库的路径。目前由微软的MLOps工程师使用,我们预计这种依赖关系映射API将在未来被MLOps工程师更广泛地采用。
英文摘要:
Dependency hell is a well-known pain point in the development of large software projects and machine learning (ML) code bases are not immune from it. In fact, ML applications suffer from an additional form, namely, "data source dependency hell". This term refers to the central role played by data and its unique quirks that often lead to unexpected failures of ML models which cannot be explained by code changes. In this paper, we present an automated dependency mapping framework that allows MLOps engineers to monitor the whole dependency map of their models in a fast paced engineering environment and thus mitigate ahead of time the consequences of any data source changes (e.g., re-train model, ignore data, set default data etc.). Our system is based on a unified and generic approach, employing techniques from static analysis, from which data sources can be identified reliably for any type of dependency on a wide range of source languages and artefacts. The dependency mapping framework is exposed as a REST web API where the only input is the path to the Git repository hosting the code base. Currently used by MLOps engineers at Microsoft, we expect such dependency map APIs to be adopted more widely by MLOps engineers in the future.
Exploring the SECURITY.md in the Dependency Chain: Preliminary Analysis of the PyPI Ecosystem
Faculty of Information and Communication Technology, Mahidol University(玛希隆大学信息通信技术学院);Graduate School of Science and Technology, Nara Institute of Science and Technology (NAIST)(奈良先端科学技术大学院大学理工学研究科);Graduate School of Information Science and Technology, The University of Osaka(大阪大学情报理工学研究科)
Data Science Methodologies: Current Challenges and Future Approaches
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)(Vicomtech基金会,巴斯克研究与技术联盟);TECNUN School of Engineering, University of Navarra(纳瓦拉大学TECNUN工程学院);Institute of Data Science and Artificial Intelligence, University of Navarra(纳瓦拉大学数据科学与人工智能研究所)
A survey study of success factors in data science projects
Vicomtech Foundation Basque Research and Technology Alliance(Vicomtech基金会巴斯克研究与技术联盟);University of Navarra TECNUN School of Engineering(纳瓦拉大学TECNUN工程学院)
An Empirical Study of Kotlin-Java Cross-Dependency Issues and Their Detection
Nanjing University of Science and Technology, China(南京理工大学);Huawei Technologies Co., Ltd.(华为技术有限公司);School of Computer Science, Wuhan University, China(武汉大学计算机学院)
DependEval: Benchmarking LLMs for Repository Dependency Understanding
Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);Beihang University(北京航空航天大学);ShanghaiTech University(上海科技大学);The University of Hong Kong(香港大学)