数据集在机器学习(ML)模型的训练和评估中发挥着核心作用。但它们也是许多不良模型行为(例如有偏预测)的根本原因。为了改变这一状况,机器学习社区正在推动一种以数据为中心的文化转变,使数据问题得到应有的重视,并开始讨论和建立围绕数据集收集与处理的更标准化的实践。迄今为止,这些提议大多是使用自然语言描述的高层指导方针,因此难以形式化并应用于具体的数据集。鉴于此,并受这些提议的启发,我们定义了一种新的领域特定语言(DSL),用于从数据集的结构、数据来源和社会关切等方面精确描述机器学习数据集。我们相信,这种DSL将有助于任何机器学习项目利用并受益于机器学习领域中以数据为中心的转变(例如,为新项目选择最合适的数据集,或更好地复现其他机器学习结果)。该DSL已实现为一个Visual Studio Code插件,并以开源许可证发布。
英文摘要:
Datasets play a central role in the training and evaluation of machine learning (ML) models. But they are also the root cause of many undesired model behaviors, such as biased predictions. To overcome this situation, the ML community is proposing a data-centric cultural shift where data issues are given the attention they deserve, and more standard practices around the gathering and processing of datasets start to be discussed and established.
So far, these proposals are mostly high-level guidelines described in natural language and, as such, they are difficult to formalize and apply to particular datasets. In this sense, and inspired by these proposals, we define a new domain-specific language (DSL) to precisely describe machine learning datasets in terms of their structure, data provenance, and social concerns. We believe this DSL will facilitate any ML initiative to leverage and benefit from this data-centric shift in ML (e.g., selecting the most appropriate dataset for a new project or better replicating other ML results). The DSL is implemented as a Visual Studio Code plugin, and it has been published under an open source license.
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
University of Maryland(马里兰大学);Massachussetts Institute of Technology(麻省理工学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加利福尼亚大学伯克利分校)
Explaining the Performance of Multi-label Classification Methods with Data Set Properties
Jožef Stefan Institute(约瑟夫·斯特凡研究所);Jožef Stefan IPSchool(约瑟夫·斯特凡知识产权学院);Faculty of Mathematics and Physics, University of Ljubljana(卢布尔雅那大学数学与物理学院);TU Berlin(柏林工业大学);Bias Variance Labs(偏差方差实验室)
Topological Data Analysis combined with Machine Learning for Predicting Permeability of Porous Media
Department of Mathematical Sciences, New Jersey Institute of Technology(新泽西理工学院数学科学系);Department of Physics, New Jersey Institute of Technology(新泽西理工学院物理系)
Machine Learning Information Fusion in Earth Observation: A Comprehensive Review of Methods, Applications and Data Sources
Universidad de Alcalá(阿尔卡拉大学);Helmholtz-Zentrum Dresden-Rossendorf(亥姆霍兹德累斯顿罗森多夫研究中心);Helmholtz Institute Freiberg for Resource Technology(亥姆霍兹弗赖贝格资源技术研究所);Universitat de València(瓦伦西亚大学);Technical University of Munich(慕尼黑工业大学);University of Natural Resources and Life Science (BOKU)(自然资源与生命科学大学(BOKU));Obuda University(奥布达大学);Oxford Brookes University(牛津布鲁克斯大学)