Machine learning (ML) technologies have become substantial in practically all aspects of our society, and data quality (DQ) is critical for the performance, fairness, robustness, safety, and scalability of ML models. With the large and complex data in data-centric AI, traditional methods like exploratory data analysis (EDA) and cross-validation (CV) face challenges, highlighting the importance of mastering DQ tools. In this survey, we review 17 DQ evaluation and improvement tools in the last 5 years. By introducing the DQ dimensions, metrics, and main functions embedded in these tools, we compare their strengths and limitations and propose a roadmap for developing open-source DQ tools for ML. Based on the discussions on the challenges and emerging trends, we further highlight the potential applications of large language models (LLMs) and generative AI in DQ evaluation and improvement for ML. We believe this comprehensive survey can enhance understanding of DQ in ML and could drive progress in data-centric AI. A complete list of the literature investigated in this survey is available on GitHub at: https://github.com/haihua0913/awesome-dq4ml.
Tool Learning with Large Language Models: A Survey
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Baidu Inc.(百度公司);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
Tool-IQA: Augmenting Image Quality Assessment with Simple Tools
National University of Singapore(新加坡国立大学);OPPO Research Institute(OPPO研究院);Nanyang Technical University(南洋理工大学);Duke University(杜克大学);City University of Hong Kong(香港城市大学);The Hong Kong Polytechnic University(香港理工大学)
Multimodal Fusion on Low-quality Data: A Comprehensive Survey
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);College of Computer Science, Sichuan University(四川大学计算机学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院)
Data Quality Toolkit: Automatic assessment of data quality and remediation for machine learning datasets
IBM Research(IBM研究院)
训练数据的质量对机器学习任务的效率、准确性和复杂性有着巨大的影响。有各种工具和技术可用于评估与一般清理和分析检查有关的数据质量。然而,这些技术不适用于检测数据问题的背景下,机器学习任务,如嘈杂的标签,存在重叠的类等,我们试图重新审视数据质量问题的背景下,建立一个机器学习管道,并建立一个工具,可以检测,解释和补救问题的数据,并系统和自动捕获所有的变化应用到数据。我们引入了用于机器学习的数据质量工具包,作为一些关键质量指标和相关补救技术的库,以分析和增强机器学习项目的结构化训练数据集的准备程度。该工具包可以减少数据准备管道的周转时间,并简化数据质量评估过程。我们的工具包通过IBM API Hub [1]平台公开提供,任何开发人员都可以使用IBM的Data Quality for AI API [2]评估数据质量。详细的教程也可以在IBM Learning Path [3]上找到。
Survey2Survey: A deep learning generative model approach for cross-survey image mapping
University of Illinois(伊利诺伊大学);Indian Institute of Technology Roorkee(印度理工学院鲁尔基分校);National Center for Supercomputing Applications(国家超级计算应用中心);Center for Astrophysical Surveys(天体物理巡天中心)
Enhancing LLM Tool Use with High-quality Instruction Data from Knowledge Graph
Ant Group, China(蚂蚁集团);School of Computer Science and Technology, Xi’an Jiaotong University, China(西安交通大学计算机科学与技术学院);School of Distance Education, Xi’an Jiaotong University, China(西安交通大学继续教育学院)