AI研究并非魔法,它必须具备可复现性与负责任性:从博士生视角看AI领域的挑战
AI Research is not Magic, it has to be Reproducible and Responsible: Challenges in the AI field from the Perspective of its PhD Students
AI总结:
该研究通过调查欧洲28名AI博士生,梳理出AI资源、实验复现、可信度与跨学科性三大挑战,并从社会和技术层面提出推行可复现倡议、完善工具与基础设施等建议,以推动负责任、可复现的AI研究。
摘要:
为揭示欧洲AI学生在研究过程中面临的挑战,我们对来自13个欧洲国家的28名AI博士生开展了调查。结果凸显了三大核心领域的挑战:(1)数据集、模型、实验等AI资源的可查找性与质量问题;(2)复现AI论文中实验的困难;(3)可信度与跨学科性的缺失。研究发现表明,尽管早期阶段的AI研究者通常倾向于分享自身的AI资源,但他们缺乏动力或知识去更多参与数据集与代码的准备、整理工作以及伦理评估,也不习惯与应用领域的资深专家开展合作。此外,我们还考察了计算机科学与人工智能领域在数据治理和可复现性方面的现有实践。例如,仅有少数学术会议和期刊主动推行可复现性评估等可复现倡议。至关重要的是,当下亟需立即推行负责任且可复现的AI研究实践,这对整个社会意义重大,对AI研究界而言尤为关键。本文提出了社会层面与技术层面相结合的建议,以应对所发现的挑战。社会层面,我们建议在AI会议和期刊中普遍推行可复现倡议,同时加强跨学科合作,尤其是在数据治理实践方面。技术层面,我们呼吁开发更完善的工具以更好地支持数据集与代码的版本控制,搭建便于AI资源的共享与发现、以及实验的共享、执行和验证的计算基础设施。
英文摘要:
With the goal of uncovering the challenges faced by European AI students during their research endeavors, we surveyed 28 AI doctoral candidates from 13 European countries. The outcomes underscore challenges in three key areas: (1) the findability and quality of AI resources such as datasets, models, and experiments; (2) the difficulties in replicating the experiments in AI papers; (3) and the lack of trustworthiness and interdisciplinarity. From our findings, it appears that although early stage AI researchers generally tend to share their AI resources, they lack motivation or knowledge to engage more in dataset and code preparation and curation, and ethical assessments, and are not used to cooperate with well-versed experts in application domains. Furthermore, we examine existing practices in data governance and reproducibility both in computer science and in artificial intelligence. For instance, only a minority of venues actively promote reproducibility initiatives such as reproducibility evaluations. Critically, there is need for immediate adoption of responsible and reproducible AI research practices, crucial for society at large, and essential for the AI research community in particular. This paper proposes a combination of social and technical recommendations to overcome the identified challenges. Socially, we propose the general adoption of reproducibility initiatives in AI conferences and journals, as well as improved interdisciplinary collaboration, especially in data governance practices. On the technical front, we call for enhanced tools to better support versioning control of datasets and code, and a computing infrastructure that facilitates the sharing and discovery of AI resources, as well as the sharing, execution, and verification of experiments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。