务实审计:一种以试点驱动的方法来审计机器学习系统
Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems
AI总结:
本文提出一种扩展欧盟AI-HLEG指南的务实机器学习审计程序,基于关注文档、问责与质量的生命周期模型,并通过两个真实试点验证,旨在推动算法审计的广泛采用。
摘要:
机器学习(ML)系统的日益普及和部署伴随着一系列伦理事件和社会关切。这也揭示了根据伦理原则对这些系统进行适当审计的必要性。为了使这种新型的算法审计成为标准实践,需要具备两个主要前提:一个针对透明度和问责制量身定制的生命周期模型,以及一个允许对审计进行适当范围界定的原则性风险评估程序。为了朝着更广泛采用机器学习审计迈出务实的一步,我们提出了一种相应的程序,该程序扩展了欧盟委员会发布的AI-HLEG指南。我们的审计程序基于一个明确关注文档、问责制和质量保证的机器学习生命周期模型,并作为审计方与被审计组织之间对齐的共同基础。我们描述了两个在不同组织的真实世界用例上进行的试点,并讨论了机器学习算法审计的不足之处及其未来方向。
英文摘要:
The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transparency and accountability, and a principled risk assessment procedure that allows the proper scoping of the audit. Aiming to make a pragmatic step towards a wider adoption of ML auditing, we present a respective procedure that extends the AI-HLEG guidelines published by the European Commission. Our audit procedure is based on an ML lifecycle model that explicitly focuses on documentation, accountability, and quality assurance; and serves as a common ground for alignment between the auditors and the audited organisation. We describe two pilots conducted on real-world use cases from two different organisations and discuss the shortcomings of ML algorithmic auditing as well as future directions thereof.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。