arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.14356cs.AIcs.SE
收藏

AI鲁棒性审计的可操作元需求

Actionable Meta-Requirements for AI Robustness Auditing

Jheronimus Academy of Data Science(赫罗尼姆斯数据科学学院);Tilburg University(蒂尔堡大学);University of Sannio(萨尼奥大学)

Joshua Owotogbe, Roya Nasiri, Indika Kumara, Damian Andrew Tamburri, Willem-Jan van den Heuvel

AI总结:

该研究针对AI鲁棒性审计框架需求未整合的问题,通过文献推导并经从业者验证完善出44项元需求,可用于辅助组织实施、评估和优化AI鲁棒性审计框架。

摘要:

AI赋能系统在运行条件发生变化时(例如输入扰动、分布偏移或外部依赖关系变更)必须保持可靠,这使得此类系统的鲁棒性审计对于判断现有证据是否足以支撑明确的鲁棒性结论至关重要。然而,设计AI鲁棒性审计框架颇具挑战性,一个关键未解决问题是确定该框架应支持的需求,这些需求尚未得到整合。我们通过元需求(指定一类审计框架应支持什么的高阶需求)来填补这一空白。在设计科学研究中,文献和领域专家是元需求的常见来源,因此我们从文献中推导初始集合,并与从业者一起验证和完善该集合。我们的系统文献综述从2890条记录中保留了74条,推导了16个类别的43项元需求。我们与128名从业者对这些元需求进行评估,获得了5504条评分和586条实质性评论,总体而言,81.20%的评分表示同意或非常同意,其中43项文献推导的元需求中有41项被归类为得到强烈支持。从业者反馈完善了这些需求,并确定了一项关于外部组件和供应链依赖关系的额外需求,最终集合包含16个类别的44项元需求,其中43项得到了定量验证,这些元需求可帮助组织实施AI鲁棒性审计框架、评估现有框架的完整性并对其进行完善。

英文摘要:

AI-enabled systems must remain dependable when the conditions under which they operate change (e.g., due to input perturbations, distribution shifts, or changes in external dependencies). This makes robustness auditing of such systems important for judging whether the available evidence is sufficient to support a defined robustness conclusion. However, designing an AI robustness auditing framework is challenging. A key unresolved issue is determining the requirements such a framework should support; these requirements have not yet been consolidated. We address this gap through meta-requirements: higher-order requirements specifying what a class of auditing frameworks should support. In design science research, literature and domain experts are common sources of meta-requirements; accordingly, we derive the initial set from literature and validate and refine it with practitioners. Our systematic literature review retained 74 of 2,890 records and derived 43 meta-requirements across 16 categories. We evaluated them with 128 practitioners, obtaining 5,504 ratings and 586 substantive comments. Overall, 81.20% of ratings indicated agreement or strong agreement. In particular, 41 of the 43 literature-derived meta-requirements were classified as strongly supported. Practitioner feedback refined the requirements and identified an additional requirement concerning external components and supply chain dependencies. The final set comprises 44 meta-requirements across 16 categories, of which 43 received quantitative validation. These meta-requirements can help organizations implement AI robustness auditing frameworks, assess existing frameworks for completeness, and refine them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑