arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.13224cs.CYcs.AI
收藏

评估一种提升AI透明度的方法:一项案例研究

Evaluating a Methodology for Increasing AI Transparency: A Case Study

IBM Research AI(IBM研究院AI部门)

David Piorkowski, John Richards, Michael Hind

AI总结:

本研究首次对Richards等人提出的以用户为中心的AI文档生成方法开展实践评估,通过医疗AI团队的应用案例验证了其可用性,同时分析了该方法的收益成本并给出改进建议。

摘要:

针对人们对人工智能(AI)潜在危害日益增长的担忧,社会各界已开始要求提高AI模型与系统构建及使用方式的透明度。为回应这些关切,已有多项研究提出了包含待模型开发者回答问题的文档模板。这些模板提供了有用的起点,但没有任何单一模板能满足各类文档使用者的需求。不过,原则上可以创建一种可重复的方法来生成真正实用的文档。Richards等人[25]提出了这样一种方法,用于识别具体的文档需求并开发满足这些需求的模板。尽管这一提案颇具前景,但尚未经过评估。\n 本文首次对这种以用户为中心的方法进行了实践评估,报告了医疗AI领域的一个团队采用该方法提升多款AI模型透明度的实践经验。研究发现,未接受过以用户为中心技术培训的开发者也能使用该方法,它能引导开发者创建出既满足使用者具体需求、又可在不同模型和用例间复用的文档模板。本文分析了该方法的收益与成本,并总结了对方法本身及配套工具的进一步改进建议。

英文摘要:

In reaction to growing concerns about the potential harms of artificial intelligence (AI), societies have begun to demand more transparency about how AI models and systems are created and used. To address these concerns, several efforts have proposed documentation templates containing questions to be answered by model developers. These templates provide a useful starting point, but no single template can cover the needs of diverse documentation consumers. It is possible in principle, however, to create a repeatable methodology to generate truly useful documentation. Richards et al. [25] proposed such a methodology for identifying specific documentation needs and creating templates to address those needs. Although this is a promising proposal, it has not been evaluated. This paper presents the first evaluation of this user-centered methodology in practice, reporting on the experiences of a team in the domain of AI for healthcare that adopted it to increase transparency for several AI models. The methodology was found to be usable by developers not trained in user-centered techniques, guiding them to creating a documentation template that addressed the specific needs of their consumers while still being reusable across different models and use cases. Analysis of the benefits and costs of this methodology are reviewed and suggestions for further improvement in both the methodology and supporting tools are summarized.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑