arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01397cs.HC
收藏

WeAudit:在审计生成式AI中为用户审计员和AI从业者提供支持

WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI

Wesley Hanwen Deng, Wang Claire, Howard Ziyu Han, Jason I. Hong, Kenneth Holstein, Motahhare Eslami

AI总结:

本文提出WeAudit系统,通过工作流支持最终用户参与AI审计,帮助其发现AI危害并生成从业者可执行的见解,揭示了用户参与AI审计与红队测试的未来机会。

摘要:

从业者和研究人员对让最终用户参与AI审计越来越感兴趣,以利用用户独特的知识和生活经验。然而,关于如何有效地为最终用户提供支持,使其能够产生对AI从业者具有可执行性的见解,我们知之甚少。通过对用户和AI从业者的形成性研究,我们首先确定了一套支持用户参与AI审计的设计目标。随后,我们开发了WeAudit,这是一个支持最终用户进行个人和集体AI审计的工作流与系统。我们通过一项为期三周的用户研究(参与者为用户审计员)以及对行业生成式AI从业者的访谈,对WeAudit进行了评估。研究结果揭示了WeAudit如何帮助用户发现并反思潜在的AI危害,以及如何以行业从业者能够采取行动的方式表达他们的发现。基于我们观察到的结果以及来自用户和从业者的反馈,我们确定了更好地支持用户参与AI审计流程的若干机会。我们讨论了对未来研究的启示,以支持在AI审计和红队测试中实现有效且负责任的用户参与。

英文摘要:

There has been growing interest from both practitioners and researchers in engaging end users in AI auditing, to draw upon users' unique knowledge and lived experiences. However, we know little about how to effectively scaffold end users in auditing in ways that can generate actionable insights for AI practitioners. Through formative studies with both users and AI practitioners, we first identified a set of design goals to support user-engaged AI auditing. We then developed WeAudit, a workflow and system that supports end users in auditing AI both individually and collectively. We evaluated WeAudit through a three-week user study with user auditors and interviews with industry Generative AI practitioners. Our findings offer insights into how WeAudit supports users in noticing and reflecting upon potential AI harms and in articulating their findings in ways that industry practitioners can act upon. Based on our observations and feedback from both users and practitioners, we identify several opportunities to better support user engagement in AI auditing processes. We discuss implications for future research to support effective and responsible user engagement in AI auditing and red-teaming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑