AHA!：通过生成危害示例促进 AI 影响评估
AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms
AI总结：
本文提出 AHA! 生成式框架，通过结合问题性 AI 行为、利益相关者影响、众包与 LLM 生成危害情境，辅助 AI 影响评估，并经 4113 项危害和 9 名专家访谈验证其有效性。
摘要：
尽管对有害 AI 后果进行变革和问责的要求不断增加，但预见部署 AI 系统的下游影响仍然是一项具有挑战性的任务。我们开发了 AHA!（Anticipating Harms of AI，预见 AI 危害），这是一个生成式框架，旨在协助 AI 从业者和决策者在开发或部署之前预见 AI 系统的潜在危害和意外后果。给定一个 AI 部署场景，AHA! 会为不同利益相关者生成可能危害的描述。为此，AHA! 系统性地考虑常见有问题的 AI 行为之间的相互作用，以及这些行为对不同利益相关者的潜在影响，并通过短篇情境叙述来描述这些条件。随后，这些情境通过提示众包工作者和大型语言模型，填入可能危害的描述。通过审查 AHA! 针对五个不同 AI 部署场景所呈现的 4113 项危害，我们发现 AHA! 能够生成有意义的危害示例，而且不同的有问题 AI 行为会导致不同类型的危害。与仅由众包群体或仅由模型生成的示例相比，同时利用这些情境提示众包群体和大型语言模型产生了更加多样化的危害示例。为评估 AHA! 的潜在实际效用，我们还对负责任 AI 专业人士进行了半结构化访谈（N=9）。参与者认为，AHA! 呈现危害的系统性方法对于伦理反思非常重要，并发现了他们认为原本不会想到的有意义的利益相关者和危害。然而，参与者对于 AHA! 应在前期使用还是作为二次检查存在不同意见，并指出 AHA! 可能会将危害预见从一个构思问题转变为一个可能要求很高的审查问题。基于我们的结果，我们讨论了构建工具以帮助从业者设想可能危害的设计启示。
英文摘要：
While demands for change and accountability for harmful AI consequences mount, foreseeing the downstream effects of deploying AI systems remains a challenging task. We developed AHA! (Anticipating Harms of AI), a generative framework to assist AI practitioners and decision-makers in anticipating potential harms and unintended consequences of AI systems prior to development or deployment. Given an AI deployment scenario, AHA! generates descriptions of possible harms for different stakeholders. To do so, AHA! systematically considers the interplay between common problematic AI behaviors as well as their potential impacts on different stakeholders, and narrates these conditions through vignettes. These vignettes are then filled in with descriptions of possible harms by prompting crowd workers and large language models. By examining 4113 harms surfaced by AHA! for five different AI deployment scenarios, we found that AHA! generates meaningful examples of harms, with different problematic AI behaviors resulting in different types of harms. Prompting both crowds and a large language model with the vignettes resulted in more diverse examples of harms than those generated by either the crowd or the model alone. To gauge AHA!'s potential practical utility, we also conducted semi-structured interviews with responsible AI professionals (N=9). Participants found AHA!'s systematic approach to surfacing harms important for ethical reflection and discovered meaningful stakeholders and harms they believed they would not have thought of otherwise. Participants, however, differed in their opinions about whether AHA! should be used upfront or as a secondary-check and noted that AHA! may shift harm anticipation from an ideation problem to a potentially demanding review problem. Drawing on our results, we discuss design implications of building tools to help practitioners envision possible harms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
