Much attention has focused on algorithmic audits and impact assessments to hold developers and users of algorithmic systems accountable. But existing algorithmic accountability policy approaches have neglected the lessons from non-algorithmic domains: notably, the importance of interventions that allow for the effective participation of third parties. Our paper synthesizes lessons from other fields on how to craft effective systems of external oversight for algorithmic deployments. First, we discuss the challenges of third party oversight in the current AI landscape. Second, we survey audit systems across domains - e.g., financial, environmental, and health regulation - and show that the institutional design of such audits are far from monolithic. Finally, we survey the evidence base around these design components and spell out the implications for algorithmic auditing. We conclude that the turn toward audits alone is unlikely to achieve actual algorithmic accountability, and sustained focus on institutional design will be required for meaningful third party involvement.
Enhancing LLMs for Governance with Human Oversight: Evaluating and Aligning LLMs on Expert Classification of Climate Misinformation for Detecting False or Misleading Claims about Climate Change
Designing AI for Prosecutorial Governance: Case Prioritization and Statutory Oversight in Mexico
Tecnológico de Monterrey(蒙特雷科技大学);Centro de Ciencia de Datos - EGobiernoyTP(数据科学中心 - EGobiernoyTP);Fiscalía General de Justicia del Estado de Zacatecas(萨卡特卡斯州总检察院)
Oversight Structures for Agentic AI in Public-Sector Organizations
Centre for Digital Governance, Hertie School(数字治理中心,赫尔姆斯学校);Oxford Internet Institute, University of Oxford(牛津互联网研究所,牛津大学);Weizenbaum Institute Berlin, Germany(贝伦贝格Weizenbaum研究所);Technical University Munich, Germany(慕尼黑技术大学)
When Should Algorithms Resign? A Proposal for AI Governance
Center for Data Science, New York University(纽约大学数据科学中心);The Alan Turing Institute(艾伦·图灵研究所);Faculty of Law, University of Cambridge(剑桥大学法学院);Berkman Klein Center for Internet & Society(伯克曼 Klein 互联网与社会中心)