发表机构
New York University; Harvard University（纽约大学; 哈佛大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究验证了用于种族盲起诉的LLM算法bc2，发现其忠实执行法律要求，并指出删除额外代理信息可消除43.1%的预测信号，表明算法验证能改进政策实施。
AI 中文摘要
加利福尼亚州最近要求该州所有检察官通过审查已删除选定种族相关代理信息的案件文件，进行“种族盲起诉”决策。我们验证了bc2，这是一个我们开发的开源、基于LLM的算法，用于自动化此删除过程，并在2025年被用于促进超过119,000个真实世界案件的种族盲审查。我们评估两个不同的问题：bc2是否忠实地执行了州的要求，以及这些要求即使被忠实执行，是否推进了种族盲决策的目标。为此，我们利用了从美国各地司法管辖区收集的近5,000份真实世界警方报告的语料库。在严格的文档级衡量标准下，我们发现最新版本的bc2在我们样本中96.7%的叙述中忠实地执行了法律要求。这一表现比早期版本的bc2有显著改进，并超过了领先的开源删除方法。我们的验证还表明，加利福尼亚州的要求遗漏了种族的关键代理信息，包括位置信息。删除这些超出州要求范围的额外代理信息，正如bc2所做的那样，消除了遵守要求后剩余的43.1%的预测信号。这些发现表明，验证不仅可以评估技术合规性：它还可以改进算法，并帮助政策制定者实现潜在的政策目标。
英文摘要
California recently required all prosecutors in the state to conduct a "race-blind charging" decision by reviewing case documents in which selected race-related proxies have been redacted. We validate bc2, an open-source, LLM-based algorithm that we developed to automate this redaction and that was used to facilitate race-blind review in more than 119,000 real-world cases in 2025. We evaluate two distinct questions: whether bc2 faithfully implements the state's requirements and whether those requirements, even when faithfully implemented, advance the goal of race-blind decision-making. To do so, we draw on a corpus of nearly 5,000 real-world police reports that we assembled from jurisdictions across the United States. Under a stringent document-level measure, we find that the latest version of bc2 faithfully implements the legal mandate on 96.7% of narratives in our sample. This performance represents a substantial improvement over earlier versions of bc2 and exceeds that of leading open-source redaction methods. Our validation also shows that California's mandate misses key proxies for race, including location information. Redacting these additional proxies beyond those covered by the state mandate, as bc2 does, eliminates 43.1% of the predictive signal that remains after compliance with the mandate. These findings show that validation can do more than assess technical compliance: it can also improve algorithms and help policymakers achieve underlying policy goals.
