认证单阶段技术辅助审查
Certifying One-Phase Technology-Assisted Reviews
摘要:
基于迭代主动学习的技术辅助审阅(TAR)工作流在文档审阅应用中被广泛使用。单阶段TAR工作流程的大多数停止规则缺乏有效的统计保证,这阻碍了它们在某些法律环境中的使用。借鉴分位数估计理论,我们提供了第一个广泛适用的和统计上有效的基于样本的停止规则的单相TAR。我们进一步从理论和经验上表明,超过召回目标,这已被视为无害或可取的在过去的评估停止规则,是一个主要来源的超额成本在一个阶段TAR工作流程。与直觉相反,在几乎所有情况下,增加抽样成本以减少过度召回会降低总成本。
英文摘要:
Technology-assisted review (TAR) workflows based on iterative active learning are widely used in document review applications. Most stopping rules for one-phase TAR workflows lack valid statistical guarantees, which has discouraged their use in some legal contexts. Drawing on the theory of quantile estimation, we provide the first broadly applicable and statistically valid sample-based stopping rules for one-phase TAR. We further show theoretically and empirically that overshooting a recall target, which has been treated as innocuous or desirable in past evaluations of stopping rules, is a major source of excess cost in one-phase TAR workflows. Counterintuitively, incurring a larger sampling cost to reduce excess recall leads to lower total cost in almost all scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。