论法律文书审查工作流程中的成本最小化
On Minimizing Cost in Legal Document Review Workflows
摘要:
技术辅助审查(TAR)是指用于法律发现和其他高召回审查任务中的文档审查的人在回路机器学习工作流。律师和法律技术专家一直在争论审查是否应该是一个单一的迭代过程(单阶段TAR工作流程),或者模型培训和审查是否应该是分开的(两阶段TAR工作流程),这对主动学习算法的选择有影响。不同目的(培训与审查)和不同文档(正面与负面示例)的人工标记的相对成本是这场辩论中被忽视的关键因素。使用一种新的成本动态分析,我们分析和经验表明,这些相对成本强烈影响是否一个阶段或两个阶段的工作流程最大限度地降低成本。我们还展示了类别流行率、分类任务难度和集合大小如何影响工作流类型的最佳选择,以及主动学习方法和停止点的最佳选择。
英文摘要:
Technology-assisted review (TAR) refers to human-in-the-loop machine learning workflows for document review in legal discovery and other high recall review tasks. Attorneys and legal technologists have debated whether review should be a single iterative process (one-phase TAR workflows) or whether model training and review should be separate (two-phase TAR workflows), with implications for the choice of active learning algorithm. The relative cost of manual labeling for different purposes (training vs. review) and of different documents (positive vs. negative examples) is a key and neglected factor in this debate. Using a novel cost dynamics analysis, we show analytically and empirically that these relative costs strongly impact whether a one-phase or two-phase workflow minimizes cost. We also show how category prevalence, classification task difficulty, and collection size impact the optimal choice not only of workflow type, but of active learning method and stopping point.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。