技术辅助评审的启发式停止规则
Heuristic Stopping Rules For Technology-Assisted Review
摘要：
技术辅助审阅（TAR）是指用于在大型集合中查找相关文档的人在回路主动学习工作流程。这些工作流程通常必须满足所找到的相关文档的比例（即召回）的目标，同时还要降低成本。为了在特定的环境中达到这种平衡，已经提出了各种启发式的停止规则，但是没有一个针对一系列回忆目标和任务进行了测试。我们提出了两个新的启发式停止规则，定量和QuantCI基于模型的估计技术的调查研究。我们将它们与一系列建议的召回方法进行比较，发现它们在达到一系列召回目标方面是准确的，同时大大降低了审查成本。
英文摘要：
Technology-assisted review (TAR) refers to human-in-the-loop active learning workflows for finding relevant documents in large collections. These workflows often must meet a target for the proportion of relevant documents found (i.e. recall) while also holding down costs. A variety of heuristic stopping rules have been suggested for striking this tradeoff in particular settings, but none have been tested against a range of recall targets and tasks. We propose two new heuristic stopping rules, Quant and QuantCI based on model-based estimation techniques from survey research. We compare them against a range of proposed heuristics and find they are accurate at hitting a range of recall targets while substantially reducing review costs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
