Scrutinizer：大规模数据驱动索赔验证的混合主动方法
Scrutinizer: A Mixed-Initiative Approach to Large-Scale, Data-Driven Claim Verification
摘要：
国际能源署（IEA）等组织花费大量时间和金钱手动检查汇总数据的文本文档。Scrutinizer系统的目标是通过支持人工事实检查员将文本声明转换为相关数据库上的SQL查询来减少验证开销。审查者协调人类事实核查小组。它通过向用户提出查询或查询片段来减少验证时间。这些建议是基于索赔文本分类器，逐步提高在一个大的文件的验证。此外，Scrutinizer使用查询候选项的试验性执行来缩小备选项集。验证过程由基于成本的优化器控制。它优化了与用户的交互，并优先考虑索赔验证。对于后者，它认为预期的验证开销以及预期的索赔效用作为分类器的训练样本。我们使用模拟和用户研究来评估Scrutinizer系统，基于实际索赔和数据，并使用IEA雇用的专业事实检查员。我们的实验一致表明，在不降低结果准确性的情况下，显著节省了验证时间。
英文摘要：
Organizations such as the International Energy Agency (IEA) spend significant amounts of time and money to manually fact check text documents summarizing data. The goal of the Scrutinizer system is to reduce verification overheads by supporting human fact checkers in translating text claims into SQL queries on an associated database. Scrutinizer coordinates teams of human fact checkers. It reduces verification time by proposing queries or query fragments to the users. Those proposals are based on claim text classifiers, that gradually improve during the verification of a large document. In addition, Scrutinizer uses tentative execution of query candidates to narrow down the set of alternatives. The verification process is controlled by a cost-based optimizer. It optimizes the interaction with users and prioritizes claim verifications. For the latter, it considers expected verification overheads as well as the expected claim utility as training samples for the classifiers. We evaluate the Scrutinizer system using simulations and a user study, based on actual claims and data and using professional fact checkers employed by IEA. Our experiments consistently demonstrate significant savings in verification time, without reducing result accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
