人类判断如何损害自动欺骗检测性能
How human judgment impairs automated deception detection performance
摘要：
背景：欺骗检测是安全从业人员普遍面临的问题。随着对更大规模方法的需求，使用机器学习的自动化方法已经获得了牵引力。然而，检测性能仍然意味着相当大的出错率。其他领域的研究结果表明，混合人机集成可以提供一个可行的路径在欺骗检测任务。方法：我们收集了一个关于参与者自传体意图的真实和欺骗性答案的语料库（n=1640），并测试了监督机器学习和人类判断的组合是否可以提高欺骗检测的准确性。人类法官被提交了真实和欺骗性陈述的自动可信度判断结果。他们可以完全否决它（混合否决条件）或在给定的边界内调整它（混合调整条件）。结果：数据表明，在两种混合条件下，人类判断都没有增加有意义的贡献。孤立的机器学习可以识别出说真话者和说谎者，总体准确率为69%。通过混合否决决策的人类参与将准确性带回到机会水平。混合调节条件对检测性能没有欺骗作用。人类的决策策略表明，真相偏见--假设对方说的是实话的倾向--可以解释这种不利影响。结论：目前的研究并不支持人类可以有意义地增加机器学习系统的欺骗检测性能的观点。
英文摘要：
Background: Deception detection is a prevalent problem for security practitioners. With a need for more large-scale approaches, automated methods using machine learning have gained traction. However, detection performance still implies considerable error rates. Findings from other domains suggest that hybrid human-machine integrations could offer a viable path in deception detection tasks. Method: We collected a corpus of truthful and deceptive answers about participants' autobiographical intentions (n=1640) and tested whether a combination of supervised machine learning and human judgment could improve deception detection accuracy. Human judges were presented with the outcome of the automated credibility judgment of truthful and deceptive statements. They could either fully overrule it (hybrid-overrule condition) or adjust it within a given boundary (hybrid-adjust condition). Results: The data suggest that in neither of the hybrid conditions did the human judgment add a meaningful contribution. Machine learning in isolation identified truth-tellers and liars with an overall accuracy of 69%. Human involvement through hybrid-overrule decisions brought the accuracy back to the chance level. The hybrid-adjust condition did not deception detection performance. The decision-making strategies of humans suggest that the truth bias - the tendency to assume the other is telling the truth - could explain the detrimental effect. Conclusion: The current study does not support the notion that humans can meaningfully add to the deception detection performance of a machine learning system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
