当谎言大多为真时:面向嵌入式谎言的自动化言语欺骗检测
When lies are mostly truthful: automated verbal deception detection for embedded lies
AI总结:
本文构建含2088条陈述及嵌入式谎言标注的新数据集,并微调Llama-3-8B进行检测,发现嵌入式谎言因高度近似真实内容而使分类准确率仅达64%。
摘要:
背景:言语欺骗检测研究依赖叙述,并通常将陈述假定为真实或欺骗性的。一种更现实的观点承认,陈述的真实性存在于一个连续体上,真实部分和欺骗部分会嵌入同一陈述之中。然而,关于嵌入式谎言的研究一直滞后。方法:我们收集了一个包含2088条真实与欺骗性陈述的新颖数据集,并标注了其中的嵌入式谎言。研究采用被试内设计,参与者先提供一段关于自传体事件的真实叙述;随后,他们通过加入嵌入式谎言,以欺骗性方式重写该陈述,并在事后高亮这些谎言,同时评判其谎言中心性、欺骗程度和来源。结果:我们表明,经过微调的语言模型(Llama-3-8B)能够以64%的准确率对真实陈述和包含嵌入式谎言的陈述进行分类。个体差异、语言属性和可解释性分析表明,将检测重点转向嵌入式谎言所面临的挑战,源于它们与真实陈述的相似性。典型的欺骗性陈述由三分之二的真实信息和三分之一的嵌入式谎言构成,这些谎言主要来自过去的个人经历,并且与其对应的真实陈述之间只有极小的语言差异。结论:我们将该数据集作为一种新颖资源提出,以应对这一挑战并推动言语欺骗检测中嵌入式谎言的研究。
英文摘要:
Background: Verbal deception detection research relies on narratives and commonly assumes statements as truthful or deceptive. A more realistic perspective acknowledges that the veracity of statements exists on a continuum with truthful and deceptive parts being embedded within the same statement. However, research on embedded lies has been lagging behind. Methods: We collected a novel dataset of 2,088 truthful and deceptive statements with annotated embedded lies. Using a within-subjects design, participants provided a truthful account of an autobiographical event. They then rewrote their statement in a deceptive manner by including embedded lies, which they highlighted afterwards and judged on lie centrality, deceptiveness, and source. Results: We show that a fined-tuned language model (Llama-3-8B) can classify truthful statements and those containing embedded lies with 64% accuracy. Individual differences, linguistic properties and explainability analysis suggest that the challenge of moving the dial towards embedded lies stems from their resemblance to truthful statements. Typical deceptive statements consisted of 2/3 truthful information and 1/3 embedded lies, largely derived from past personal experiences and with minimal linguistic differences with their truthful counterparts. Conclusion: We present this dataset as a novel resource to address this challenge and foster research on embedded lies in verbal deception detection.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。