arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.11432cs.CL
收藏

欺骗检测能更深入吗?欺骗推理的数据集、评估与基准

Can Deception Detection Go Deeper? Dataset, Evaluation, and Benchmark for Deception Reasoning

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai Jiao Tong University(上海交通大学);Inner Mongolia University(内蒙古大学);Tsinghua University(清华大学);Peking University(北京大学)

Kang Chen, Zheng Lian, Haiyang Sun, Rui Liu, Jiangyan Yi, Bin Liu, Jianhua Tao

AI总结:

该研究将欺骗检测拓展为欺骗推理任务,通过结合事实矛盾与意图分析谎言成因,构建了数据集并定义评估指标,可作为大语言模型复杂推理能力的评估基准。

摘要:

欺骗检测因其在现实场景中的重要性受到越来越多的关注,其主要目标是从手势、面部表情、韵律等多模态线索中检测欺骗行为。但这些依据通常具有主观性,且与个人习惯相关。因此,我们将欺骗检测拓展为欺骗推理,进一步为主观判断提供客观证据支撑。具体而言,我们提供潜在谎言与基础事实,再结合事实矛盾及其背后的意图分析该语句可能为谎言的原因。与欺骗检测相比,该任务更适用于现实场景,例如在审讯中,警方需基于确凿证据判断一个人是否在说谎。本文呈现了我们在该任务上的初步尝试,包括构建数据集与定义评估指标。同时,该任务可作为评估大语言模型复杂推理能力的基准。我们的代码与数据见补充材料。

英文摘要:

Deception detection has attracted increasing attention due to its importance in real-world scenarios. Its main goal is to detect deceptive behaviors from multimodal clues such as gestures, facial expressions, prosody, etc. However, these bases are usually subjective and related to personal habits. Therefore, we extend deception detection to deception reasoning, further providing objective evidence to support subjective judgment. Specifically, we provide potential lies and basic facts and then analyze why this sentence may be a lie by combining factual inconsistencies and intent behind them. Compared with deception detection, this task is more applicable to real-world scenarios. For example, in interrogation, the police should judge whether a person is lying based on solid evidence. This paper presents our initial attempts at this task, including constructing a dataset and defining evaluation metrics. Meanwhile, this task can serve as a benchmark for evaluating the complex reasoning capability of large language models. Our code and data are provided in the supplementary material.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑