arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.15875cs.CLcs.AI
收藏

真实回答的语言属性

Linguistic Properties of Truthful Response

University of Pennsylvania(宾夕法尼亚大学)

Bruce W. Lee, Benedict Florance Arockiaraj, Helen Jin

AI总结:

本文利用220个手工语言特征分析GPT-3回答,发现不同规模模型的真实性语言特征相似,并训练仅依赖风格的SVM验证不评估内容即可检测真实性的可能。

摘要:

我们使用包含220个手工构建语言特征的大型集合,研究LLM不真实回答这一现象。我们聚焦于GPT-3模型,发现不同模型规模下回答的语言特征概况相似。也就是说,不同规模的LLM对给定提示如何作出回应,在语言属性层面保持相似。我们通过训练 support vector machines(支持向量机)扩展这一发现,这些模型仅依赖模型回答的风格成分来对陈述的真实性进行分类。尽管数据集规模限制了我们当前的发现,但我们表明无需评估内容本身也有可能实现真实性检测。但与此同时,在解释结果时必须考虑到我们实验范围的有限性。

英文摘要:

We investigate the phenomenon of an LLM's untruthful response using a large set of 220 handcrafted linguistic features. We focus on GPT-3 models and find that the linguistic profiles of responses are similar across model sizes. That is, how varying-sized LLMs respond to given prompts stays similar on the linguistic properties level. We expand upon this finding by training support vector machines that rely only upon the stylistic components of model responses to classify the truthfulness of statements. Though the dataset size limits our current findings, we show the possibility that truthfulness detection is possible without evaluating the content itself. But at the same time, the limited scope of our experiments must be taken into account in interpreting the results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑