我们使用包含220个手工构建语言特征的大型集合,研究LLM不真实回答这一现象。我们聚焦于GPT-3模型,发现不同模型规模下回答的语言特征概况相似。也就是说,不同规模的LLM对给定提示如何作出回应,在语言属性层面保持相似。我们通过训练 support vector machines(支持向量机)扩展这一发现,这些模型仅依赖模型回答的风格成分来对陈述的真实性进行分类。尽管数据集规模限制了我们当前的发现,但我们表明无需评估内容本身也有可能实现真实性检测。但与此同时,在解释结果时必须考虑到我们实验范围的有限性。
英文摘要:
We investigate the phenomenon of an LLM's untruthful response using a large set of 220 handcrafted linguistic features. We focus on GPT-3 models and find that the linguistic profiles of responses are similar across model sizes. That is, how varying-sized LLMs respond to given prompts stays similar on the linguistic properties level. We expand upon this finding by training support vector machines that rely only upon the stylistic components of model responses to classify the truthfulness of statements. Though the dataset size limits our current findings, we show the possibility that truthfulness detection is possible without evaluating the content itself. But at the same time, the limited scope of our experiments must be taken into account in interpreting the results.
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Key Laboratory of AI Safety, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室)
The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses
Warsaw University of Technology(华沙理工大学);University of Warsaw(华沙大学)
TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention
Drexel University(德雷塞尔大学);University of Electronic Science and Technology of China(电子科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));LLNL(劳伦斯利弗莫尔国家实验室);Lehigh University(莱斯大学)
Cross-linguistic disagreement as a conflict of semantic alignment norms in multilingual AI~Linguistic Diversity as a Problem for Philosophy, Cognitive Science, and AI~
Japan Advanced Institute of Science and Technology(日本先进科学技术学院);Victoria University of Wellington(惠灵顿维多利亚大学);Macquarie University(麦考瑞大学);Nanzan University(南山大学);Kyoto University(京都大学)
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations
Institute for Logic, Language and Computation (ILLC), University of Amsterdam(逻辑、语言与计算研究所(ILLC),阿姆斯特丹大学);Institute for Language, Cognition and Computation (ILCC), University of Edinburgh(语言、认知与计算研究所(ILCC),爱丁堡大学)
Reasoning-Token Spikes Under Prompted Untruthful Responses in Large Language Models
Chapman University(查普曼大学);INSERM U992, Cognitive Neuroimaging Unit, NeuroSpin Center(法国国家健康与医学研究院U992认知神经影像单元,神经自旋中心);University of California, Los Angeles(加州大学洛杉矶分校);California Institute of Technology(加州理工学院)
The Right to Be Remembered: Preserving Maximally Truthful Digital Memory in the Age of AI
Insilico Medicine AI Limited(英矽智能人工智能有限公司);Insilico Medicine US Inc(英矽智能美国公司);Insilico Medicine Hong Kong Ltd.(英矽智能香港有限公司);Duke University(杜克大学);Duke Kunshan University(昆山杜克大学);University of Louisville(路易斯维尔大学)
Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries
Siebel School of Computing and Data Science, The Grainger College of Engineering, University of Illinois Urbana-Champaign(Siebel计算与数据科学学院,格拉inger工程学院,伊利诺伊大学厄巴纳-香槟分校);Department of Psychology, College of Liberal Arts and Sciences, University of Illinois Urbana-Champaign(心理学系,文理学院,伊利诺伊大学厄巴纳-香槟分校);School of Interactive Computing, College of Computing, Georgia Institute of Technology(交互计算学院,计算机学院,佐治亚理工学院)
Comprehensive Evaluation of Large Language Model Responses: A Multi-Factor Scoring System
School of Computer Science and Engineering, Beihang University, Beijing, China(北京航空航天大学计算机科学与工程学院);Data Science and Intelligent Computing Laboratory, Hangzhou International Innovation Institute, Beihang University, Hangzhou, Zhejiang(北京航空航天大学杭州国际创新研究院数据科学与智能计算实验室)
A Multi-Task Role-Playing Agent Capable of Imitating Character Linguistic Styles
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Huawei Technologies Co., Ltd.(华为技术有限公司)