arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.04831cs.AIcs.SE
收藏

代码幻觉

Code Hallucination

Cisco Research(思科研究中心);Purdue University(普渡大学)

Mirza Masfiqur Rahman, Ashish Kundu

AI总结:

本文提出HallTrigger技术,利用LLM的三种动态属性构造提示,无需访问模型内部即可高效触发任意代码幻觉,并验证了LLM幻觉对软件开发的广泛影响。

摘要:

诸如大型语言模型之类的生成模型被广泛用作代码副驾驶和整个程序的生成工具。然而,它们生成的程序在集成方面往往具有可疑的正确性、真实性和可靠性,因为它们可能不遵循用户需求,提供不正确和/或无意义的输出,甚至包含语义/语法错误——这些总体被称为LLM幻觉。在这项工作中,我们提出了几种类型的代码幻觉。我们使用大型语言模型手动生成了此类幻觉代码。我们还提出了一种技术——HallTrigger,以展示生成任意代码幻觉的高效方法。我们的方法利用LLM的3种不同动态属性来设计提示,从而能够成功触发模型的幻觉,而无需访问模型架构或参数。来自流行黑盒模型的结果表明,HallTrigger确实有效,并且普遍存在的LLM幻觉对软件开发产生了巨大影响。

英文摘要:

Generative models such as large language models are extensively used as code copilots and for whole program generation. However, the programs they generate often have questionable correctness, authenticity and reliability in terms of integration as they might not follow the user requirements, provide incorrect and/or nonsensical outputs, or even contain semantic/syntactic errors - overall known as LLM hallucination. In this work, we present several types of code hallucination. We have generated such hallucinated code manually using large language models. We also present a technique - HallTrigger, in order to demonstrate efficient ways of generating arbitrary code hallucination. Our method leverages 3 different dynamic attributes of LLMs to craft prompts that can successfully trigger hallucinations from models without the need to access model architecture or parameters. Results from popular blackbox models suggest that HallTrigger is indeed effective and the pervasive LLM hallucination have sheer impact on software development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑