arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.01668cs.CRcs.SE
收藏

LineBreaker:使用大型语言模型查找Token不一致性漏洞

LineBreaker: Finding Token-Inconsistency Bugs with Large Language Models

Indiana University Bloomington(印第安纳大学布鲁明顿分校);San Diego State University(圣地亚哥州立大学);The Hong Kong University of Science and Technology(香港科技大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学)

Hongbo Chen, Yifan Zhang, Xing Han, Tianhao Mao, Huanyao Rong, Yuheng Zhang, XiaoFeng Wang, Luyi Xing, Xun Chen, Hang Zhang

AI总结:

本文提出级联TIB检测系统\ ame,利用小型高效代码语言模型过滤无关代码,结合GPT-4的语义理解能力,在154个GitHub仓库中发现123个新缺陷,显著提升了Token不一致性漏洞检测的精确度、召回率和可扩展性。

摘要:

Token不一致性漏洞(TIBs)涉及对语法有效但错误的代码token的误用,例如误用的变量和错误的函数调用,这通常会导致软件漏洞。与简单的语法漏洞不同,TIBs发生在语义层面且非常隐蔽——有时它们会多年未被发现。传统检测方法,如静态分析和动态测试,由于其多变且依赖上下文的特性,在处理TIBs时常常面临困难。然而,像GPT-4这样的大型语言模型(LLMs)的进展,通过利用这些模型的语义理解能力,为自动化TIB检测提供了新的机遇。本文报告了针对LLMs检测TIBs能力的首次系统性测量,揭示了尽管GPT-4展现出潜力,但它在精确度和可扩展性方面表现出局限性。具体而言,其检测能力因模型倾向于关注不包含TIBs的代码片段而受到削弱;其可扩展性问题则源于GPT-4的高昂成本以及需要检查的海量代码。为了应对这些挑战,我们引入了\ ame,一个新颖的级联TIB检测系统。\ ame利用更小的、特定于代码且高效的语言模型来过滤掉大量不太可能包含TIBs的代码片段,从而在精确度、召回率和可扩展性方面显著提升了系统性能。我们在154个Python和C GitHub仓库上评估了\ ame,每个仓库拥有超过1,000个星标,发现了123个新缺陷,其中45%可被利用来破坏程序功能。在我们提交的69个修复中,已有41个被确认或合并。

英文摘要:

Token-inconsistency bugs (TIBs) involve the misuse of syntactically valid yet incorrect code tokens, such as misused variables and erroneous function invocations, which can often lead to software bugs. Unlike simple syntactic bugs, TIBs occur at the semantic level and are subtle - sometimes they remain undetected for years. Traditional detection methods, such as static analysis and dynamic testing, often struggle with TIBs due to their versatile and context-dependent nature. However, advancements in large language models (LLMs) like GPT-4 present new opportunities for automating TIB detection by leveraging these models' semantic understanding capabilities. This paper reports the first systematic measurement of LLMs' capabilities in detecting TIBs, revealing that while GPT-4 shows promise, it exhibits limitations in precision and scalability. Specifically, its detection capability is undermined by the model's tendency to focus on the code snippets that do not contain TIBs; its scalability concern arises from GPT-4's high cost and the massive amount of code requiring inspection. To address these challenges, we introduce \name, a novel and cascaded TIB detection system. \name leverages smaller, code-specific, and highly efficient language models to filter out large numbers of code snippets unlikely to contain TIBs, thereby significantly enhancing the system's performance in terms of precision, recall, and scalability. We evaluated \name on 154 Python and C GitHub repositories, each with over 1,000 stars, uncovering 123 new flaws, 45\% of which could be exploited to disrupt program functionalities. Out of our 69 submitted fixes, 41 have already been confirmed or merged.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑