arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.04510cs.SEcs.AI
收藏

CGP-Tuning:面向代码漏洞检测的结构感知软提示微调

CGP-Tuning: Structure-Aware Soft Prompt Tuning for Code Vulnerability Detection

School of Computer Science and Engineering, University of New South Wales (UNSW)(新南威尔士大学计算机科学与工程学院);Department of Electrical Engineering and Information Technology, University of Naples Federico II(那不勒斯费德里克二世大学电气工程与信息技术系)

Ruijun Feng, Hammond Pearce, Pietro Liguori, Yulei Sui

AI总结:

CGP-Tuning提出一种结构感知的代码图增强软提示微调方法,通过类型感知嵌入和高效跨模态对齐模块提升代码漏洞检测性能,并在多个开源代码大模型上取得显著优于现有基线的效果。

摘要:

大语言模型(LLMs)已被提出作为检测软件漏洞的强大工具,其中通常采用任务特定的微调来为LLMs提供漏洞相关知识。然而,现有的微调技术往往将源代码视为纯文本,丢失了代码中固有的基于图的结构信息。图增强的软提示微调通过将结构信息转化为LLM能够理解的上下文线索来解决这一问题。然而,当前方法主要针对一般图相关任务设计,并且更侧重于邻接信息,在保留代码图中丰富的语义信息(例如控制流/数据流)方面存在不足。它们也无法在跨模态对齐模块中捕捉图-文本交互的同时保证计算效率。本文提出了CGP-Tuning,一种新的代码图增强、结构感知的软提示微调方法,用于漏洞检测。CGP-Tuning引入了类型感知嵌入来捕捉代码图中丰富的语义信息,并设计了一个高效的跨模态对齐模块,在融入图-文本交互的同时实现线性计算成本。该方法在最新的DiverseVul数据集以及三个先进的开源代码大语言模型CodeLlama、CodeGemma和Qwen2.5-Coder上进行了评估。实验结果表明,CGP-Tuning带来了模型无关的性能提升,并保持了实用的推理速度,比最佳的图增强软提示微调基线平均高出四个百分点,比未微调的零样本提示高出15个百分点。

英文摘要:

Large language models (LLMs) have been proposed as powerful tools for detecting software vulnerabilities, where task-specific fine-tuning is typically employed to provide vulnerability-specific knowledge to the LLMs. However, existing fine-tuning techniques often treat source code as plain text, losing the graph-based structural information inherent in code. Graph-enhanced soft prompt tuning addresses this by translating the structural information into contextual cues that the LLM can understand. However, current methods are primarily designed for general graph-related tasks and focus more on adjacency information, they fall short in preserving the rich semantic information (e.g., control/data flow) within code graphs. They also fail to ensure computational efficiency while capturing graph-text interactions in their cross-modal alignment module. This paper presents CGP-Tuning, a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection. CGP-Tuning introduces type-aware embeddings to capture the rich semantic information within code graphs, along with an efficient cross-modal alignment module that achieves linear computational costs while incorporating graph-text interactions. It is evaluated on the latest DiverseVul dataset and three advanced open-source code LLMs, CodeLlama, CodeGemma, and Qwen2.5-Coder. Experimental results show that CGP-Tuning delivers model-agnostic improvements and maintains practical inference speed, surpassing the best graph-enhanced soft prompt tuning baseline by an average of four percentage points and outperforming non-tuned zero-shot prompting by 15 percentage points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑