检索增强的少样本提示与微调在代码漏洞检测中的比较
Retrieval-Augmented Few-Shot Prompting Versus Fine-Tuning for Code Vulnerability Detection
AI总结：
本研究比较了检索增强的少样本提示与微调方法在代码漏洞检测中的性能，发现检索增强提示在效率和效果上均优于微调方法。
摘要：
少样本提示已成为一种实用的替代方法，用于利用大型语言模型（LLMs）在专业任务中的能力。然而，其有效性在复杂领域中严重依赖于上下文示例的选择和质量。在本工作中，我们研究了检索增强提示作为一种策略，以提高代码漏洞检测中的少样本性能，其目标是识别给定代码片段中存在于预定义漏洞类别集中的一个或多个安全相关弱点。我们使用Gemini-1.5-Flash模型进行系统评估，采用三种方法：（1）使用随机选择示例的标准少样本提示；（2）使用语义相似示例的检索增强提示；（3）基于检索示例的标签分配，该方法根据检索示例分配标签而无需模型推理。我们的结果表明，检索增强提示在所有提示策略中表现最突出。在20次提示下，其F1得分为74.05%，部分匹配准确率为83.90%。我们进一步将此方法与零样本提示和几种微调模型进行比较，包括Gemini-1.5-Flash和较小的开源模型，如DistilBERT、DistilGPT2和CodeBERT。检索增强提示在零样本提示（F1得分为36.35%，部分匹配准确率为20.30%）和微调Gemini（F1得分为59.31%，部分匹配准确率为53.10%）方面均表现更优，同时避免了微调模型所需的时间和成本。另一方面，微调CodeBERT的性能更高（F1得分为91.22%，部分匹配准确率为91.30%），但需要额外的训练、维护努力和资源。
英文摘要：
Few-shot prompting has emerged as a practical alternative to fine-tuning for leveraging the capabilities of large language models (LLMs) in specialized tasks. However, its effectiveness depends heavily on the selection and quality of in-context examples, particularly in complex domains. In this work, we examine retrieval-augmented prompting as a strategy to improve few-shot performance in code vulnerability detection, where the goal is to identify one or more security-relevant weaknesses present in a given code snippet from a predefined set of vulnerability categories. We perform a systematic evaluation using the Gemini-1.5-Flash model across three approaches: (1) standard few-shot prompting with randomly selected examples, (2) retrieval-augmented prompting using semantically similar examples, and (3) retrieval-based labeling, which assigns labels based on retrieved examples without model inference. Our results show that retrieval-augmented prompting consistently outperforms the other prompting strategies. At 20 shots, it achieves an F1 score of 74.05% and a partial match accuracy of 83.90%. We further compare this approach against zero-shot prompting and several fine-tuned models, including Gemini-1.5-Flash and smaller open-source models such as DistilBERT, DistilGPT2, and CodeBERT. Retrieval-augmented prompting outperforms both zero-shot (F1 score: 36.35%, partial match accuracy: 20.30%) and fine-tuned Gemini (F1 score: 59.31%, partial match accuracy: 53.10%), while avoiding the training time and cost associated with model fine-tuning. On the other hand, fine-tuning CodeBERT yields higher performance (F1 score: 91.22%, partial match accuracy: 91.30%) but requires additional training, maintenance effort, and resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
