MultiVer:零样本多智能体漏洞检测
MultiVer: Zero-Shot Multi-Agent Vulnerability Detection
AI总结:
MultiVer通过零样本多智能体系统实现漏洞检测的高召回率,超越微调模型性能,但精度有所下降。
摘要:
我们提出了MultiVer,一种零样本多智能体系统,用于漏洞检测,实现了最先进的召回率而无需微调。一个四智能体集合(安全、正确性、性能、风格)通过并集投票在PyVul上实现了82.7%的召回率,超过微调后的GPT-3.5(81.3%)1.4个百分点——这是首个超越微调性能的零样本系统。在SecurityEval上,相同的架构实现了91.7%的检测率,与专门系统相当。召回率的提升是以精度为代价的:48.8%的精度相对于微调基线的63.9%,得到61.4%的F1。消融实验隔离了组件贡献:多智能体集合在单智能体安全分析上增加了17个百分点的召回率。这些结果表明,在安全应用中,假阴性成本高于假阳性时,零样本多智能体集合可以匹配并超越微调模型在最重要的指标上。
英文摘要:
We present MultiVer, a zero-shot multi-agent system for vulnerability detection that achieves state-of-the-art recall without fine-tuning. A four-agent ensemble (security, correctness, performance, style) with union voting achieves 82.7% recall on PyVul, exceeding fine-tuned GPT-3.5 (81.3%) by 1.4 percentage points -- the first zeroshot system to surpass fine-tuned performance on this benchmark. On SecurityEval, the same architecture achieves 91.7% detection rate, matching specialized systems. The recall improvement comes at a precision cost: 48.8% precision versus 63.9% for fine-tuned baselines, yielding 61.4% F1. Ablation experiments isolate component contributions: the multi-agent ensemble adds 17 percentage points recall over single-agent security analysis. These results demonstrate that for security applications where false negatives are costlier than false positives, zero-shot multi-agent ensembles can match and exceed fine-tuned models on the metric that matters most.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。