EvalSVA：面向下一代软件漏洞评估的多智能体评估器
EvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
AI总结：
EvalSVA提出多智能体评估器框架，利用多个大语言模型协作，在有限数据下提升软件漏洞评估的准确率和F1分数，并构建多语言数据集以支持评估。
摘要：
软件漏洞（SV）评估是确定漏洞不同方面（如攻击向量和范围）的关键过程，帮助开发者有效优先安排漏洞缓解工作。由于漏洞的复杂性和标记数据的稀缺性，这一过程具有挑战性且劳动密集。为缓解上述挑战，我们引入EvalSVA，一个多智能体评估器团队，自主审议并评估漏洞评估的各个方面。具体而言，我们提出一个基于多智能体的框架，模拟真实场景中的漏洞评估策略，将多个大型语言模型（LLM）集成到一个小组中，以在有限数据下增强漏洞评估的有效性。我们还设计了多样化的通信策略，以自主讨论和评估漏洞的不同方面。此外，我们基于CVSS新标准构建了一个多语言漏洞评估数据集，分别包含699、888和1310个与漏洞相关的提交，涉及C++、Python和Java。实验结果表明，与先前方法相比，EvalSVA在漏洞评估中平均优于44.12%的准确率和43.29%的F1分数。这表明EvalSVA提供了类似人类的过程，并为漏洞评估生成推理和答案。EvalSVA还可以帮助人类专家进行漏洞评估，提供更多解释和细节。
英文摘要：
Software Vulnerability (SV) assessment is a crucial process of determining different aspects of SVs (e.g., attack vectors and scope) for developers to effectively prioritize efforts in vulnerability mitigation. It presents a challenging and laborious process due to the complexity of SVs and the scarcity of labeled data. To mitigate the above challenges, we introduce EvalSVA, a multi-agent evaluators team to autonomously deliberate and evaluate various aspects of SV assessment. Specifically, we propose a multi-agent-based framework to simulate vulnerability assessment strategies in real-world scenarios, which employs multiple Large Language Models (LLMs) into an integrated group to enhance the effectiveness of SV assessment in the limited data. We also design diverse communication strategies to autonomously discuss and assess different aspects of SV. Furthermore, we construct a multi-lingual SV assessment dataset based on the new standard of CVSS, comprising 699, 888, and 1,310 vulnerability-related commits in C++, Python, and Java, respectively. Our experimental results demonstrate that EvalSVA averagely outperforms the 44.12\% accuracy and 43.29\% F1 for SV assessment compared with the previous methods. It shows that EvalSVA offers a human-like process and generates both reason and answer for SV assessment. EvalSVA can also aid human experts in SV assessment, which provides more explanation and details for SV assessment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
