自主开源软件威胁检测：基于分类对齐的大语言模型
Autonomous OSS Threat Detection via Taxonomy-Aligned LLMs
查看机构详情
- BRAC University（布拉格大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文提出基于分类对齐提示的GPT-4框架，在999个真实开源供应链事件上实现97%分类准确率，证明提示对齐优于模型规模与微调。
中文摘要 AI 辅助
开源软件（OSS）生态系统面临来自复杂供应链攻击的日益增长的威胁，包括域名仿冒（typosquatting）、依赖混淆（dependency confusion）、特洛伊源混淆（Trojan Source obfuscation）、恶意构建注入（malicious build injection）以及CI/CD流水线投毒（CI/CD pipeline poisoning）。现有检测方法依赖于基于签名和基于规则的系统，这些系统难以泛化到攻击变体和新兴威胁模式。本文提出了一种基于分类对齐的大语言模型框架，用于自动检测和分类OSS供应链威胁。我们引入了一个结构化的AV-xxx威胁分类体系，涵盖五类攻击，并构建了一个包含999个经过验证的真实世界OSS供应链事件的数据集，这些事件来源于GitHub安全公告、CISA警报以及2018年至2026年的安全研究报告。通过使用GPT-4进行基于分类对齐的提示工程，我们的框架在所有五类威胁上实现了97.0%的多类分类准确率和97.0%的宏F1分数。与五个传统机器学习基线、一个零样本开源大语言模型和两个微调神经模型的对比评估揭示了一个令人惊讶的发现：微调的Llama 3.1 8B（70.5%）和SecRoBERTa（77.5%）均不如简单的TF-IDF分类器（82.3%），而未进行分类对齐的Mistral 7B仅达到65.7%。这些结果证实，分类对齐的提示而非模型规模、领域预训练或微调是实现高分类准确率的关键因素。我们的数据集和代码已公开，以支持可复现的供应链安全研究。
英文摘要
Open source software (OSS) ecosystems face growing threats from sophisticated supply chain attacks including typosquatting, dependency confusion, Trojan Source obfuscation, malicious build injection, and CI/CD pipeline poisoning. Existing detection approaches rely on signature-based tools and rule-based systems that struggle to generalize across attack variants and emerging threat patterns. In this paper we propose a taxonomy-aligned large language model framework for automated detection and classification of OSS supply chain threats. We introduce a structured AV-xxx threat taxonomy covering five attack categories and construct a curated dataset of 999 verified real-world OSS supply chain incidents sourced from GitHub Security Advisories, CISA alerts, and security research reports spanning 2018 to 2026. Using taxonomy-aligned prompt engineering with GPT-4, our framework achieves 97.0\% multi-class classification accuracy and 97.0\% macro F1 score across all five threat categories. Comparative evaluation against five traditional machine learning baselines, one zero-shot open source LLM, and two fine-tuned neural models reveals a surprising finding: fine-tuned Llama 3.1 8B (70.5%) and SecRoBERTa (77.5%) both underperform simple TF-IDF classifiers (82.3%), while Mistral 7B without taxonomy alignment achieves only 65.7%. These results confirm that taxonomy-aligned prompting rather than model scale, domain pretraining, or fine-tuning is the critical factor enabling high classification accuracy. Our dataset and code are publicly available to support reproducible supply chain security research.
