arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02641cs.CRcs.AI
收藏

对零样本和少样本钓鱼网址检测的大型语言模型进行基准测试

Benchmarking Large Language Models for Zero-shot and Few-shot Phishing URL Detection

University of North Carolina at Pembroke（北卡罗来纳大学帕克分校）

Najmul Hasan, Prashanth BusiReddyGari

备注：
9 pages, accepted at the 39th Conference on Neural Information Processing Systems (NeurIPS 2025 LAW Workshop)
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文通过统一的零样本和少样本提示框架，评估了大型语言模型在钓鱼网址检测中的性能，揭示了不同模型在泛化能力和实用性上的差异。

摘要：

统一资源定位符（URL）是在连接优先的时代引入的，用于定义访问和定位资源，尽管在网络安全时代引入了反应性保护措施如HTTPS，但其历史上仍然有限，缺乏对未来安全、信任或对抗欺诈和滥用的未来保障机制。在当前的AI优先威胁环境中，欺骗性URL由于网络犯罪分子广泛使用生成式AI以及AI对AI的军备竞赛，已经达到了前所未有的复杂性，产生了能够使用户和传统检测工具几乎无法区分的上下文感知钓鱼网站和URL。尽管2024年AI生成的钓鱼攻击只占过滤绕过攻击的小部分，但自2022年以来，钓鱼攻击量已增长了4000%以上，其中近50%的攻击逃避检测。鉴于威胁环境的加速升级，以及钓鱼战术比标记数据的产生速度快，零样本和少样本学习与大型语言模型（LLMs）提供了一个及时且适应性强的解决方案，使在最小监督下实现泛化。鉴于在大规模网络安全防御系统中，钓鱼URL检测的重要性，我们提出了一个综合的LLM基准测试，基于统一的零样本和少样本提示框架，并揭示了操作上的权衡。我们的评估使用了一个平衡的数据集和一致的提示，提供了性能、泛化能力和模型效用的详细分析，通过准确率、精确率、召回率、F1分数、AUROC和AUPRC量化，以反映分类质量和实际威胁检测设置中的实用性。我们得出结论，少样本提示在多个LLM上提升了性能。

英文摘要：

The Uniform Resource Locator (URL), introduced in a connectivity-first era to define access and locate resources, remains historically limited, lacking future-proof mechanisms for security, trust, or resilience against fraud and abuse, despite the introduction of reactive protections like HTTPS during the cybersecurity era. In the current AI-first threatscape, deceptive URLs have reached unprecedented sophistication due to the widespread use of generative AI by cybercriminals and the AI-vs-AI arms race to produce context-aware phishing websites and URLs that are virtually indistinguishable to both users and traditional detection tools. Although AI-generated phishing accounted for a small fraction of filter-bypassing attacks in 2024, phishing volume has escalated over 4,000% since 2022, with nearly 50% more attacks evading detection. At the rate the threatscape is escalating, and phishing tactics are emerging faster than labeled data can be produced, zero-shot and few-shot learning with large language models (LLMs) offers a timely and adaptable solution, enabling generalization with minimal supervision. Given the critical importance of phishing URL detection in large-scale cybersecurity defense systems, we present a comprehensive benchmark of LLMs under a unified zero-shot and few-shot prompting framework and reveal operational trade-offs. Our evaluation uses a balanced dataset with consistent prompts, offering detailed analysis of performance, generalization, and model efficacy, quantified by accuracy, precision, recall, F1 score, AUROC, and AUPRC, to reflect both classification quality and practical utility in threat detection settings. We conclude few-shot prompting improves performance across multiple LLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑