Large Language Models have become integral to software development, yet they frequently generate vulnerable code. Existing code vulnerability detection benchmarks employ binary classification, lacking the CWE-level specificity required for actionable feedback in iterative correction systems. We present ALPHA (Adaptive Learning via Penalty in Hierarchical Assessment), the first function-level Python benchmark that evaluates both LLMs and SAST tools using hierarchically aware, CWE-specific penalties. ALPHA distinguishes between over-generalisation, over-specification, and lateral errors, reflecting practical differences in diagnostic utility. Evaluating seven LLMs and two SAST tools, we find LLMs substantially outperform SAST, though SAST demonstrates higher precision when detections occur. Critically, prediction consistency varies dramatically across models (8.26%-81.87% agreement), with significant implications for feedback-driven systems. We further outline a pathway for future work incorporating ALPHA penalties into supervised fine-tuning, which could provide principled hierarchy-aware vulnerability detection pending empirical validation.
Case Study: Fine-tuning Small Language Models for Accurate and Private CWE Detection in Python Code
Institute of Information and Communication Technology, Bangladesh University of Engineering Technology（孟加拉工程科技大学信息与通信技术研究所）；Hajee Mohammad Danesh Science and Technology University（海杰莫哈默德丹什科学与技术大学）
Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection
Singapore Management University（新加坡管理大学）；Hanoi University of Science and Technology（河内理工大学）；The University of Melbourne（墨尔本大学）；University of Melbourne（墨尔本大学）
An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
Department of Computer Science, Birzeit University（计算机科学系，巴勒斯坦比泽大学）；King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学）；University of Central Florida（中央佛罗里达大学）
SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI
UC Santa Barbara（加州大学圣巴巴拉分校）；UC Berkeley（加州大学伯克利分校）；VirtueAI；UIUC（伊利诺伊大学香槟分校）；UChicago（芝加哥大学）；University of Oxford（牛津大学）；UK AI Safety Institute（英国人工智能安全研究所）
Unsaid, Unsafe? Implicit Security Obligations in LLM-Based RTL Code Generation
Zhejiang University（浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究所）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机学院）
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
Sun Yat-sen University（中山大学）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）
