Third-Party Risk Assessment (TPRA) relies on large repositories of cybersecurity compliance questions used to assess external suppliers against standards such as ISO/IEC 27001 and NIST. In practice, not all questions are relevant for a specific supplier and selecting questions for a given assessment context remains a manual and time-consuming task. Existing question retrieval approaches based on lexical or semantic similarity can identify topically related questions, but they often fail to capture the underlying assessment intent, including control domain and evaluation scope.
To address this limitation, we investigate whether an explicit semantic label space can improve intent-aware TPRA question selection. In particular, we separate label space discovery from large-scale label assignment. We start by discovering overlapping clusters of semantically similar questions and then exploit LLMs to assign unique labels for each cluster. Second, we propagate labels through k-nearest neighbors (kNN) for a larger-scale question annotation. Question retrieval is finally achieved by similarity measure of the query with respect to the extracted labels instead of the questions themselves. This reduces repeated LLM calls while preserving label consistency.
Experimental results show that the proposed semi-supervised framework reduces labeling cost and runtime compared with per-question LLM annotation while maintaining label quality and improving efficiency. Furthermore, label-based retrieval achieves better alignment with cybersecurity control domains and assessment scope than similarity-based retrieval, highlighting the value of semantic labels as an intermediate representation.
Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing
Tianjin University(天津大学);Changchun University of Science and Technology(长春理工大学);Jilin Province Key Laboratory of Network and Information Security(吉林省网络与信息安全重点实验室);Shihezi University(石河子大学)
Detection of adversarial intent in Human-AI teams using LLMs
Department of Mechanical Engineering, University of California at Santa Barbara(加州大学圣巴巴拉分校机械工程系);Department of Computer Science, University of California at Santa Barbara(加州大学圣巴巴拉分校计算机科学系)
Human Vulnerability Assessment in Cybersecurity: A Systematic Literature Review of Methods, Models, and Instruments
Hellenic Mediterranean University(希腊地中海大学);National Centre for Scientific Research Demokritos(德谟克利特国家科学研究中心);Industrial Systems Institute of the Athena Research Center(雅典研究中心工业系统研究所)
Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation
Harvard University(哈佛大学);Google DeepMind Canada(谷歌深Mind加拿大);Google DeepMind United States(谷歌深Mind美国);Google DeepMind New York United States(谷歌深Mind纽约美国)
AgentBnB: A Browser-Based Cybersecurity Tabletop Exercise with Large Language Model Support and Retrieval-Aligned Scaffolding
School of Electrical and Computer Engineering(电气与计算机工程学院);Georgia Institute of Technology(佐治亚理工学院);School of Computational Science and Engineering(计算科学与工程学院)
Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices
The University of British Columbia(不列颠哥伦比亚大学);University of Virginia(弗吉尼亚大学);Indian Institute of Technology Madras(马德拉斯印度理工学院);National Research Council Canada(加拿大国家研究委员会)