基于信号的人工智能系统运营安全模型访问风险分析

中文摘要 AI 辅助

人工智能系统如今在安全、金融、医疗、消费技术和大规模云服务等领域无处不在，它们每天处理大量数据并做出重要决策。这种广泛采用带来了广泛的攻击面，对手可通过多种方式攻击已部署模型。以往的分类法常将逃避攻击分为白盒、灰盒和黑盒，但未考虑不同部署场景下的输出信号差异。为此，我们引入基于信号的模型访问风险分类法（SMART），这是一个面向部署的框架，根据已部署人工智能系统的信息信号性质和丰富程度对攻击者访问进行分类。利用该分类法，我们对不同信息暴露水平的逃避攻击进行了结构化概述，突出部署接口如何影响攻击能力，为更安全的人工智能部署和采购决策提供参考。

英文摘要

Artificial intelligence (AI) systems are now ubiquitous across domains such as security, finance, healthcare, consumer technology, and large-scale cloud services, where they process massive volumes of data and make consequential decisions daily. This widespread adoption has created a broad attack surface through which adversaries can manipulate, evade, extract information from, or otherwise subvert deployed models. Depending on system design and exposure, attackers may have very different forms of access: some observe only final decisions, while others receive confidence scores, intermediate representations, or even full model parameters. While previous surveys typically organize evasion attacks into white-box, gray-box, and black-box categories based on the attacker's knowledge of model internals (architecture, parameters, gradients), this taxonomy often conflates different deployment scenarios that provide vastly different output signals, all labeled as ``black-box'' despite enabling fundamentally different attack strategies. Understanding how evasion attack strategies adapt to the specific information signals returned by deployed systems is critical for organizations making procurement and deployment decisions. To address this gap, we introduce the Signal-based Model Access Risk Taxonomy (SMART), a deployment-oriented framework that classifies attacker access according to the nature and richness of the information signals available from deployed AI systems. Using this taxonomy, we provide a structured overview of evasion attacks across progressively richer levels of information exposure, highlighting how deployment interfaces influence attack capabilities and informing more secure AI deployment and procurement decisions.

发表机构

• Oak Ridge National Laboratory（奥克兰国家实验室）
• Oak Ridge Institute for Science and Education（奥克兰科学与教育研究所）
• DHS Science and Technology Directorate（国土安全科学技术局）

机构由 AI 辅助整理，请以论文原文为准。