SparSEEty：通过确定性侧信道从利用稀疏性的大语言模型（LLM）服务系统中提取 token
SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels
浏览论文内容
中文总结 AI 辅助
SparSEEty 是一种针对利用稀疏性的 LLM 服务系统的 token 提取攻击，通过 CVM 侧信道构建预言机、降低监控开销并反转激活轨迹，可高准确率重建 token，监控开销仅 3.7%-7.2%。
中文摘要 AI 辅助
现代大语言模型（LLM）存在激活稀疏性，即对于给定输入 token，仅激活其部分神经元。研究人员利用该特性，通过省略与非激活神经元相关的权重访问和计算来优化 LLM 服务系统。但遗憾的是，此类优化会产生依赖输入的权重访问，可通过侧信道泄露。我们提出 SparSEEty，这是一种新型 token 提取攻击，利用利用稀疏性的 LLM 服务系统引入的依赖输入的神经元权重访问。SparSEEty 首先在 LLM 推理期间，利用神经元权重访问侧信道构建神经元激活预言机，随后将激活轨迹反转以重建输入 token，形成端到端的 token 提取攻击。我们针对在 Intel TDX 机密虚拟机（CVM）内部受保护的 LLM 服务系统实例化 SparSEEty，解决三大关键挑战：（i）利用 CVM 暴露的侧信道组合构建神经元激活预言机；（ii）降低神经元激活监控的推理时间开销以保证隐蔽性；（iii）将部分二元激活轨迹准确反转回 token。我们的评估表明，SparSEEty 可在各类模型和数据集上以始终较高的 BLEU 分数（>0.95）重建提示和响应 token，同时带来 3.7%至 7.2%的监控开销。
英文摘要
Modern large language models (LLMs) exhibit activation sparsity, wherein only a subset of their neurons is activated for given input tokens. Researchers have leveraged this property to optimize LLM serving systems by omitting weight accesses and computations pertaining to inactive neurons. Unfortunately, however, such optimizations create input-dependent weight accesses, which can be leaked over side channels. We present SparSEEty, a new token extraction attack that exploits input-dependent neuron weight accesses introduced by sparsity-exploiting LLM serving systems. SparSEEty first constructs a neuron-activation oracle using neuron weight access side channels during LLM inference, and then inverts the activation traces to reconstruct the input tokens, forming an end-to-end token extraction attack. We instantiate SparSEEty against an LLM serving system protected inside an Intel TDX confidential virtual machine (CVM), addressing three key challenges: (i) constructing a neuron-activation oracle using a combination of side channels exposed by CVMs, (ii) reducing inference-time overheads of neuron activation monitoring for covertness, and (iii) accurately inverting partial binary activation traces back to tokens. Our evaluation shows that SparSEEty can reconstruct both prompt and response tokens with consistently high BLEU scores (>0.95) across various models and datasets, while incurring monitoring overheads of 3.7% to 7.2%.
发表机构
- Yonsei University（延世大学）
机构由 AI 辅助整理，请以论文原文为准。
