发表机构
North Carolina State University（北卡罗来纳州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
TranScope首次通过微架构级分析揭示LLM训练数据影响执行足迹，提出低成本高鲁棒性的成员推断工具，AUC达0.9。
AI 中文摘要
成员关系是机器学习中最基本的隐私原语：迄今为止，尚未有基于硬件的黑盒模型分布外检测方法能够针对具有掩蔽置信度的常数时间、静态神经网络进行演示。本文首次在周期级别上考察了大型语言模型和视觉变换器如何与各种现代微架构组件（包括集成加速器）交互，随着LLM规模的增大，并回答了即使在没有输入依赖分支、动态优化或提前退出以及常数时间模型中，模型训练所用的数据是否会影响其执行足迹的问题。结果证实答案是肯定的，并识别出哪些现代硬件组件（如TLB或片上加速器）会揭示或放大这种效应。结果还回答了该信号是否具有足够的信息量来可靠地分类成员关系的分布内/分布外属性。为了理解原因，我们进行了系统的根因分析，发现变换器的分词步骤（在训练期间发生）改变了模型在推理期间获取词汇标记时访问的局部性，从而以一种先前未知的数据依赖方式改变了页表访问模式和TLB，导致微架构状态根据输入是否在变换器训练数据的分布内而显著变化。基于上述观察，我们引入了TranScope：第一个用于检测成员信息的微架构工具，具有低成本、无需替代模型、且显著更高的鲁棒性，例如，PETAL的最佳先前报告AUC为0.6，而我们的AUC为0.9。这重新将硬件引入为一种机会（例如，首次用于检查版权侵权的工具）和一种推断成员关系（MIA）的新渠道。
英文摘要
Membership is the root privacy primitive in machine learning: to date, no hardware-based out-of-distribution detection on black-box models has been demonstrated against constant-time, static neural networks with masked confidence. This paper performs the first cycle-level examination of how large language models and vision transformers interact with various modern microarchitecture components, including integrated accelerators, as LLMs scale in size and answers the question of whether the data that a model was trained on affects its execution footprint even without any input-dependent branch, dynamic optimization, or early exit and in constant-time models. The results confirm that the answer is yes and identify which modern hardware components, such as TLBs or on-core accelerators, reveal or amplify that effect. The results also answer whether the signal is informative enough to reliably classify the in-/vs/out-of-distribution property of membership. To understand why, we perform a systematic root cause analysis and find that the transformer's tokenization steps, which happen during training, alter the locality of the accesses the model makes to fetch the vocabulary token later during inference and, as a result, change the page table access patterns and TLB in a previously unknown data-dependent way, causing microarchitectural state to vary significantly based on whether or not the input was in the distribution of the transformer training data. Building on the above observation, we introduce TranScope: the first microarchitecture tool for detecting membership information with low cost, no need for a surrogate model, and significantly higher robustness, e.g., 0.6 AUC for PETAL (best previously reported) vs 0.9 AUC (ours). This reintroduces hardware as both an opportunity, e.g., a tool for checking copyright violation for the first time, and a new channel for inferring membership (MIA).
