发表机构
University of Kentucky; Washington University in St. Louis（肯塔基大学; 圣路易斯华盛顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对具身强化学习分布式训练，提出TRACE时间梯度反演攻击，利用跨时间相关性与闭式动作恢复，自回归重建私有轨迹，实现高PSNR与近完美动作恢复，且速度远超基线。
AI 中文摘要
具身强化学习智能体中的分布式学习通过在设备上保留原始传感器数据并仅向服务器传输策略梯度，提供了一定程度的隐私保护。然而，时间结构可以将这种泄漏放大到超出单帧攻击的程度。我们引入了对连续编码的时间重建攻击（TRACE），这是一种摊销的时间梯度反演攻击，能够从每步策略学习梯度中自回归地重建私有观察-动作轨迹序列。该攻击利用了先前单帧方法忽略的两个结构信号：（i）连续具身梯度之间的跨时间相关性，我们通过条件互信息界对其进行了形式化；（ii）从策略头梯度结构中进行的闭式动作恢复，我们证明了当标准熵正则化足够小时该恢复是精确的。在保留的具身场景中，TRACE达到了18.8 dB的PSNR，在每重建帧3-4.5毫秒内实现了近乎完美的动作恢复，在所有重建指标上优于基于学习的基线，并且比优化攻击快数个数量级。进一步的评估表明，TRACE在循环、残差和紧凑型Transformer受害者架构、多模态输入以及更大的离散动作空间上具有更广泛的适用性。防御实验表明，保护时间梯度流可能需要序列感知的隐私机制。
英文摘要
Distributed learning in embodied reinforcement-learning agents offers a degree of privacy by retaining raw sensor data on-device and transmitting only policy gradients to the server. Yet temporal structure can amplify this leakage beyond single-frame attacks. We introduce Temporal Reconstruction Attack on Consecutive Encodings (TRACE), an amortized temporal gradient-inversion attack that autoregressively reconstructs the sequence of private observation-action trajectories from per-step policy-learning gradients. The attack exploits two structural signals ignored by prior single-frame methods: (i) cross-time correlation between successive embodied gradients, which we formalize via a conditional mutual-information bound, and (ii) closed-form action recovery from policy-head gradient structure, which we prove exact when standard entropy regularization is sufficiently small. On held-out embodied scenes, TRACE reaches $18.8$ dB PSNR with near-perfect action recovery at $3$-$4.5$ ms per reconstructed frame, dominating the learning-based baseline across all reconstruction metrics and exceeding optimization attacks while running orders of magnitude faster. Further evaluation demonstrates TRACE's broader applicability across recurrent, residual, and compact transformer victim architectures, multi-modal inputs, and larger discrete action spaces. Defense experiments suggest that protecting temporal gradient streams may require sequence-aware privacy mechanisms.
CommentsAccepted at NeurIPS 2026
