FLARE：利用通用对抗掩码对深度强化学习智能体进行指纹识别
FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks
AI总结：
本文提出FLARE，利用不可迁移的通用对抗掩码作为指纹，通过动作一致性验证深度强化学习策略的所有权，实验表明其能有效识别盗版副本且无误报，并对模型修改攻击具有鲁棒性。
摘要：
我们提出FLARE，这是首个指纹识别机制，用于验证一个可疑的深度强化学习（DRL）策略是否是另一个（受害者）策略的非法副本。我们首先证明，可以找到不可迁移的通用对抗掩码，即扰动，以生成对抗样本，这些样本能够成功地从受害者策略迁移到其修改版本，但不能迁移到独立训练的策略。FLARE利用这些掩码作为指纹，通过测量在此类掩码扰动下的状态上的动作一致值来验证被盗DRL策略的真实所有权。我们的实证评估表明，FLARE是有效的（在被盗副本上达到100%的动作一致性），并且不会错误地指控独立策略（无误报）。FLARE对模型修改攻击也具有鲁棒性，并且更了解情况的对手在不负面影响智能体性能的情况下难以轻易规避它。我们还表明，由于DRL策略的固有特性，并非所有通用对抗掩码都适合作为指纹候选。DRL问题的时空动态特性和序贯决策过程使得刻画DRL策略的决策边界更加困难，也使搜索能够捕捉其几何结构的通用掩码更加困难。
英文摘要：
We propose FLARE, the first fingerprinting mechanism to verify whether a suspected Deep Reinforcement Learning (DRL) policy is an illegitimate copy of another (victim) policy. We first show that it is possible to find non-transferable, universal adversarial masks, i.e., perturbations, to generate adversarial examples that can successfully transfer from a victim policy to its modified versions but not to independently trained policies. FLARE employs these masks as fingerprints to verify the true ownership of stolen DRL policies by measuring an action agreement value over states perturbed by such masks. Our empirical evaluations show that FLARE is effective (100% action agreement on stolen copies) and does not falsely accuse independent policies (no false positives). FLARE is also robust to model modification attacks and cannot be easily evaded by more informed adversaries without negatively impacting agent performance. We also show that not all universal adversarial masks are suitable candidates for fingerprints due to the inherent characteristics of DRL policies. The spatio-temporal dynamics of DRL problems and sequential decision-making process make characterizing the decision boundary of DRL policies more difficult, as well as searching for universal masks that capture the geometry of it.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
