arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.19582cs.LG
收藏

基于优势的时序攻击在强化学习中

Advantage-based Temporal Attack in Reinforcement Learning

School of Computer Science and Engineering, Sun Yat-sen University(计算机科学与工程学院,中山大学);Nanyang Technological University(南洋理工大学)

Shenghong He

AI总结:

本文提出基于优势的对抗变压器(AAT),通过多尺度因果自注意力机制和加权优势机制,生成具有更强时间相关性的对抗示例,提升强化学习中的攻击性能。

摘要:

大量研究显示,深度强化学习(DRL)模型容易受到对抗性构造的输入(即对抗性示例)的影响,这些输入可以误导智能体采取次优或危险的行为。最近的方法通过利用未来奖励来引导对抗扰动生成过程中的时间步(即基于奖励的攻击)来提高攻击效果。然而,这些方法无法捕捉扰动生成过程中不同时间步之间的依赖关系,导致当前扰动与先前扰动之间的时间相关性较弱。在本文中,我们提出了一种新的方法,称为基于优势的对抗变压器(AAT),能够生成具有更强时间相关性的对抗示例(即时间相关的对抗示例)以提高攻击性能。AAT采用多尺度因果自注意力(MSCSA)机制,动态捕捉不同时间点的历史信息与当前状态之间的依赖关系,从而增强当前扰动与先前扰动之间的相关性。此外,AAT引入了加权优势机制,该机制量化了扰动在给定状态下的有效性,并通过采样高优势区域引导生成过程朝着高绩效的对抗示例发展。广泛的实验表明,AAT在Atari、DeepMind控制套件和Google足球任务上,其性能与主流对抗攻击基线相匹配或超越。

英文摘要:

Extensive research demonstrates that Deep Reinforcement Learning (DRL) models are susceptible to adversarially constructed inputs (i.e., adversarial examples), which can mislead the agent to take suboptimal or unsafe actions. Recent methods improve attack effectiveness by leveraging future rewards to guide adversarial perturbation generation over sequential time steps (i.e., reward-based attacks). However, these methods are unable to capture dependencies between different time steps in the perturbation generation process, resulting in a weak temporal correlation between the current perturbation and previous perturbations.In this paper, we propose a novel method called Advantage-based Adversarial Transformer (AAT), which can generate adversarial examples with stronger temporal correlations (i.e., time-correlated adversarial examples) to improve the attack performance. AAT employs a multi-scale causal self-attention (MSCSA) mechanism to dynamically capture dependencies between historical information from different time periods and the current state, thus enhancing the correlation between the current perturbation and the previous perturbation. Moreover, AAT introduces a weighted advantage mechanism, which quantifies the effectiveness of a perturbation in a given state and guides the generation process toward high-performance adversarial examples by sampling high-advantage regions. Extensive experiments demonstrate that the performance of AAT matches or surpasses mainstream adversarial attack baselines on Atari, DeepMind Control Suite and Google football tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑