arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.18062cs.LGcs.AI
收藏

突破障碍:增强平滑深度强化学习智能体的效用与鲁棒性

Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents

UC San Diego(加州大学圣地亚哥分校)

Chung-En Sun, Sicun Gao, Tsui-Wei Weng

AI总结:

针对平滑DRL智能体清洁奖励低、鲁棒性弱的问题,提出S-DQN和S-PPO算法,显著提升效用与鲁棒性,并引入更有效的平滑攻击。

摘要:

鲁棒性在深度强化学习(DRL)中仍然是一个至关重要的问题,随机平滑已成为增强这一属性的关键技术。然而,当前平滑DRL智能体的性能存在显著差距,通常表现为清洁奖励极低且鲁棒性较弱。针对这一挑战,我们的研究引入了创新算法,旨在训练有效的平滑鲁棒DRL智能体。我们提出了S-DQN和S-PPO,这两种新方法在标准RL基准上展示了清洁奖励、经验鲁棒性和鲁棒性保证方面的显著改进。值得注意的是,我们的S-DQN和S-PPO智能体在最强攻击下不仅平均比现有平滑智能体高出$2.16\ imes$,而且还平均比先前鲁棒训练的智能体高出$2.13\ imes$。这代表了该领域的重大飞跃。此外,我们引入了平滑攻击(Smoothed Attack),在降低平滑智能体奖励方面比现有对抗攻击有效$1.89\ imes$。

英文摘要:

Robustness remains a paramount concern in deep reinforcement learning (DRL), with randomized smoothing emerging as a key technique for enhancing this attribute. However, a notable gap exists in the performance of current smoothed DRL agents, often characterized by significantly low clean rewards and weak robustness. In response to this challenge, our study introduces innovative algorithms aimed at training effective smoothed robust DRL agents. We propose S-DQN and S-PPO, novel approaches that demonstrate remarkable improvements in clean rewards, empirical robustness, and robustness guarantee across standard RL benchmarks. Notably, our S-DQN and S-PPO agents not only significantly outperform existing smoothed agents by an average factor of $2.16\times$ under the strongest attack, but also surpass previous robustly-trained agents by an average factor of $2.13\times$. This represents a significant leap forward in the field. Furthermore, we introduce Smoothed Attack, which is $1.89\times$ more effective in decreasing the rewards of smoothed agents than existing adversarial attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑