ASGARD：基于强化学习的无人机韧性动作空间防护

中文摘要 AI 辅助

强化学习（RL）控制器最近已被用于无人驾驶飞行器（UAV）的导航与控制。然而，它们容易受到动作空间攻击的影响，这种攻击会在策略生成动作命令之后、执行器执行之前覆盖动作命令。虽然大多数现有防御措施针对的是对策略输入的攻�击，但那些处理动作空间攻击的防御措施在训练时重新训练策略，并且无法在运行时对损坏的动作具有韧性。我们提出ASGARD，一个两阶段师生管道，用于使基于RL的UAV控制对动作空间攻击具有韧性。在教师阶段，编码器将UAV的物理状态与动作攻击相关的特权信息相结合，产生一个动作攻击感知的潜在表示，用于训练RL控制策略和一个监视器，该监视器向执行器输出修正后的动作命令。在学生阶段，编码器和监视器均通过其教师对应物的监督学习进行训练，以便仅使用UAV的物理状态历史在机载上运行。我们在针对UAV上不同动作命令的攻击场景中评估ASGARD。我们发现ASGARD对动作空间攻击具有韧性，并且在攻击下仍能完成任务。我们进一步发现ASGARD能够泛化到未见过的攻击，并对隐蔽攻击保持韧性。

英文摘要

Reinforcement learning (RL) controllers have been recently adopted for Unmanned Aerial Vehicles (UAV) navigation and control. However, they are susceptible to action-space attacks that overwrite the action commands after the policy generates them and before the actuators execute them. While most existing defenses target attacks on the policy's inputs, those addressing action-space attacks retrain the policy at training time and are not resilient to corrupted actions at runtime. We propose ASGARD, a two-phase teacher-student pipeline for making RL-based UAV control resilient to action-space attacks. In the teacher phase, an encoder combines the UAV's physical state with action-attack-related privileged information to produce an action-attack-aware latent that trains the RL control policy and a monitor that outputs corrected action commands to the actuators. In the student phase, both the encoder and the monitor are trained via supervised learning from their teacher counterparts to run on-board using only the UAV's physical state history. We evaluate ASGARD across attack scenarios targeting different action commands on UAV. We find that ASGARD is resilient to action-space attacks and completes the missions despite the attack. We further find that ASGARD generalizes to unseen attacks and remains resilient against stealthy attacks.

发表机构

• The University of British Columbia（不列颠哥伦比亚大学）

机构由 AI 辅助整理，请以论文原文为准。