基于HJB启发有限动作风险滤波的保持自动驾驶仪的残差Q学习用于固定翼无人机指令监督
Autopilot-Preserving Residual Q-Learning with HJB-Inspired Finite-Action Risk Filtering for Fixed-Wing UAV Command Supervision
AI总结:
提出一种保持自动驾驶仪的残差指令监督框架,通过HJB方程启发的半离散值迭代评价器和控制Lyapunov/屏障函数启发的有限动作屏蔽,选择有限有界动作集中的残差,显著降低路径跟踪误差。
摘要:
固定翼无人机必须在风、阵风和湍流下保持空速、高度和航向参考,这些通道耦合使得纠正一个通道可能恶化另一个。经典自动驾驶仪能很好地稳定机身,但在强侧风遇到激进转弯时适应能力差,而直接作用于舵面的强化学习策略将探索风险集中在执行器接口。我们在未改变的自动驾驶仪之上放置一个学习型监督器,而不是在其内部:它从指令空速、高度和航向的有限有界动作集中选择一个残差;修改后的参考在到达自动驾驶仪之前被投影到允许的指令包络内,自动驾驶仪仍然是唯一面向执行器的控制器。新颖之处在于残差的选择方式。HJB残差使用半离散值迭代评价器(基于Hamilton-Jacobi-Bellman方程精神)对候选动作评分,通过无操作相对哈密顿优势排序,并通过控制Lyapunov函数和控制屏障函数启发的有限动作屏蔽进行过滤,该屏蔽始终保留无操作回退。在共享的12状态运行时(固定植物、自动驾驶仪和执行器模型)上,HJB残差将均方根路径跟踪误差降低到44.809米,而基线自动驾驶仪为338.617米,表格Q残差为88.809米,相比基线降低86.77%,相比Q学习降低49.54%。增益集中在基线表现最差的区域,但伴随空速误差的测量上升,因此没有方法在所有指标上占优。我们呈现这种保持自动驾驶仪的残差指令监督设计,并完整报告其权衡基准。
英文摘要:
A fixed-wing UAV must hold airspeed, altitude, and heading references under wind, gusts, and turbulence, channels coupled so that correcting one can degrade another. Classical autopilots stabilize the airframe well but adapt poorly when a hard crosswind meets an aggressive turn, while reinforcement-learning (RL) policies acting directly on the surfaces concentrate exploration risk at the actuator interface. We place a learned supervisor above an unchanged autopilot rather than inside it: it selects a residual from a finite, bounded action set on the commanded airspeed, altitude, and heading; the modified reference is projected into an admissible command envelope before reaching the autopilot, which stays the only actuator-facing controller. What is new is how the residual is chosen. HJB residual scores candidates with a semi-discrete value-iteration critic in the spirit of the Hamilton-Jacobi-Bellman (HJB) equation, ranks them by a no-op-relative Hamiltonian advantage, and filters them through a control-Lyapunov- and control-barrier-inspired finite-action shield that always keeps a no-op fallback. On a shared 12-state runtime holding the plant, autopilot, and actuator model fixed, so the comparison is at the package level, HJB residual lowers mean RMS path-tracking error to 44.809 m, against 338.617 m for the baseline autopilot and 88.809 m for a tabular-Q residual, an 86.77% reduction over the baseline and 49.54% over Q-learning. The gain concentrates where the baseline fails worst and comes with a measured rise in airspeed error, so no method dominates every metric. We present this autopilot-preserving residual command-supervision design and benchmark with its trade-offs reported intact.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。