一种用于求解带无人机旅行商问题的端到端深度强化学习方法
An End-to-End Deep Reinforcement Learning Approach for Solving the Traveling Salesman Problem with Drones
AI总结：
本文提出一种端到端分层Actor-Critic深度强化学习框架，结合Transformer编码器与Minimal Gated Unit解码器，并引入优化的k近邻稀疏注意力机制，在TSP-D基准实例上以更短计算时间和更低训练成本取得优于启发式算法和现有强化学习方法的性能。
摘要：
卡车-无人机协同系统在最后一公里物流中的出现，使带无人机旅行商问题（TSP-D）成为经典路径优化的重要扩展。在该问题中，同步的车辆协调有望带来显著的运营效率提升和环境影响降低，但同时也引入了超出传统优化范式能力范围的NP难组合复杂性。深度强化学习通过自监督策略学习和自适应决策，为解决TSP-D的固有挑战提供了具有理论基础的框架。本研究提出了一种用于求解TSP-D问题的分层Actor-Critic深度强化学习框架。该架构由两个主要部分组成：一个受Transformer启发的编码器和一个高效的Minimal Gated Unit解码器。编码器引入了一种新颖的、经过优化的k近邻稀疏注意力机制，专门用于聚焦相关的空间关系，并通过整合全局节点特征进一步增强。Minimal Gated Unit解码器处理这些编码表示，以高效生成解序列。整个框架在异步优势actor-critic范式下运行。实验结果表明，在不同规模（N=10至100）的基准TSP-D实例上，与高性能启发式算法和现有强化学习方法相比，所提模型能够在更短的平均计算时间内获得具有竞争力甚至更优的解。此外，与先进的强化学习算法基准相比，所提框架在实现更优最终性能的同时，显著减少了所需的总训练时间，凸显了其在训练效率方面的显著优势。
英文摘要：
The emergence of truck-drone collaborative systems in last-mile logistics has positioned the Traveling Salesman Problem with Drones (TSP-D) as a pivotal extension of classical routing optimization, where synchronized vehicle coordination promises substantial operational efficiency and reduced environmental impact, yet introduces NP-hard combinatorial complexity beyond the reach of conventional optimization paradigms. Deep reinforcement learning offers a theoretically grounded framework to address TSP-D's inherent challenges through self-supervised policy learning and adaptive decision-making. This study proposes a hierarchical Actor-Critic deep reinforcement learning framework for solving the TSP-D problem. The architecture consists of two primary components: a Transformer-inspired encoder and an efficient Minimal Gated Unit decoder. The encoder incorporates a novel, optimized k-nearest neighbors sparse attention mechanism specifically for focusing on relevant spatial relationships, further enhanced by the integration of global node features. The Minimal Gated Unit decoder processes these encoded representations to efficiently generate solution sequences. The entire framework operates within an asynchronous advantage actor-critic paradigm. Experimental results show that, on benchmark TSP-D instances of various scales (N=10 to 100), the proposed model can obtain competitive or even superior solutions in shorter average computation times compared to high-performance heuristic algorithms and existing reinforcement learning methods. Moreover, compared to advanced reinforcement learning algorithm benchmarks, the proposed framework significantly reduces the total training time required while achieving superior final performance, highlighting its notable advantage in training efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
