学习面向在线3D箱装问题的实际可行策略
Learning Practically Feasible Policies for Online 3D Bin Packing
AI总结：
针对在线3D箱装问题，本文利用同策略actor-critic强化学习，并结合堆叠树稳定性分析、解耦放置策略和由远及近奖励设计，学习出显著优于现有方法且可实际部署的装箱策略。
摘要：
我们研究在线3D箱装问题（Online 3D Bin Packing Problem），这是经典箱装问题中一个具有挑战性且具有实际应用价值的变体。在该问题中，物品被递交给智能体，但不会告知完整序列信息。智能体必须在不改变物品到达顺序的情况下，将这些物品直接稳定地装入目标箱体，且不允许后续调整。在线3D-BPP可以自然地表述为马尔可夫决策过程（MDP）。我们采用深度强化学习，特别是同策略actor-critic框架，来求解这个具有受约束动作空间的MDP。为了学习实际可行的装箱策略，我们提出了三个关键设计。首先，我们基于一种新颖的堆叠树提出了在线装箱稳定性分析方法。该方法实现了较高的分析准确率，同时将计算复杂度从O(N^2)降低至O(N log N)，因此特别适合强化学习训练。其次，我们提出了面向放置不同维度的解耦装箱策略学习，使高分辨率空间离散化成为可能，从而实现较高的装箱精度。第三，我们引入了一种奖励函数，指示机器人按照由远及近的顺序放置物品，从而简化机械臂运动规划中的避障问题。此外，我们还对若干关键实现问题进行了全面讨论。大量评估表明，我们学习得到的策略显著优于最先进方法，并且可实际用于真实世界应用。
英文摘要：
We tackle the Online 3D Bin Packing Problem, a challenging yet practically useful variant of the classical Bin Packing Problem. In this problem, the items are delivered to the agent without informing the full sequence information. Agent must directly pack these items into the target bin stably without changing their arrival order, and no further adjustment is permitted. Online 3D-BPP can be naturally formulated as Markov Decision Process (MDP). We adopt deep reinforcement learning, in particular, the on-policy actor-critic framework, to solve this MDP with constrained action space. To learn a practically feasible packing policy, we propose three critical designs. First, we propose an online analysis of packing stability based on a novel stacking tree. It attains a high analysis accuracy while reducing the computational complexity from $O(N^2)$ to $O(N \log N)$, making it especially suited for RL training. Second, we propose a decoupled packing policy learning for different dimensions of placement which enables high-resolution spatial discretization and hence high packing precision. Third, we introduce a reward function that dictates the robot to place items in a far-to-near order and therefore simplifies the collision avoidance in movement planning of the robotic arm. Furthermore, we provide a comprehensive discussion on several key implemental issues. The extensive evaluation demonstrates that our learned policy outperforms the state-of-the-art methods significantly and is practically usable for real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
