仓库SLAM吞吐量控制的离线强化学习
Offline Reinforcement Learning for Warehouse SLAM Throughput Control
AI总结：
提出离线强化学习框架优化仓库SLAM吞吐量控制，通过历史状态表示、动作空间抽象和奖励函数设计，结合多种离线RL算法，实验表明CQL策略提升系统健康度22.97%，减少平均节流时长3.18%。
摘要：
我们提出了一个离线强化学习（RL）框架，用于优化仓库履约环境中的SLAM吞吐量控制。SLAM（扫描/贴标/应用/清单）吞吐量直接影响系统拥堵和运营效率。我们的基于RL的控制方法动态推荐SLAM吞吐量设置，通过智能调整节流行为，自适应地平衡吞吐量最大化与下游稳定性。我们包含一个历史信息状态表示、用于延迟影响控制的动作空间抽象，以及一个捕获上游和下游运营指标的奖励函数。我们的方法是算法无关的，能够在统一架构下集成多种离线RL方法。我们使用三种最先进的离线RL算法实例化框架，并使用来自大规模仓库的去标识化历史操作日志离线训练模型。使用全面的多方法策略评估策略性能，包括无模型方法（通过回归模型进行即时奖励估计和长视界Fitted Q Evaluation（FQE））以及基于模型的Deep Koopman动力学评估。实证结果表明，CQL策略始终优于其他方案，将系统健康度提高22.97%，并将平均节流时长减少3.18%。这些发现证明了离线RL在安全且可扩展的仓库吞吐量控制优化中的潜力。
英文摘要：
We present an offline reinforcement learning (RL) framework for optimizing SLAM throughput control in a warehouse fulfillment environment. SLAM (Scan/Label/Apply/Manifest) throughput directly influences system congestion and operational efficiency. Our RL-based control approach dynamically recommends SLAM throughput settings that adaptively balance throughput maximization with downstream stability through intelligent adjustment of throttling behavior. We include a history-informed state representation, action space abstraction for delayed-impact control, and a reward function that captures both upstream and downstream operational metrics. Our approach is algorithm-agnostic, enabling integration of multiple offline RL methods under a unified architecture. We instantiate our framework with three state-of-the-art offline RL algorithms, and trained the models offline using de-identified historical operational logs from a large-scale warehouse. Policy performance is evaluated using a comprehensive multi-method strategy. These include model-free approaches including immediate reward estimation via regression models and long-horizon Fitted Q Evaluation (FQE), as well as model-based Deep Koopman dynamics evaluation. Empirical results reveal that the CQL policy consistently outperforms alternatives, improving system health by 22.97% and reducing average throttling duration by 3.18%. These findings demonstrate the potential of offline RL for safe and scalable warehouse throughput control optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
