arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.24883cs.LG
收藏

学习人员配置:用于仓库人员配置优化的离线强化学习与微调大语言模型

Learning to Staff: Offline Reinforcement Learning and Fine-Tuned LLMs for Warehouse Staffing Optimization

Amazon, Fulfillment Technologies & Robotics(亚马逊,物流技术与机器人)

Kalle Kujanpää, Yuying Zhu, Kristina Klinkner, Shervin Malmasi

AI总结:

本文研究了优化半自动化仓库分拣系统实时人员配置的机器学习方法,通过离线强化学习和微调大语言模型,提升了仓库运营效率。

摘要:

我们研究了优化半自动化仓库分拣系统实时人员配置的机器学习方法。操作决策可以在不同抽象层次上支持,具有不同的权衡。我们评估了两种方法,每种方法都在匹配的模拟环境中进行。首先,我们使用离线强化学习训练定制的Transformer策略,基于详细的历史状态表示,在学习的模拟器中实现了比历史基线高2.4%的吞吐量。在高流量仓库运营中,这种改进带来了显著的节省。其次,我们探索了在抽象化、可读的人类状态描述上操作的LLM。这些对于仓库管理人员使用高级操作摘要做出决策是天然的。我们系统地比较了提示技术、自动提示优化和微调策略。虽然仅使用提示本身证明不足,但监督微调结合直接偏好优化在模拟器生成的偏好上实现了与历史基线相匹配或略高的性能。我们的发现表明,这两种方法都为AI辅助的操作决策提供了可行的路径。离线强化学习在任务特定的架构上表现优异。LLM支持人类可读的输入,并可以与一个可以整合管理者偏好的迭代反馈循环结合。

英文摘要:

We investigate machine learning approaches for optimizing real-time staffing decisions in semi-automated warehouse sortation systems. Operational decision-making can be supported at different levels of abstraction, with different trade-offs. We evaluate two approaches, each in a matching simulation environment. First, we train custom Transformer-based policies using offline reinforcement learning on detailed historical state representations, achieving a 2.4% throughput improvement over historical baselines in learned simulators. In high-volume warehouse operations, improvements of this size translate to significant savings. Second, we explore LLMs operating on abstracted, human-readable state descriptions. These are a natural fit for decisions that warehouse managers make using high-level operational summaries. We systematically compare prompting techniques, automatic prompt optimization, and fine-tuning strategies. While prompting alone proves insufficient, supervised fine-tuning combined with Direct Preference Optimization on simulator-generated preferences achieves performance that matches or slightly exceeds historical baselines in a hand-crafted simulator. Our findings demonstrate that both approaches offer viable paths toward AI-assisted operational decision-making. Offline RL excels with task-specific architectures. LLMs support human-readable inputs and can be combined with an iterative feedback loop that can incorporate manager preferences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑