arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07342cs.AI
收藏

SupChain-Bench：针对现实世界供应链管理的大语言模型基准测试

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management

Alibaba Group（阿里巴巴集团）；Peking University（北京大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

Shengyue Guan, Yihao Liu, Lang Cao

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SupChain-Bench基准测试，用于评估大语言模型在现实世界供应链管理中的表现，揭示模型执行可靠性差距，并提出SupChain-ReAct框架提升工具调用性能。

摘要：

大型语言模型（LLMs）在复杂推理和基于工具的决策制定中展现出潜力，推动其应用于现实世界供应链管理。然而，供应链工作流需要可靠的长周期、多步骤编排，基于领域特定程序，这对当前模型仍具挑战性。为系统评估LLM在此场景下的表现，我们引入SupChain-Bench，一个统一的现实世界基准，评估供应链领域知识和基于标准操作程序（SOPs）的长周期工具编排。我们的实验揭示了模型在执行可靠性方面的显著差距。我们进一步提出SupChain-ReAct，一个无SOP框架，能够自主合成可执行程序以进行工具使用，实现了最强且最一致的工具调用性能。我们的工作建立了一个原则性的基准，用于研究现实世界运营环境中可靠的长周期编排，并突显了LLM在供应链代理中的显著改进空间。

英文摘要：

Large language models (LLMs) have shown promise in complex reasoning and tool-based decision making, motivating their application to real-world supply chain management. However, supply chain workflows require reliable long-horizon, multi-step orchestration grounded in domain-specific procedures, which remains challenging for current models. To systematically evaluate LLM performance in this setting, we introduce SupChain-Bench, a unified real-world benchmark that assesses both supply chain domain knowledge and long-horizon tool-based orchestration grounded in standard operating procedures (SOPs). Our experiments reveal substantial gaps in execution reliability across models. We further propose SupChain-ReAct, an SOP-free framework that autonomously synthesizes executable procedures for tool use, achieving the strongest and most consistent tool-calling performance. Our work establishes a principled benchmark for studying reliable long-horizon orchestration in real-world operational settings and highlights significant room for improvement in LLM-based supply chain agents.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑