Chat2Workflow: 用自然语言生成可执行可视化工作流的基准
Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language
AI总结：
提出Chat2Workflow基准，用于评估大语言模型从自然语言生成可执行可视化工作流的能力，并设计了一个智能体基线以提升性能。
摘要：
目前，可执行的可视化工作流已成为实际工业部署中的主流范式，提供了强大的可靠性和可控性。然而，在当前实践中，此类工作流几乎完全通过手动工程构建：开发人员必须仔细设计工作流，为每个步骤编写提示，并随着需求的变化反复修改逻辑——这使得开发成本高昂、耗时且容易出错。为了研究大语言模型能否自动化这一多轮交互过程，我们引入了Chat2Workflow，一个直接从自然语言生成可执行可视化工作流的基准，并提出了一个稳健的智能体基线以提高性能。该基准基于大量真实业务工作流构建，每个实例的设计使得生成的工作流可以转换并直接部署到实际工作流平台（如Dify和Coze）上。实验结果表明，尽管最先进的语言模型通常能捕捉高层次意图，但在生成正确、稳定且可执行的工作流方面仍存在困难，尤其是在面对复杂且不断变化的需求时。尽管我们的智能体基线带来了高达6.05%的解决率提升，但剩余的现实差距使Chat2Workflow成为推进工业级自动化的基础。代码可在https://github.com/zjunlp/Chat2Workflow获取。
英文摘要：
At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely constructed through manual engineering: developers must carefully design workflows, write prompts for each step, and repeatedly revise the logic as requirements evolve -- making development costly, time-consuming, and error-prone. To study whether large language models can automate this multi-round interaction process, we introduce Chat2Workflow, a benchmark for generating executable visual workflows directly from natural language, and propose a robust agentic baseline to improve performance. The benchmark is built from a large collection of real-world business workflows, with each instance designed so that the generated workflow can be transformed and directly deployed to practical workflow platforms such as Dify and Coze. Experimental results show that while state-of-the-art language models can often capture high-level intent, they struggle to generate correct, stable, and executable workflows, especially given complex and evolving requirements. Although our agentic baseline yields up to 6.05% resolve rate gains, the remaining real-world gap positions Chat2Workflow as a foundation for advancing industrial-grade automation. Code is available at https://github.com/zjunlp/Chat2Workflow.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
