发表机构
INSAIT; Sofia University “St. Kliment Ohridski”; Adobe Research（未知机构; 索非亚大学“圣克莱门特·奥赫里德斯基”分校; Adobe研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究视觉创作系统中工作流生成问题，提出以知识为中心的框架，通过知识反转、注入和可逆推理进行工作流生成，实验证明该方法生成的工作流在多样性、结构连贯性和执行成功率上优于现有系统。
AI 中文摘要
在诸如ComfyUI等视觉创作系统中，工作流生成不仅需要句法准确性，还需要对模块化组合进行专家级推理。现有的大语言模型方法往往将其视为直接的文本到JSON生成任务，存在结构脆弱性问题，且缺乏有效设计所需的经验知识。我们认为成功的工作流生成需要对知识本身进行建模，包括其结构、层次和推理动态。为此，我们提出了一个以知识为中心的框架，该框架学习在多个抽象层次上对知识进行反转、注入和推理。我们首先进行知识反转，从大量实际工作流中提取层次表示，然后通过监督微调进行知识注入，在推理过程中，模型进行可逆推理以合成可执行工作流，并通过自我优化增强结构连贯性。大量实验表明，我们的方法比现有系统生成的工作流具有更丰富的节点多样性、更连贯的结构和更高的执行成功率，为知识驱动的智能工作流生成奠定了新基础。
英文摘要
Workflow generation in visual creation systems such as ComfyUI demands not only syntactic accuracy but also expert-level reasoning over modular compositions. Existing large language model (LLM) approaches often treat this as a direct text-to-JSON generation task, struggling with structural brittleness and lacking the experiential knowledge required for effective design. We argue that successful workflow generation requires modeling knowledge itself, including its structure, hierarchy, and reasoning dynamics. To this end, we propose a knowledge-centric framework that learns to invert, inject, and infer with knowledge across multiple abstraction levels. We first perform knowledge inversion to distill hierarchical representations, ranging from full pseudo-codes and skeletons to high-level strategies, from large collections of real-world workflows. We then conduct knowledge injection through supervised fine-tuning, teaching the model to reason from task descriptions to strategies and from strategies to executable structures. During inference, the model performs reversible reasoning to synthesize executable workflows, augmented by self-refinement for structural coherence. Extensive experiments demonstrate that our method produces workflows with richer node diversity, more coherent structures, and higher execution success rates than existing systems, establishing a new foundation for knowledge-driven, agentic workflow generation.
CommentsAccepted to ECCV 2026
