发表机构
Institute of Trustworthy Embodied AI, Fudan University; Singapore Management University（复旦大学可信体智能研究院; 新加坡管理大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对现有食谱生成方法的过程结构捕捉不足问题，构建了ReGraph食谱图数据集并提出RGL两阶段框架，可让LMM从食物图像生成结构化食谱图，提升了烹饪实体与过程关系的生成效果。
AI 中文摘要
近期，大型多模态模型（Large Multimodal Models, LMMs）在从食物图像生成食谱方面取得了令人瞩目的性能。然而，烹饪是一个结构化的转换过程，其中食材会通过有序操作发生状态变化，而自由形式的食谱语言会使相应的实体、中间状态和依赖关系在很大程度上变得隐含且纠缠。图表示法能将这种过程性知识显式化并具备可组合性，为评估模型输出是否编码了过程层面的知识而非仅仅呈现合理的文本描述提供了结构化基础。为解决这一局限，我们提出了ReGraph，这是一个大规模食谱图数据集，它将食材、烹饪操作和工具表示为实体，使用实体属性描述食材状态变化，并采用带类型的关系来编码操作目标、操作目的地和过程顺序。ReGraph还纳入了显式的食谱推理思维链（Recipe Reasoning Chain-of-Thought, RR-CoT）轨迹，为过程分解和结构化图生成提供辅助监督。基于ReGraph，我们提出了食谱图学习（Recipe Graph Learning, RGL），这是一个两阶段框架，能使LMM从食物图像生成以结构化食谱图形式呈现的合理细粒度烹饪工作流。在确定性、模式感知的匹配协议下，我们的实验揭示了文本生成质量与可恢复过程结构之间存在巨大差距：现有方法生成的食谱在文本生成分数上具有竞争力，但在ReGraph模式下仅产生有限的与参考对齐的实体和关系结构。相比之下，在两个代表性LMM主干模型上，RGL始终能改进烹饪实体和过程关系的生成，而我们的分析进一步表明，细粒度的食材状态捕捉仍然是最具挑战性的维度。
英文摘要
Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transformation process in which ingredients undergo state changes through ordered actions,while free-form recipe language leaves the corresponding entities, intermediate states, and dependencies largely implicit and entangled.A graph representation makes this procedural knowledge explicit and compositional, providing a structured basis for assessing whether model outputs encode process-level knowledge rather than merely presenting plausible textual descriptions. To address this limitation, we present ReGraph, a large-scale recipe graph dataset that represents ingredients, cooking actions, and tools as entities, uses entity attributes to describe ingredient state changes, and employs typed relations to encode manipulation targets, destinations, and procedural ordering. ReGraph further incorporates explicit Recipe Reasoning Chain-of-Thought (RR-CoT) traces, providing auxiliary supervision for procedural decomposition and structured graph generation. Building on ReGraph, we propose Recipe Graph Learning (RGL), a two-stage framework that enables LMMs to generate a plausible fine-grained cooking workflow from a food image in the form of a structured recipe graph. Under a deterministic, schema-aware matching protocol, our experiments reveal a substantial gap between text-generation quality and recoverable procedural structure: recipes produced by existing approaches achieve competitive text-generation scores yet yield limited reference-aligned entity and relation structure under the ReGraph schema. In contrast, across two representative LMM backbones, RGL consistently improves the generation of cooking entities and procedural relations, while our analysis further shows that fine-grained ingredient-state capture remains the most challenging dimension.
