VisualChef：基于掩码修复生成烹饪场景下的视觉辅助内容
VisualChef: Generating Visual Aids in Cooking via Mask Inpainting
AI总结：
针对烹饪过程缺乏一致视觉引导的问题，提出VisualChef方法，通过基于掩码的视觉定位生成保留环境一致性的烹饪动作视觉辅助内容，还配套自动化帧提取流程，在三个第一视角数据集上表现优于现有最优方法。
摘要：
烹饪不仅需要遵循操作说明，还需要理解、执行并监控每一个步骤——在缺乏视觉引导的情况下，这一过程可能颇具挑战。尽管食谱图片和视频能提供有用的参考线索，但它们在焦点、工具和场景布置上往往缺乏一致性。为更好地支撑烹饪流程，我们提出了VisualChef，一种专为烹饪场景定制的上下文视觉辅助内容生成方法。给定初始帧和指定动作，VisualChef能够生成同时展示动作执行过程和操作对象最终状态的图像，同时保留初始帧中的环境信息。此前的研究试图通过生成详细的文本描述来引导图像生成，以此整合从大语言模型中提取的知识，这类方法需要细粒度的视觉-文本对齐，还涉及额外的标注工作。与之不同，VisualChef通过基于掩码的视觉定位简化了对齐过程。我们的核心思路是识别与动作相关的物体并对其分类，从而实现有针对性的修改，在保持环境一致性的同时反映预期的动作和结果。此外，我们提出了一套自动化流程，用于提取高质量的初始帧、动作帧和最终状态帧。我们在三个第一视角视频数据集上对VisualChef进行了定量和定性评估，结果表明其性能优于现有最优方法。
英文摘要：
Cooking requires not only following instructions but also understanding, executing, and monitoring each step - a process that can be challenging without visual guidance. Although recipe images and videos offer helpful cues, they often lack consistency in focus, tools, and setup. To better support the cooking process, we introduce VisualChef, a method for generating contextual visual aids tailored to cooking scenarios. Given an initial frame and a specified action, VisualChef generates images depicting both the action's execution and the resulting appearance of the object, while preserving the initial frame's environment. Previous work aims to integrate knowledge extracted from large language models by generating detailed textual descriptions to guide image generation, which requires fine-grained visual-textual alignment and involves additional annotations. In contrast, VisualChef simplifies alignment through mask-based visual grounding. Our key insight is identifying action-relevant objects and classifying them to enable targeted modifications that reflect the intended action and outcome while maintaining a consistent environment. In addition, we propose an automated pipeline to extract high-quality initial, action, and final state frames. We evaluate VisualChef quantitatively and qualitatively on three egocentric video datasets and show its improvements over state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
