SWEET：基于图像编辑的稀疏世界建模用于具身任务执行
SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution
AI总结：
本文研究图像编辑模型能否作为稀疏视觉世界模型用于机器人操作，通过预测任务级未来状态而非密集视频生成，提出SWEET框架实现稀疏视觉规划，结合语言指令和空间引导生成关键帧，并通过扩散动作预测器生成可执行动作，实验表明其在不同场景中提升关键帧预测能力。
摘要：
视觉预测已成为具身控制的有前景范式，其中未来观察被生成并转化为动作。然而，密集视频生成计算成本高且对许多操作任务而言往往不必要，其进展可以总结为少量任务相关视觉状态。本文研究图像编辑模型能否作为稀疏视觉世界模型用于机器人操作，通过预测任务级未来状态而非密集视频生成。我们首先在相同的机器人数据设置下比较视频生成模型Wan2.2和图像编辑模型FLUX-Kontext，发现图像编辑能生成更可靠的任务级关键帧，具有更好的视觉保真度和显著更低的推理成本。受此启发，我们提出SWEET，一种单次稀疏视觉规划框架，通过连续图像编辑生成一系列任务相关操作关键帧，基于语言指令和可选箭头式空间引导。一个目标条件化的扩散动作预测器将相邻想象的关键帧转换为可执行的动作块。为了减少真实与编辑视觉子目标之间的不匹配，我们进一步引入混合训练策略，使用过滤后的编辑目标。在DROID和RoboMimic上的实验表明，SWEET在已见和未见场景中均提升了关键帧预测能力，并实现了从序列关键帧规划到可执行机器人动作的完整流程，表明图像编辑是具身视觉预测中一个有前景但尚未被广泛探索的方向。
英文摘要：
Visual prediction has emerged as a promising paradigm for embodied control, where future observations are generated and then translated into actions. However, dense video generation is computationally expensive and often unnecessary for many manipulation tasks, whose progress can be summarized by a small number of task-relevant visual states. In this work, we study whether image editing models can serve as sparse visual world models for robot manipulation by predicting task-level future states without dense video rollout. We first conduct a controlled comparison between the video generation model Wan2.2 and the image editing model FLUX-Kontext under the same robotic data setting, and find that image editing produces more reliable task-level keyframes with better visual fidelity and substantially lower inference cost. Motivated by this observation, we propose SWEET, a one-shot sparse visual planning framework that progressively generates a sequence of task-relevant manipulation keyframes through successive image editing, conditioned on language instructions and optional arrow-based spatial guidance. A goal-conditioned diffusion action predictor then converts adjacent imagined keyframes into executable action chunks. To reduce the mismatch between real and edited visual subgoals, we further introduce a mixed-training strategy with filtered edited targets. Experiments on DROID and RoboMimic show that SWEET improves keyframe prediction across seen and unseen scenes and enables a full pipeline from sequential keyframe planning to executable robot actions, suggesting that image editing is a promising and underexplored direction for embodied visual prediction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
