arXiv 2506.031072025-06-11cs.CV

ByteMorph：针对非刚性运动的指令引导图像编辑基准测试

ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions

ByteDance Seed（字节跳动种子）；University of Southern California（南加州大学）；University of Tokyo（东京大学）；University of California Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；University of California Los Angeles（加州大学洛杉矶分校）

Di Chang, Mingdeng Cao, Yichun Shi, Bo Liu, Shengqu Cai, Shijie Zhou, Weilin Huang, Gordon Wetzstein, Mohammad Soleymani, Peng Wang

备注：: Website: https://boese0601.github.io/bytemorph Dataset: https://huggingface.co/datasets/ByteDance-Seed/BM-6M Benchmark: https://huggingface.co/datasets/ByteDance-Seed/BM-Bench Code: https://github.com/ByteDance-Seed/BM-code Demo: https://huggingface.co/spaces/Boese0601/ByteMorph-Demo

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有图像编辑方法难以处理非刚性运动动态编辑的问题，研究团队提出包含600万对训练数据的ByteMorph框架及ByteMorpher基线模型，构建多类型非刚性运动编辑基准并开展综合评估。

摘要：

通过指令编辑图像以呈现非刚性运动、相机视角偏移、物体形变、人体关节运动及复杂交互，是计算机视觉领域中一个具有挑战性但研究不足的问题。现有方法和数据集主要聚焦于静态场景或刚性变换，限制了其处理涉及动态运动的表现力编辑的能力。为填补这一空白，我们提出ByteMorph，一个专注于非刚性运动的指令式图像编辑综合框架。ByteMorph包含一个大规模数据集ByteMorph-6M，以及一个基于Diffusion Transformer（DiT，扩散变换器）构建的强基线模型ByteMorpher。ByteMorph-6M包含超过600万对高分辨率图像编辑对用于训练，还有一个精心策划的评估基准ByteMorph-Bench。两者都涵盖了多样环境、人体形态和物体类别下的多种非刚性运动类型。该数据集通过运动引导的数据生成、分层合成技术和自动字幕构建，以确保多样性、真实性和语义一致性。我们进一步对学术和商业领域近期的指令式图像编辑方法进行了综合评估。

英文摘要：

Editing images with instructions to reflect non-rigid motions, camera viewpoint shifts, object deformations, human articulations, and complex interactions, poses a challenging yet underexplored problem in computer vision. Existing approaches and datasets predominantly focus on static scenes or rigid transformations, limiting their capacity to handle expressive edits involving dynamic motion. To address this gap, we introduce ByteMorph, a comprehensive framework for instruction-based image editing with an emphasis on non-rigid motions. ByteMorph comprises a large-scale dataset, ByteMorph-6M, and a strong baseline model built upon the Diffusion Transformer (DiT), named ByteMorpher. ByteMorph-6M includes over 6 million high-resolution image editing pairs for training, along with a carefully curated evaluation benchmark ByteMorph-Bench. Both capture a wide variety of non-rigid motion types across diverse environments, human figures, and object categories. The dataset is constructed using motion-guided data generation, layered compositing techniques, and automated captioning to ensure diversity, realism, and semantic coherence. We further conduct a comprehensive evaluation of recent instruction-based image editing methods from both academic and commercial domains.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。