跳出边界框思考：无约束生成式物体合成
Thinking Outside the BBox: Unconstrained Generative Object Compositing
AI总结：
针对现有生成式物体合成依赖输入掩码、生成受限的问题，提出无约束生成式物体合成任务与扩散模型，可生成超掩码效果、自动放置物体，性能优于现有方法。
摘要：
将物体合成到图像中涉及多个复杂的子任务，例如物体放置与缩放、色彩/光照协调、视角/几何调整以及阴影/反射生成。近期的生成式图像合成方法利用扩散模型同时处理多个子任务。然而，现有模型存在局限性，因为它们在训练过程中依赖于对原始物体进行掩码处理，这将生成范围限制在输入掩码之内。此外，获取能指定物体在新图像中位置和尺度的准确输入掩码极具挑战性。为克服这些局限，我们定义了一个全新的无约束生成式物体合成问题，即生成过程不受掩码约束，并在合成配对数据集上训练了一个基于扩散的模型。我们的首创模型能够生成超出掩码范围的物体效果（如阴影和反射），提升了图像真实感。此外，若提供空掩码，我们的模型会自动将物体放置在多样的自然位置并调整尺度，加快了合成工作流程。我们的模型在多项质量指标和用户研究中均优于现有的物体放置与合成模型。
英文摘要：
Compositing an object into an image involves multiple non-trivial sub-tasks such as object placement and scaling, color/lighting harmonization, viewpoint/geometry adjustment, and shadow/reflection generation. Recent generative image compositing methods leverage diffusion models to handle multiple sub-tasks at once. However, existing models face limitations due to their reliance on masking the original object during training, which constrains their generation to the input mask. Furthermore, obtaining an accurate input mask specifying the location and scale of the object in a new image can be highly challenging. To overcome such limitations, we define a novel problem of unconstrained generative object compositing, i.e., the generation is not bounded by the mask, and train a diffusion-based model on a synthesized paired dataset. Our first-of-its-kind model is able to generate object effects such as shadows and reflections that go beyond the mask, enhancing image realism. Additionally, if an empty mask is provided, our model automatically places the object in diverse natural locations and scales, accelerating the compositing workflow. Our model outperforms existing object placement and compositing models in various quality metrics and user studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
