AI 中文总结
本文提出Depth-to-RGB框架，利用冻结深度估计器预测合成场景深度，通过编码器特征监督和AnyInsertion++基准，在几何和光度质量上超越多个基线，显著降低深度误差并提升身份保持。
AI 中文摘要
基于参考的对象合成利用背景图像、参考图像和二维合成掩码来插入或替换对象。这些输入指导外观和位置，但使完成场景的几何结构隐式化，这可能会扭曲对象结构或改变周围环境。我们的Depth-to-RGB（D2R）框架预测了RGB输入中尚未观察到的场景的合成深度。它利用配对完成场景的编码器特征作为目标，学习对冻结深度估计器的参考条件修正。未改变的解码器将修正后的表示映射到预期场景的深度，而单独训练的渲染器在RGB合成期间保持该深度固定。在匹配的架构和训练下，编码器特征监督相对于解码深度监督将OOD Stage-1 AbsRel降低了31.4%。我们还引入了AnyInsertion++，具有配对的分布内和类别不相交分割，以评估超出合成训练类别的泛化能力。完整的D2R系统在配对分割上，在估计器导出的几何和光度质量方面领先于12个开源和3个闭源基线。在类别不相交数据上，D2R将AbsRel降低了43.7%，并将PSNR相对于匹配的RGB基线提高了2.4 dB。在三个非配对基准上，D2R在身份指标上领先，并将平均CLIP参考余弦距离相对于最强基线降低了55%。项目页面：此https URL
英文摘要
Reference-based object compositing inserts or replaces an object using a background image, a reference image, and a 2D compositing mask. These inputs guide appearance and placement but leave the completed scene's geometry implicit, which can distort object structure or alter the surroundings. Our Depth-to-RGB (D2R) framework predicts composite depth for a scene not yet observed in the RGB inputs. It learns reference-conditioned corrections to a frozen depth estimator using encoder features of paired completed scenes as targets. The unchanged decoder maps the corrected representation to the intended scene's depth, which a separately trained renderer holds fixed during RGB synthesis. Under matched architecture and training, encoder-feature supervision reduces OOD Stage-1 AbsRel by 31.4% relative to decoded-depth supervision. We also introduce AnyInsertion++ with paired in-distribution and category-disjoint splits to evaluate generalization beyond compositing training categories. The complete D2R system leads 12 open-source and 3 closed-source baselines in estimator-derived geometry and photometric quality on both paired splits. On category-disjoint data, D2R reduces AbsRel by 43.7% and improves PSNR by 2.4 dB over the matched RGB baseline. Across three unpaired benchmarks, D2R leads both identity metrics and reduces mean CLIP reference cosine distance by 55% relative to the strongest baseline. Project page: https://shjo-april.github.io/Depth2RGB/
