WarpHammer: 利用3D物体先验稠密化场景扭曲以实现极端视角合成
WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesis
AI总结：
针对投影条件新视角合成在大轨道运动下扭曲稀疏导致生成失败的问题，提出无需训练的WarpHammer框架，通过引入3D生成先验的物体显式重建来补充缺失前景并遮挡不应可见的背景点，恢复外观和相机线索，并首次支持融合外部辅助物体视图。
摘要：
投影条件新视角合成（NVS）将输入视图的显式3D重建扭曲到目标相机，并对生成器进行条件化处理。这种方法在小视角变化下效果良好，但在大轨道运动下急剧退化：围绕轨道物体的扭曲变得稀疏，隐藏表面主导新视角，出现镜面伪影，导致生成器丢失像素内容和扭曲携带的隐式相机线索。我们提出WarpHammer，一个无需训练的框架，通过从原生3D生成先验（如SAM3D）获得的物体显式3D重建来增强扭曲场景，从而解决这种失败模式。重建的物体添加了缺失的前景表面，并遮挡了不应再可见的背景点，无需微调基础模型即可恢复外观和相机线索。相同的显式物体表示进一步解锁了当前NVS流水线不支持的能力：融合来自目标场景外部的物体辅助视图，例如，一辆车的随意快照与同一车型的制造商工作室照片。我们联合处理参考图像和辅助图像，使用预训练的多视图几何基础模型，预测统一点云，并将其融合到3D物体重建中。这比单图像重建产生更准确的几何形状，且无需用户提供辅助视图的相机位姿。在五个基准测试中，WarpHammer在强基线崩溃的视角偏差下生成稳定的新视图，并且是首个能够自然融合外部来源、位姿未知的辅助物体视图的场景级NVS方法。
英文摘要：
Projection-conditioned novel view synthesis (NVS) warps an explicit 3D reconstruction of the input view into the target camera and conditions a generator on the warped rendering. This works well for small viewpoint changes but degrades sharply under large orbital motion: the warp becomes sparse around the orbited object, where hidden surfaces dominate the new view and mirror-like artifacts emerge, causing the generator to lose both pixel content and the implicit camera cue carried by the warp. We introduce WarpHammer, a training-free framework that resolves this failure mode by augmenting the warped scene with an explicit 3D reconstruction of the object obtained from a native 3D generative prior (e.g., SAM3D). The reconstructed object adds missing foreground surfaces and occludes background points that should no longer be visible, restoring both appearance and camera cues without fine-tuning the base model. The same explicit object representation further unlocks a capability current NVS pipelines do not support: incorporating auxiliary views of the object from sources outside the target scene, for example, a casual snapshot of a car paired with a manufacturer studio shot of the same model. We process the reference and auxiliary images jointly with a pretrained multi-view geometry foundation model, which predicts a unified point cloud that we fuse into the 3D object reconstruction. This yields substantially more faithful geometry than single-image reconstruction, without requiring user-provided camera poses for the auxiliary views. On five benchmarks, WarpHammer produces stable novel views at viewpoint deviations where strong baselines collapse, and is the first scene-level NVS method that can naturally fuse auxiliary, pose-unknown object views from an external source.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
