Cascade-Zero123：利用自提示邻近视图从单张图像生成高度一致的3D
Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Views
AI总结：
Cascade-Zero123通过级联两个Zero-1-to-3模型，利用自提示邻近视图增强条件信息，从单张图像生成高度一致的多视图3D，显著改善了复杂场景下的几何与外观一致性。
摘要：
从单张图像合成多视图3D是一项重要但具有挑战性的任务。Zero-1-to-3方法通过将2D潜在扩散模型提升到3D范围取得了巨大成功。目标视图图像以单视图源图像和相机姿态作为条件信息生成。然而，由于单张输入图像的高度稀疏性，Zero-1-to-3倾向于在不同视图之间产生几何和外观不一致，尤其是对于复杂物体。为了解决这个问题，我们提出以自提示的方式为生成模型提供更多条件信息。我们构建了一个包含两个Zero-1-to-3模型的级联框架，命名为Cascade-Zero123，该框架逐步从源图像中提取3D信息。具体来说，首先由第一个模型生成几个邻近视图，然后将这些视图与源图像一起作为生成条件输入到第二阶段模型中。通过增强的自提示条件图像，我们的Cascade-Zero123比Zero-1-to-3生成更一致的新视图图像。实验结果展示了显著的提升，尤其是对于各种复杂和具有挑战性的场景，包括昆虫、人体、透明物体和堆叠的多个物体等。更多演示和代码可在https://cascadezero123.github.io获取。
英文摘要：
Synthesizing multi-view 3D from one single image is a significant but challenging task. Zero-1-to-3 methods have achieved great success by lifting a 2D latent diffusion model to the 3D scope. The target view image is generated with a single-view source image and the camera pose as condition information. However, due to the high sparsity of the single input image, Zero-1-to-3 tends to produce geometry and appearance inconsistency across views, especially for complex objects. To tackle this issue, we propose to supply more condition information for the generation model but in a self-prompt way. A cascade framework is constructed with two Zero-1-to-3 models, named Cascade-Zero123, which progressively extract 3D information from the source image. Specifically, several nearby views are first generated by the first model and then fed into the second-stage model along with the source image as generation conditions. With amplified self-prompted condition images, our Cascade-Zero123 generates more consistent novel-view images than Zero-1-to-3. Experiment results demonstrate remarkable promotion, especially for various complex and challenging scenes, involving insects, humans, transparent objects, and stacked multiple objects etc. More demos and code are available at https://cascadezero123.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
