AI 中文总结
本文提出ASV3D框架，通过零样本适配和优化适配两种策略，改进单视图3D重建，在基准及真实数据集上提升了重建精度与鲁棒性，效果优于基线方法。
AI 中文摘要
从单张图像重建3D物体是计算机视觉中极具挑战性的研究问题，核心难点在于缺少完成3D结构所需的关键视角信息，使用额外视角或可解决该问题，但目前尚无机制能将额外视角融入单视图3D重建原理。我们提出ASV3D框架以解决该挑战，该框架用于适配单视图3D物体重建，使其可利用一张额外图像的支持处理测试时数据。我们引入两种适配策略：（i）零样本适配方案，无需重新训练即可利用辅助图像提升物体重建质量；（ii）优化适配方案，通过对比学习进一步提升视觉保真度和跨视图一致性。我们将ASV3D应用于改进两个最先进的单视图3D重建流水线，在基准数据集和真实世界数据集上进行验证。结果表明，我们的方法在无约束多视图输入下始终提升重建精度和鲁棒性，在定量指标和人类偏好上均优于基线方法。我们在项目页面发布代码和真实世界物体数据集，网址为this https URL。
英文摘要
Reconstruction of 3D objects from a single image is a fundamental research topic in computer vision. The key challenge is the lack of information from critical viewpoints to complete 3D structures. Using an additional view may help to resolve the issue. However, there is no mechanism that can integrate the extra view into the diffusion-based single-view 3D reconstruction principle. We address this challenge by proposing ASV3D, a framework for adapting diffusion-based single-view 3D object reconstruction to test-time data with support from one additional image. We introduce two adaptation strategies: (i) a zero-shot adaptation scheme that leverages the auxiliary image to improve the reconstruction quality of an object without retraining, and (ii) an optimised adaptation scheme that further enhances visual fidelity and cross-view consistency via contrastive learning. We apply our ASV3D to improve two state-of-the-art diffusion-based single-view 3D reconstruction pipelines on both benchmark and real-world datasets. Results demonstrate that our approach consistently improves reconstruction accuracy and robustness under unconstrained multi-view inputs, outperforming the baselines in both quantitative metrics and human preference. We publish our code and the real-world object dataset on our project page at https://github.com/YNhuHuynh/ASV3D/tree/main.
