DreamSat：面向空间目标新视角合成的通用3D模型研究
DreamSat: Towards a General 3D Model for Novel View Synthesis of Space Objects
AI总结：
针对航天领域缺乏专用3D重建工具的问题，研究团队提出DreamSat方法，通过微调Zero123 XL并集成到DreamGaussian框架，实现了单视角航天器图像的高质量3D重建，多项指标均有提升且保留了框架高效性。
摘要：
新视角合成（NVS）能够生成场景的新图像，或将一组2D图像转换为完整的3D模型。在空间域感知领域，随着空间环境日益拥挤，NVS可对空间目标和碎片进行精确测绘，提升空间操作的安全性与效率。同样，在交会与近距操作任务中，3D模型可提供目标目标的形状、尺寸、姿态等细节，助力更好地规划和预测目标行为。\n本研究探索了这类重建技术的泛化能力，旨在避免每个新场景都需要重新训练的问题，提出了一种从单视角图像进行航天器3D重建的新方法DreamSat：我们在包含190个高质量航天器模型的优质数据集上，对当前最先进的单视角重建模型Zero123 XL进行微调，并将其集成到DreamGaussian框架中。\n我们在包含30张从未见过的航天器图像的测试集上验证了该方法的效果，在多项指标上实现了重建质量的稳定提升，包括对比语言-图像预训练（CLIP）得分提升0.33%、峰值信噪比（PSNR）提升2.53%、结构相似性指数（SSIM）提升2.38%，以及学习感知图像块相似度（LPIPS）提升0.16%。\n我们的方法利用最先进的扩散模型和3D高斯溅射技术，解决了航天工业缺乏特定领域3D重建工具的问题。该方法在保留DreamGaussian框架高效性的同时，提升了航天器重建的准确性和细节丰富度。本研究的代码可在GitHub获取（https://github.com/ARCLab-MIT/space-nvs）。
英文摘要：
Novel view synthesis (NVS) enables to generate new images of a scene or convert a set of 2D images into a comprehensive 3D model. In the context of Space Domain Awareness, since space is becoming increasingly congested, NVS can accurately map space objects and debris, improving the safety and efficiency of space operations. Similarly, in Rendezvous and Proximity Operations missions, 3D models can provide details about a target object's shape, size, and orientation, allowing for better planning and prediction of the target's behavior. In this work, we explore the generalization abilities of these reconstruction techniques, aiming to avoid the necessity of retraining for each new scene, by presenting a novel approach to 3D spacecraft reconstruction from single-view images, DreamSat, by fine-tuning the Zero123 XL, a state-of-the-art single-view reconstruction model, on a high-quality dataset of 190 high-quality spacecraft models and integrating it into the DreamGaussian framework. We demonstrate consistent improvements in reconstruction quality across multiple metrics, including Contrastive Language-Image Pretraining (CLIP) score (+0.33%), Peak Signal-to-Noise Ratio (PSNR) (+2.53%), Structural Similarity Index (SSIM) (+2.38%), and Learned Perceptual Image Patch Similarity (LPIPS) (+0.16%) on a test set of 30 previously unseen spacecraft images. Our method addresses the lack of domain-specific 3D reconstruction tools in the space industry by leveraging state-of-the-art diffusion models and 3D Gaussian splatting techniques. This approach maintains the efficiency of the DreamGaussian framework while enhancing the accuracy and detail of spacecraft reconstructions. The code for this work can be accessed on GitHub (https://github.com/ARCLab-MIT/space-nvs).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
