F3D-Gaus：基于循环聚合高斯泼溅的ImageNet前馈3D感知生成
F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Aggregative Gaussian Splatting
AI总结：
本文提出F3D-Gaus前馈流水线，通过循环聚合约束和视频模型先验增强像素对齐高斯泼溅，在ImageNet等单目数据集上实现高质量、多视角一致的3D感知生成，并提升训练与推理效率。
摘要：
本文解决的是从单目数据集（如ImageNet）进行可泛化3D感知生成的问题。该任务的关键挑战在于，在没有多视角或动态数据的情况下学习鲁棒的3D感知表示，同时确保不同视角下纹理和几何的一致性。尽管一些基线方法能够进行3D感知生成，但其生成图像的质量仍落后于最先进的2D生成方法，后者擅长生成高质量、细节丰富的图像。为了解决这一严重局限，我们提出了一种基于像素对齐高斯泼溅（Gaussian Splatting）的新型前馈流水线，命名为F3D-Gaus，能够从单目输入生成更真实、更可靠的3D渲染结果。此外，我们引入了一种自监督的循环聚合约束，以强化所学3D表示中的跨视角一致性。该训练策略天然支持多个对齐高斯原语的聚合，并显著缓解了单视角像素对齐高斯泼溅中固有的插值局限。进一步地，我们引入视频模型先验进行几何感知细化，增强了宽视角场景下精细细节的生成能力，并提升了模型捕捉复杂3D纹理的能力。大量实验表明，我们的方法不仅能够从单目数据集实现高质量、多视角一致的3D感知生成，还显著提高了训练和推理效率。
英文摘要：
This paper tackles the problem of generalizable 3D-aware generation from monocular datasets, e.g., ImageNet. The key challenge of this task is learning a robust 3D-aware representation without multi-view or dynamic data, while ensuring consistent texture and geometry across different viewpoints. Although some baseline methods are capable of 3D-aware generation, the quality of the generated images still lags behind state-of-the-art 2D generation approaches, which excel in producing high-quality, detailed images. To address this severe limitation, we propose a novel feed-forward pipeline based on pixel-aligned Gaussian Splatting, coined as F3D-Gaus, which can produce more realistic and reliable 3D renderings from monocular inputs. In addition, we introduce a self-supervised cycle-aggregative constraint to enforce cross-view consistency in the learned 3D representation. This training strategy naturally allows aggregation of multiple aligned Gaussian primitives and significantly alleviates the interpolation limitations inherent in single-view pixel-aligned Gaussian Splatting. Furthermore, we incorporate video model priors to perform geometry-aware refinement, enhancing the generation of fine details in wide-viewpoint scenarios and improving the model's capability to capture intricate 3D textures. Extensive experiments demonstrate that our approach not only achieves high-quality, multi-view consistent 3D-aware generation from monocular datasets, but also significantly improves training and inference efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
