HybridOcc：NeRF增强的基于Transformer的多相机3D占据预测
HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Prediction
AI总结：
针对自动驾驶3D语义场景补全中不可见体素遮挡导致几何生成粗糙的问题，提出融合Transformer与NeRF的HybridOcc方法，通过混合查询、多尺度变换、占据感知采样提升性能，在两个基准数据集上验证有效。
摘要：
基于视觉的3D语义场景补全（SSC）通过3D体素表示描述自动驾驶场景。然而，场景表面对不可见体素的遮挡给当前SSC方法在生成精细3D几何结构方面带来了挑战。本文提出HybridOcc，这是一种由Transformer框架与NeRF表示生成、并在由粗到精的SSC预测框架中优化的混合3D体素查询提议方法。HybridOcc基于混合查询提议，通过Transformer范式聚合上下文特征，同时结合NeRF表示获取深度监督。Transformer分支包含多个尺度，采用空间交叉注意力实现2D到3D的转换。新设计的NeRF分支通过体绘制隐式推断场景占据情况（涵盖可见与不可见体素），并显式捕获场景深度而非生成RGB颜色。此外，我们提出了一种创新的占据感知光线采样方法，用于适配SSC任务而非聚焦于场景表面，进一步提升了整体性能。在nuScenes和SemanticKITTI数据集上开展的大量实验证明了HybridOcc在SSC任务上的有效性。
英文摘要：
Vision-based 3D semantic scene completion (SSC) describes autonomous driving scenes through 3D volume representations. However, the occlusion of invisible voxels by scene surfaces poses challenges to current SSC methods in hallucinating refined 3D geometry. This paper proposes HybridOcc, a hybrid 3D volume query proposal method generated by Transformer framework and NeRF representation and refined in a coarse-to-fine SSC prediction framework. HybridOcc aggregates contextual features through the Transformer paradigm based on hybrid query proposals while combining it with NeRF representation to obtain depth supervision. The Transformer branch contains multiple scales and uses spatial cross-attention for 2D to 3D transformation. The newly designed NeRF branch implicitly infers scene occupancy through volume rendering, including visible and invisible voxels, and explicitly captures scene depth rather than generating RGB color. Furthermore, we present an innovative occupancy-aware ray sampling method to orient the SSC task instead of focusing on the scene surface, further improving the overall performance. Extensive experiments on nuScenes and SemanticKITTI datasets demonstrate the effectiveness of our HybridOcc on the SSC task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
