InverseMatrixVT3D：一种用于3D占用预测的高效投影矩阵方法
InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D Occupancy Prediction
AI总结：
本文提出InverseMatrixVT3D，利用两个稀疏投影矩阵和矩阵乘法高效将多视图图像特征转换为3D特征体，结合全局-局部注意力融合与多尺度监督，在nuScenes和SemanticKITTI上实现了顶尖的弱势道路使用者检测性能。
摘要：
本文介绍了InverseMatrixVT3D，这是一种将多视图图像特征转换为3D特征体以进行3D语义占用预测的高效方法。现有的构建3D体的方法通常依赖于深度估计、特定于设备的算子或transformer查询，这阻碍了3D占用模型的广泛采用。相比之下，我们的方法利用两个投影矩阵来存储静态映射关系，并通过矩阵乘法高效生成全局鸟瞰图（BEV）特征和局部3D特征体。具体而言，我们通过在多视图图像特征图和两个稀疏投影矩阵之间执行矩阵乘法来实现这一点。我们引入了一种针对投影矩阵的稀疏矩阵处理技术，以优化GPU内存使用。此外，提出了一个全局-局部注意力融合模块，将全局BEV特征与局部3D特征体集成以获得最终的3D体。我们还采用了多尺度监督机制以进一步提升性能。在nuScenes和SemanticKITTI数据集上进行的大量实验表明，我们的方法不仅因其简单性和有效性而脱颖而出，而且在检测弱势道路使用者（VRU）方面达到了顶级性能，这对于自动驾驶和道路安全至关重要。代码已公开于：https://github.com/DanielMing123/InverseMatrixVT3D
英文摘要：
This paper introduces InverseMatrixVT3D, an efficient method for transforming multi-view image features into 3D feature volumes for 3D semantic occupancy prediction. Existing methods for constructing 3D volumes often rely on depth estimation, device-specific operators, or transformer queries, which hinders the widespread adoption of 3D occupancy models. In contrast, our approach leverages two projection matrices to store the static mapping relationships and matrix multiplications to efficiently generate global Bird's Eye View (BEV) features and local 3D feature volumes. Specifically, we achieve this by performing matrix multiplications between multi-view image feature maps and two sparse projection matrices. We introduce a sparse matrix handling technique for the projection matrices to optimize GPU memory usage. Moreover, a global-local attention fusion module is proposed to integrate the global BEV features with the local 3D feature volumes to obtain the final 3D volume. We also employ a multi-scale supervision mechanism to enhance performance further. Extensive experiments performed on the nuScenes and SemanticKITTI datasets reveal that our approach not only stands out for its simplicity and effectiveness but also achieves the top performance in detecting vulnerable road users (VRU), crucial for autonomous driving and road safety. The code has been made available at: https://github.com/DanielMing123/InverseMatrixVT3D
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
