面向3D点云生成的采用极端掩码的扩散Transformer快速训练方法
Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation
AI总结：
针对高分辨率3D体素扩散模型训练成本过高的问题，提出FastDiT-3D，通过体素感知掩码策略和MoE机制，在近99%掩码率下实现SOTA生成效果，训练成本仅为原方法的6.5%。
摘要：
扩散Transformer近期在生成高质量3D点云方面展现出显著成效。然而，由于体素的额外维度带来注意力算子的三次方复杂度，针对高分辨率3D体素训练基于体素的扩散模型成本仍然极高，令人难以承受。受3D相较于2D存在固有冗余性的启发，我们提出FastDiT-3D，这是一种专为高效3D点云生成定制的新型掩码扩散Transformer，可大幅降低训练成本。具体而言，我们从掩码自编码器中汲取灵感，在掩码化的体素化点云上动态执行去噪过程。我们还提出了一种新颖的体素感知掩码策略，可从体素化点云中自适应聚合背景/前景信息。我们的方法在接近99%的极端掩码比例下实现了最优性能。此外，为提升多类别3D生成效果，我们在3D扩散模型中引入了混合专家（Mixture-of-Expert, MoE）机制。每个类别可借助不同专家学习独立的扩散路径，从而缓解梯度冲突。在ShapeNet数据集上的实验结果表明，我们的方法在高保真度和多样化3D点云生成方面达到了最优水平。我们的FastDiT-3D在生成128分辨率体素点云时，提升了1-最近邻准确率和覆盖率指标，且仅需原有训练成本的6.5%。
英文摘要：
Diffusion Transformers have recently shown remarkable effectiveness in generating high-quality 3D point clouds. However, training voxel-based diffusion models for high-resolution 3D voxels remains prohibitively expensive due to the cubic complexity of attention operators, which arises from the additional dimension of voxels. Motivated by the inherent redundancy of 3D compared to 2D, we propose FastDiT-3D, a novel masked diffusion transformer tailored for efficient 3D point cloud generation, which greatly reduces training costs. Specifically, we draw inspiration from masked autoencoders to dynamically operate the denoising process on masked voxelized point clouds. We also propose a novel voxel-aware masking strategy to adaptively aggregate background/foreground information from voxelized point clouds. Our method achieves state-of-the-art performance with an extreme masking ratio of nearly 99%. Moreover, to improve multi-category 3D generation, we introduce Mixture-of-Expert (MoE) in 3D diffusion model. Each category can learn a distinct diffusion path with different experts, relieving gradient conflict. Experimental results on the ShapeNet dataset demonstrate that our method achieves state-of-the-art high-fidelity and diverse 3D point cloud generation performance. Our FastDiT-3D improves 1-Nearest Neighbor Accuracy and Coverage metrics when generating 128-resolution voxel point clouds, using only 6.5% of the original training cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
