高效可变形卷积网络：重新思考面向视觉应用的动态与稀疏算子
Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications
AI总结：
本文提出可变形卷积 v4（DCNv4），通过移除 softmax 归一化和优化内存访问，在收敛速度与前向速度上显著超越 DCNv3，并在分类、分割及图像生成等任务中表现优异，可作为未来视觉模型的基础算子。
摘要：
我们提出可变形卷积 v4（DCNv4），这是一种为广泛视觉应用设计的高效且有效的算子。DCNv4 通过两项关键增强解决了其前身 DCNv3 的局限性：1. 在空间聚合中移除 softmax 归一化，以增强其动态特性和表达能力；2. 优化内存访问，以最小化冗余操作从而实现加速。这些改进使得与 DCNv3 相比收敛速度显著加快，处理速度大幅提升，DCNv4 的前向速度达到三倍以上。DCNv4 在各类任务中展现出卓越性能，包括图像分类、实例分割和语义分割，尤其是图像生成。当集成到诸如潜在扩散模型中的 U-Net 等生成模型中时，DCNv4 优于其基线，凸显了其增强生成模型的可能性。在实际应用中，将 InternImage 模型中的 DCNv3 替换为 DCNv4 以创建 FlashInternImage，无需进一步修改即可实现高达 80% 的速度提升和进一步的性能改进。DCNv4 在速度和效率方面的进步，加上其在多种视觉任务中的稳健性能，展示了其作为未来视觉模型基础构建模块的潜力。
英文摘要：
We introduce Deformable Convolution v4 (DCNv4), a highly efficient and effective operator designed for a broad spectrum of vision applications. DCNv4 addresses the limitations of its predecessor, DCNv3, with two key enhancements: 1. removing softmax normalization in spatial aggregation to enhance its dynamic property and expressive power and 2. optimizing memory access to minimize redundant operations for speedup. These improvements result in a significantly faster convergence compared to DCNv3 and a substantial increase in processing speed, with DCNv4 achieving more than three times the forward speed. DCNv4 demonstrates exceptional performance across various tasks, including image classification, instance and semantic segmentation, and notably, image generation. When integrated into generative models like U-Net in the latent diffusion model, DCNv4 outperforms its baseline, underscoring its possibility to enhance generative models. In practical applications, replacing DCNv3 with DCNv4 in the InternImage model to create FlashInternImage results in up to 80% speed increase and further performance improvement without further modifications. The advancements in speed and efficiency of DCNv4, combined with its robust performance across diverse vision tasks, show its potential as a foundational building block for future vision models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
