面向Transformer高分辨率图像分割的自适应分块方法
Adaptive Patching for High-resolution Image Segmentation with Transformers
AI总结：
针对高分辨率图像分割中Transformer小patch计算内存开销过高的问题，提出自适应分块预处理方法，依据图像细节减少patch数量，兼容任意注意力模型，在病理数据集上实现更优分割效果与6.9倍几何平均加速。
摘要：
基于注意力的模型在包括分割在内的图像分析领域正快速普及。将图像输入Transformer编码器的标准方法是将图像划分为多个图像块（patch），再将这些块作为线性token序列输入模型。对于高分辨率图像（例如显微病理图像），若要使用分割任务中更具优势的小尺寸图像块，注意力模型的二次方计算与内存开销会导致其无法应用。现有解决方案要么采用定制化的复杂多分辨率模型，要么使用近似注意力机制。我们受高性能计算（HPC）中的自适应网格细化（Adaptive Mesh Refinement, AMR）方法启发，提出将自适应图像分块作为预处理步骤，根据图像细节程度减少输入模型的图像块数量，降幅可达数个数量级。该方法的额外开销可忽略不计，且能与任意基于注意力的模型无缝兼容，即作为预处理步骤可被任意注意力模型无成本采用。我们在真实病理数据集上验证了该方法的分割质量优于当前最优（SoTA）分割模型，同时在最高64K²分辨率、最多2048块GPU的设置下，几何平均加速比达6.9倍。
英文摘要：
Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for real-world pathology datasets while gaining a geomean speedup of $6.9\times$ for resolutions up to $64K^2$, on up to $2,048$ GPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
