ReGLA：基于门控线性注意力网络的高效感受野建模
ReGLA: Efficient Receptive-Field Modeling with Gated Linear Attention Network
AI总结：
ReGLA通过结合高效卷积与门控线性注意力，实现了高分辨率图像的高效建模，在ImageNet-1K上达到80.85%的Top-1精度，同时保持低延迟和优越的下游任务性能。
摘要：
在高分辨率图像上平衡精度和延迟是轻量模型面临的关键挑战，特别是对于基于Transformer的架构，往往存在过高的延迟。为了解决这个问题，我们引入了ReGLA，一种轻量级混合网络系列，它结合了高效的卷积用于局部特征提取与基于ReLU的门控线性注意力用于全局建模。设计中包含了三个关键创新：高效的大型感受野（ELRF）模块，用于在保持大感受野的同时提升卷积效率；ReLU门控调制注意力（RGMA）模块，用于在保持线性复杂度的同时增强局部特征表示；以及多教师蒸馏策略，以提升下游任务的性能。广泛的实验验证了ReGLA的优越性；特别是ReGLA-M在ImageNet-1K上实现了80.85%的Top-1精度，仅在512px时达到4.98 ms的延迟。此外，ReGLA在下游任务中优于同样规模的iFormer模型，在COCO目标检测中达到3.1%的AP提升，在ADE20K语义分割中达到3.6%的mIoU提升，确立了其在高分辨率视觉应用中的最先进解决方案。
英文摘要：
Balancing accuracy and latency on high-resolution images is a critical challenge for lightweight models, particularly for Transformer-based architectures that often suffer from excessive latency. To address this issue, we introduce \textbf{ReGLA}, a series of lightweight hybrid networks, which integrates efficient convolutions for local feature extraction with ReLU-based gated linear attention for global modeling. The design incorporates three key innovations: the Efficient Large Receptive Field (ELRF) module for enhancing convolutional efficiency while preserving a large receptive field; the ReLU Gated Modulated Attention (RGMA) module for maintaining linear complexity while enhancing local feature representation; and a multi-teacher distillation strategy to boost performance on downstream tasks. Extensive experiments validate the superiority of ReGLA; particularly the ReGLA-M achieves \textbf{80.85\%} Top-1 accuracy on ImageNet-1K at $224px$, with only \textbf{4.98 ms} latency at $512px$. Furthermore, ReGLA outperforms similarly scaled iFormer models in downstream tasks, achieving gains of \textbf{3.1\%} AP on COCO object detection and \textbf{3.6\%} mIoU on ADE20K semantic segmentation, establishing it as a state-of-the-art solution for high-resolution visual applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
