NAF：通过邻域注意力过滤实现零样本特征上采样
NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering
AI总结：
NAF通过邻域注意力过滤实现零样本特征上采样，无需重新训练即可超越VFM特定上采样器，在多个下游任务中取得最佳性能。
摘要：
视觉基础模型（VFMs）提取空间下采样的表示，对像素级任务提出了挑战。现有的上采样方法面临根本性的权衡：经典滤波器速度快且应用广泛，但依赖固定的结构；而现代上采样器通过可学习的、针对VFM特定的结构实现更高的精度，但需要为每个VFM重新训练。我们引入了邻域注意力过滤（NAF），通过交叉尺度邻域注意力和旋转位置嵌入（RoPE）学习自适应的空间和内容权重，仅依靠高分辨率输入图像进行指导。NAF实现零样本上采样：它可以从任何VFM上采样特征而无需重新训练，使其成为首个超越VFM特定上采样器、并在多个下游任务中实现最先进性能的VFM无关架构。它保持了高效率，可扩展到2K特征图，并在18 FPS下重建中间分辨率图。除了特征上采样，NAF在图像修复任务中也表现出色，突显了其通用性。代码和检查点可在https://github.com/valeoai/NAF获取。
英文摘要：
Vision Foundation Models (VFMs) extract spatially downsampled representations, posing challenges for pixel-level tasks. Existing upsampling approaches face a fundamental trade-off: classical filters are fast and broadly applicable but rely on fixed forms, while modern upsamplers achieve superior accuracy through learnable, VFM-specific forms at the cost of retraining for each VFM. We introduce Neighborhood Attention Filtering (NAF), which bridges this gap by learning adaptive spatial-and-content weights through Cross-Scale Neighborhood Attention and Rotary Position Embeddings (RoPE), guided solely by the high-resolution input image. NAF operates zero-shot: it upsamples features from any VFM without retraining, making it the first VFM-agnostic architecture to outperform VFM-specific upsamplers and achieve state-of-the-art performance across multiple downstream tasks. It maintains high efficiency, scaling to 2K feature maps and reconstructing intermediate-resolution maps at 18 FPS. Beyond feature upsampling, NAF demonstrates strong performance on image restoration, highlighting its versatility. Code and checkpoints are available at https://github.com/valeoai/NAF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
