MambaEye：一种无尺寸依赖的视觉编码器与因果序列处理
MambaEye: A Size-Agnostic Visual Encoder with Causal Sequential Processing
AI总结：
MambaEye通过因果序列处理和相对移动嵌入，实现无尺寸依赖的视觉编码，适用于任意分辨率和扫描模式。
摘要：
尽管数十年来取得了显著进展，但真正输入尺寸无关的视觉编码器——人类视觉的基本特征——仍然难以实现。我们通过提出MambaEye，一种新颖的因果序列编码器，来解决这一限制。该编码器利用低复杂度且基于因果过程的纯Mamba2骨干网络。与以往基于Mamba的视觉编码器通常采用双向处理不同，我们的严格单向方法保留了状态空间模型的内在因果性，使模型能够在输入序列的任意点生成预测。核心创新是我们使用相对移动嵌入，该嵌入编码连续块之间的空间位移，为平移不变性提供强归纳偏差，使模型本质上能够适应任意图像分辨率和扫描模式。为此，我们引入了一种新的扩散启发式损失函数，提供密集、逐步的监督，训练模型在收集更多视觉证据时逐步建立信心。我们证明MambaEye在广泛的图像分辨率上表现出稳健的性能，尤其是在ImageNet-1K分类任务中更高的分辨率如1536²上。这一成就是在保持线性时间与内存复杂度相对于块数的情况下实现的。
英文摘要：
Despite decades of progress, a truly input-size agnostic visual encoder-a fundamental characteristic of human vision-has remained elusive. We address this limitation by proposing \textbf{MambaEye}, a novel, causal sequential encoder that leverages the low complexity and causal-process based pure Mamba2 backbone. Unlike previous Mamba-based vision encoders that often employ bidirectional processing, our strictly unidirectional approach preserves the inherent causality of State Space Models, enabling the model to generate a prediction at any point in its input sequence. A core innovation is our use of relative move embedding, which encodes the spatial shift between consecutive patches, providing a strong inductive bias for translation invariance and making the model inherently adaptable to arbitrary image resolutions and scanning patterns. To achieve this, we introduce a novel diffusion-inspired loss function that provides dense, step-wise supervision, training the model to build confidence as it gathers more visual evidence. We demonstrate that MambaEye exhibits robust performance across a wide range of image resolutions, especially at higher resolutions such as $1536^2$ on the ImageNet-1K classification task. This feat is achieved while maintaining linear time and memory complexity relative to the number of patches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
