MirrorLA: 用于视觉线性注意力的特征图反射
MirrorLA: Reflecting Feature Map for Vision Linear Attention
AI总结：
MirrorLA通过几何框架和可学习反射实现线性注意力的非负特征图重定向，提升性能并保持表示忠实性。
摘要：
线性注意力显著降低了Transformer的计算复杂度，从二次降至线性，但其性能始终劣于基于softmax的注意力。我们发现这种退化的原因是核特征图上的非负性约束：标准投影如ReLU充当"被动截断"运算符，无差别地丢弃负域中的语义信息。我们提出了MirrorLA，一个几何框架，将被动截断替换为主动重定向。通过利用可学习的Householder反射，MirrorLA将特征几何旋转到非负正交域以最大化信息保留。我们的方法通过一致的多尺度设计恢复表示密度：首先通过块状等距性优化局部可区分性，利用方差感知调制稳定长上下文动态以多样化激活，最后通过跨头反射整合分散的子空间以诱导全局协方差混合。MirrorLA在标准基准上实现了最先进的性能，证明了严格线性效率可以在不牺牲表示忠实性的情况下实现。
英文摘要：
Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance. We identify the root cause of this degradation as the non-negativity constraint imposed on kernel feature maps: standard projections like ReLU act as "passive truncation" operators, indiscriminately discarding semantic information residing in the negative domain. We propose MirrorLA, a geometric framework that substitutes passive truncation with active reorientation. By leveraging learnable Householder reflections, MirrorLA rotates the feature geometry into the non-negative orthant to maximize information retention. Our approach restores representational density through a cohesive, multi-scale design: it first optimizes local discriminability via block-wise isometries, stabilizes long-context dynamics using variance-aware modulation to diversify activations, and finally, integrates dispersed subspaces via cross-head reflections to induce global covariance mixing. MirrorLA achieves state-of-the-art performance across standard benchmarks, demonstrating that strictly linear efficiency can be achieved without compromising representational fidelity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
