arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.03589cs.CV

视觉非因果梯形曼巴:在具有二阶动力学的视觉状态空间模型中消除定向扫描

Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics

University of Southern California(南加州大学);DEVCOM Army Research Office(国防高级研究计划局陆军研究办公室)

Anvitha Ramachandran, Dhruv Parikh, Haoyang Fan, Rajgopal Kannan, Viktor Prasanna

AI总结:

研究针对视觉状态空间模型中定向扫描的问题,提出非因果梯形曼巴模型,通过二阶非因果建模使图像令牌单步交互,消除定向扫描,提升性能,在多视觉任务中优于其他模型。

摘要:

状态空间模型(SSMs)已成为视觉Transformer的替代方案,但大多数视觉SSMs继承了因果序列建模中的定向令牌扫描。虽然对顺序数据有效,但定向扫描会引入空间偏差和方向敏感表示。我们提出了视觉非因果梯形曼巴(VNCT),这是一种二阶非因果视觉SSM,它使所有图像令牌能够在一次遍历中相互作用,消除定向扫描并实现低单图像推理延迟。VNCT表现出更强的方向鲁棒性表示,在图像旋转和翻转下性能下降减少,同时将边界IoU提高多达3.7个点,从而实现更准确的边界保留和对象定位。在ImageNet-1K分类、COCO对象检测和实例分割以及ADE20K语义分割中,VNCT始终优于定向扫描视觉SSM和一阶非因果SSM。这些结果表明,定向扫描对于高性能视觉SSM是不必要的,并且二阶非因果状态空间建模为视觉识别提供了一种简单、高效且强大的替代方案。

英文摘要:

State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑