arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.09808cs.CVcs.AI

Famba-V:带有跨层令牌融合的快速视觉Mamba

Famba-V: Fast Vision Mamba with Cross-Layer Token Fusion

The Ohio State University(俄亥俄州立大学)

Hui Shen, Zhongwei Wan, Xin Wang, Mi Zhang

AI总结:

本研究提出Famba-V快速视觉Mamba模型,通过一套跨层策略融合不同层相似令牌而非全层统一融合,在CIFAR-100上验证其可减少Vim模型训练时间与峰值内存,实现更优准确率-效率权衡。

摘要:

Mamba与Vision Mamba(Vim,视觉Mamba)模型已展现出作为Transformer架构基础方法替代方案的潜力。本研究提出Fast Mamba for Vision(Famba-V,快速视觉Mamba),这是一种提升Vim模型训练效率的跨层令牌融合技术。Famba-V的核心思路是基于一套跨层策略,识别并融合不同Vim层中的相似令牌,而非像现有研究提出的那样在所有层统一应用令牌融合。我们在CIFAR-100数据集上评估了Famba-V的性能。结果表明,Famba-V能够通过减少训练时间和训练期间的峰值内存占用,提升Vim模型的训练效率。此外,所提出的跨层策略使Famba-V实现了更优的准确率-效率权衡。这些结果共同证明,Famba-V是一种颇具前景的Vim模型效率增强技术。

英文摘要:

Mamba and Vision Mamba (Vim) models have shown their potential as an alternative to methods based on Transformer architecture. This work introduces Fast Mamba for Vision (Famba-V), a cross-layer token fusion technique to enhance the training efficiency of Vim models. The key idea of Famba-V is to identify and fuse similar tokens across different Vim layers based on a suit of cross-layer strategies instead of simply applying token fusion uniformly across all the layers that existing works propose. We evaluate the performance of Famba-V on CIFAR-100. Our results show that Famba-V is able to enhance the training efficiency of Vim models by reducing both training time and peak memory usage during training. Moreover, the proposed cross-layer strategies allow Famba-V to deliver superior accuracy-efficiency trade-offs. These results all together demonstrate Famba-V as a promising efficiency enhancement technique for Vim models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑