arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.16394cs.LG
收藏

Transformer^-1:面向资源受限部署的输入自适应计算

Transformer^-1: Input-Adaptive Computation for Resource-Constrained Deployment

Lumen AI;Tengzhou No. 1 Middle School(滕州第一中学)

Lumen AI, Tengzhou No. 1 Middle School, Shihao Ji, Zihui Song, Fucheng Zhong, Jisen Jia, Zhaobo Wu, Zheyi Cao, Xu Tianhao

AI总结:

针对动态场景下固定计算范式导致的资源浪费问题,提出基于深度自适应的Transformer^-1架构,通过双层控制机制、下界理论及层折叠等技术,在保持准确率的同时显著降低FLOPs和内存,实现资源受限环境下的高效部署。

摘要:

针对深度学习模型在动态场景下因固定计算范式导致的资源浪费问题,本文提出了一种基于深度自适应原则的Transformer^-1架构。该架构通过建立复杂度与计算的联合优化模型,实现了输入特征与计算资源之间的动态匹配。我们的核心贡献包括:(1)设计了一种双层控制机制,由复杂度预测器和强化学习策略网络组成,实现了计算路径的端到端优化;(2)推导了动态计算的下界理论,证明了系统在理论上能达到最优效率;(3)提出了一种层折叠技术和CUDA Graph预编译方案,克服了动态架构的工程瓶颈。在ImageNet-1K基准测试中,与标准Transformer相比,我们的方法在保持相当准确率(±0.3%)的同时,将FLOPs减少了42.7%,峰值内存使用量降低了34.1%。此外,我们还在Jetson AGX Xavier平台上进行了实际部署,验证了该方法在资源受限环境中的有效性和实用价值。为了进一步验证该方法的通用性,我们还在多个自然语言处理任务上进行了实验,并在资源效率方面取得了显著提升。

英文摘要:

Addressing the resource waste caused by fixed computation paradigms in deep learning models under dynamic scenarios, this paper proposes a Transformer$^{-1}$ architecture based on the principle of deep adaptivity. This architecture achieves dynamic matching between input features and computational resources by establishing a joint optimization model for complexity and computation. Our core contributions include: (1) designing a two-layer control mechanism, composed of a complexity predictor and a reinforcement learning policy network, enabling end-to-end optimization of computation paths; (2) deriving a lower bound theory for dynamic computation, proving the system's theoretical reach to optimal efficiency; and (3) proposing a layer folding technique and a CUDA Graph pre-compilation scheme, overcoming the engineering bottlenecks of dynamic architectures. In the ImageNet-1K benchmark test, our method reduces FLOPs by 42.7\% and peak memory usage by 34.1\% compared to the standard Transformer, while maintaining comparable accuracy ($\pm$0.3\%). Furthermore, we conducted practical deployment on the Jetson AGX Xavier platform, verifying the effectiveness and practical value of this method in resource-constrained environments. To further validate the generality of the method, we also conducted experiments on several natural language processing tasks and achieved significant improvements in resource efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑