通过全局架构因子的高斯过程探索MobileViT的效率导航
Navigating Efficiency in MobileViT through Gaussian Process on Global Architecture Factors
AI总结：
本文利用高斯过程建模MobileViT性能与分辨率、宽度、深度等全局架构因子之间的非线性关系，提出架构缩减公式以迭代生成更小的MobileViT V2，在满足MACs约束下实现超越CNN和移动ViT的精度与效率。
摘要：
大量技术已被精心设计用于为卷积神经网络（CNN）获得最优架构，然而对视觉Transformer（ViT）的类似关注却相对缺乏。尽管ViT在各种视觉任务中取得了显著成功，但其重量级特性带来了计算成本方面的挑战。在本文中，我们利用高斯过程系统地探索MobileViT的性能与全局架构因子之间的非线性和不确定性关系，这些因子包括分辨率、宽度和深度（包括倒置残差块的深度和ViT块的深度），以及分辨率-深度和分辨率-宽度等联合因子。我们提出了扭转全局架构因子神奇4D立方体的设计原则，以在更高模型精度下最小化模型尺寸和计算成本。我们引入了一种架构缩减公式，通过迭代推导出更小的MobileViT V2，同时遵守指定的乘加运算（MACs）约束。实验结果表明，我们的公式在多样化数据集上显著优于CNN和移动ViT。
英文摘要：
Numerous techniques have been meticulously designed to achieve optimal architectures for convolutional neural networks (CNNs), yet a comparable focus on vision transformers (ViTs) has been somewhat lacking. Despite the remarkable success of ViTs in various vision tasks, their heavyweight nature presents challenges of computational costs. In this paper, we leverage the Gaussian process to systematically explore the nonlinear and uncertain relationship between performance and global architecture factors of MobileViT, such as resolution, width, and depth including the depth of in-verted residual blocks and the depth of ViT blocks, and joint factors including resolution-depth and resolution-width. We present design principles twisting magic 4D cube of the global architecture factors that minimize model sizes and computational costs with higher model accuracy. We introduce a formula for downsizing architectures by iteratively deriving smaller MobileViT V2, all while adhering to a specified constraint of multiply-accumulate operations (MACs). Experiment results show that our formula significantly outperforms CNNs and mobile ViTs across diversified datasets
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
