arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.09992cs.CV
收藏

LION:隐式视觉提示微调

LION: Implicit Vision Prompt Tuning

National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心);Huawei Cloud & AI(华为云计算与人工智能);School of Mathematical Sciences, Peking University(北京大学数学科学学院);National Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);Peng Cheng Laboratory(鹏城实验室)

Haixin Wang, Jianlong Chang, Xiao Luo, Jinan Sun, Zhouchen Lin, Qi Tian

AI总结:

针对视觉Transformer计算成本高的问题,提出高效视觉模型LION,通过在冻结的主干网络两端插入平衡隐式层并基于彩票假说剪枝,在减少11.5%训练参数的同时提升了性能和泛化能力。

摘要:

尽管近期在各种视觉任务中取得了具有竞争力的性能,视觉Transformer仍然存在计算成本过高的问题。近期,视觉提示学习为该问题提供了一种经济的解决方案,无需微调整个大规模模型。然而,由于插入了大量提示块和复杂的提示设计,现有模型的效率仍远未达到令人满意的程度。在本文中,我们提出了一种名为impLicit vIsion prOmpt tuNing (LION) 的高效视觉模型,其灵感来源于能够在各种复杂任务中保持稳定内存成本的深度隐式模型。具体而言,我们仅在预训练的主干网络两端插入两个平衡隐式层,并冻结主干网络中的参数。此外,我们根据彩票假说对这两层中的参数进行剪枝。我们的LION在广泛的数据集上取得了令人瞩目的性能。特别是,与最先进的基线VPT相比,我们的LION在减少高达11.5%的训练参数数量的同时获得了更高的性能,尤其是在具有挑战性的场景下。此外,我们发现提出的LION具有良好的泛化性能,使其成为未来促进迁移学习的简便方法。

英文摘要:

Despite recent competitive performance across a range of vision tasks, vision Transformers still have an issue of heavy computational costs. Recently, vision prompt learning has provided an economic solution to this problem without fine-tuning the whole large-scale models. However, the efficiency of existing models are still far from satisfactory due to insertion of extensive prompts blocks and trick prompt designs. In this paper, we propose an efficient vision model named impLicit vIsion prOmpt tuNing (LION), which is motivated by deep implicit models with stable memory costs for various complex tasks. In particular, we merely insect two equilibrium implicit layers in two ends of the pre-trained main backbone with parameters in the backbone frozen. Moreover, we prune the parameters in these two layers according to lottery hypothesis. The performance obtained by our LION are promising on a wide range of datasets. In particular, our LION reduces up to 11.5% of training parameter numbers while obtaining higher performance compared with the state-of-the-art baseline VPT, especially under challenging scenes. Furthermore, we find that our proposed LION had a good generalization performance, making it an easy way to boost transfer learning in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑