ViM：用于统一下游传输的Vision中间件
ViM: Vision Middleware for Unified Downstream Transferring
摘要：
基础模型在海量数据上进行预训练，并通过微调转移到下游任务。这项工作提出了视觉中间件（ViM），一个新的学习范式，目标是统一从一个单一的基础模型转移到各种下游任务。ViM由一系列轻量级插件模块组成，每个模块都是在具有共享冻结主干的中游数据集上独立学习的。由于从中游任务继承了丰富的知识，下游任务可以从模块动物园的充分聚合中受益。这种设计有三大优点。从效率方面来看，上游骨干网只需训练一次，就可以在不进行调优的情况下被所有下游任务重用。从可伸缩性方面来看，我们可以很容易地将额外的模块添加到ViM中，而不会影响现有的模块。从性能方面来看，ViM可以包含尽可能多的中游任务，缩小上游和下游之间的任务差距。考虑到这些好处，我们相信，社区可以共同维护和开发的ViM将成为帮助基础模型的强大工具。
英文摘要：
Foundation models are pre-trained on massive data and transferred to downstream tasks via fine-tuning. This work presents Vision Middleware (ViM), a new learning paradigm that targets unified transferring from a single foundation model to a variety of downstream tasks. ViM consists of a zoo of lightweight plug-in modules, each of which is independently learned on a midstream dataset with a shared frozen backbone. Downstream tasks can then benefit from an adequate aggregation of the module zoo thanks to the rich knowledge inherited from midstream tasks. There are three major advantages of such a design. From the efficiency aspect, the upstream backbone can be trained only once and reused for all downstream tasks without tuning. From the scalability aspect, we can easily append additional modules to ViM with no influence on existing modules. From the performance aspect, ViM can include as many midstream tasks as possible, narrowing the task gap between upstream and downstream. Considering these benefits, we believe that ViM, which the community could maintain and develop together, would serve as a powerful tool to assist foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
