arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.11449cs.AI
收藏

通过模型标签学习使小模型具备零样本选择与重用能力

Enabling Small Models for Zero-Shot Selection and Reuse through Model Label Learning

Nanjing University(南京大学)

Jia Zhang, Zhi Zhou, Lan-Zhe Guo, Yu-Feng Li

AI总结:

本文提出模型标签学习(MLL)新范式,通过语义有向无环图和分类头组合优化算法,在模型中心中有效选择和重用专家模型,实现低成本、高可扩展的零样本任务解决能力。

摘要:

像CLIP这样的视觉语言模型(VLMs)通过文本与图像对齐在图像分类任务中展现了令人印象深刻的零样本能力,但与特定任务的专家模型相比性能较差。相反,专家模型在其专业领域表现优异,但缺乏对新任务的零样本能力。如何同时获得专家模型的高性能和零样本能力是一个重要的研究方向。本文试图证明,通过构建模型中心并使用模型标签将模型与其功能对齐,可以通过有效选择和重用中心中的模型以零样本方式解决新任务。我们引入了一种新范式——模型标签学习(MLL),它通过语义有向无环图(SDAG)弥合模型与其功能之间的差距,并利用分类头组合优化(CHCO)算法为新任务选择有能力的模型。与基础模型范式相比,它成本更低且更具可扩展性,即零样本能力随模型中心规模的增长而增强。在七个真实世界数据集上的实验验证了MLL的有效性和效率,表明专家模型可以有效地被重用于零样本任务。我们的代码将公开发布。

英文摘要:

Vision-language models (VLMs) like CLIP have demonstrated impressive zero-shot ability in image classification tasks by aligning text and images but suffer inferior performance compared with task-specific expert models. On the contrary, expert models excel in their specialized domains but lack zero-shot ability for new tasks. How to obtain both the high performance of expert models and zero-shot ability is an important research direction. In this paper, we attempt to demonstrate that by constructing a model hub and aligning models with their functionalities using model labels, new tasks can be solved in a zero-shot manner by effectively selecting and reusing models in the hub. We introduce a novel paradigm, Model Label Learning (MLL), which bridges the gap between models and their functionalities through a Semantic Directed Acyclic Graph (SDAG) and leverages an algorithm, Classification Head Combination Optimization (CHCO), to select capable models for new tasks. Compared with the foundation model paradigm, it is less costly and more scalable, i.e., the zero-shot ability grows with the sizes of the model hub. Experiments on seven real-world datasets validate the effectiveness and efficiency of MLL, demonstrating that expert models can be effectively reused for zero-shot tasks. Our code will be released publicly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑