arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.13883cs.AI
收藏

MLink:连接来自多个领域的黑盒模型以实现协同推理

MLink: Linking Black-Box Models from Multiple Domains for Collaborative Inference

University of Science and Technology of China(中国科学技术大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Huawei Cloud(华为云)

Mu Yuan, Lan Zhang, Zimu Zheng, Yi-Nan Zhang, Xiang-Yang Li

AI总结:

本文提出模型连接学习任务及MLink调度算法,通过连接异构黑盒模型实现协同推理,在GPU内存预算下显著降低计算量并保持高准确率。

摘要:

模型推理的成本效率对于现实世界的机器学习(ML)应用至关重要,尤其是对于延迟敏感型任务和资源受限设备。一个典型的困境是:为了提供复杂的智能服务(例如智慧城市),我们需要多个ML模型的推理结果,但成本预算(例如GPU内存)不足以运行所有这些模型。在本工作中,我们研究黑盒ML模型之间的潜在关系,并提出了一个新颖的学习任务:模型连接,其旨在通过学习不同黑盒模型输出空间之间的映射(称为模型连接)来桥接它们的知识。我们提出了支持连接异构黑盒ML模型的模型连接设计。此外,为了解决分布差异挑战,我们提出了模型连接的适配和聚合方法。基于我们提出的模型连接,我们开发了一个名为MLink的调度算法。通过模型连接实现的多模型协同推理,MLink可以在成本预算下提高所获得推理结果的准确率。我们在包含七个不同ML模型的多模态数据集以及包含六个ML模型和3,264小时视频的两个现实世界视频分析系统上评估了MLink。实验结果表明,我们提出的模型连接可以在各种黑盒模型之间有效构建。在GPU内存预算下,MLink可以节省66.7%的推理计算,同时保持94%的推理准确率,这优于多任务学习、基于深度强化学习的调度器和帧过滤基线。

英文摘要:

The cost efficiency of model inference is critical to real-world machine learning (ML) applications, especially for delay-sensitive tasks and resource-limited devices. A typical dilemma is: in order to provide complex intelligent services (e.g. smart city), we need inference results of multiple ML models, but the cost budget (e.g. GPU memory) is not enough to run all of them. In this work, we study underlying relationships among black-box ML models and propose a novel learning task: model linking, which aims to bridge the knowledge of different black-box models by learning mappings (dubbed model links) between their output spaces. We propose the design of model links which supports linking heterogeneous black-box ML models. Also, in order to address the distribution discrepancy challenge, we present adaptation and aggregation methods of model links. Based on our proposed model links, we developed a scheduling algorithm, named MLink. Through collaborative multi-model inference enabled by model links, MLink can improve the accuracy of obtained inference results under the cost budget. We evaluated MLink on a multi-modal dataset with seven different ML models and two real-world video analytics systems with six ML models and 3,264 hours of video. Experimental results show that our proposed model links can be effectively built among various black-box models. Under the budget of GPU memory, MLink can save 66.7% inference computations while preserving 94% inference accuracy, which outperforms multi-task learning, deep reinforcement learning-based scheduler and frame filtering baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑