MOSEL:使用动态模态选择的推理服务
MOSEL: Inference Serving Using Dynamic Modality Selection
AI总结:
本文提出MOSEL推理服务系统,通过按请求动态选择输入模态,在保证精度前提下将吞吐量提升3.6倍、作业完成时间缩短11倍。
摘要:
多年来的快速发展帮助机器学习模型达到了以往难以实现的目标,有时甚至超越了人类能力。然而,为了达到期望的精度,模型规模及其计算需求急剧增加。因此,尽管近期在构建推理服务系统以及基于输入动态调整模型的算法方法方面已有工作,但要使这些模型的预测服务满足应用的任何目标延迟和成本要求,仍然是一个关键挑战。在本文中,我们引入一种动态形式,即模态选择,在保持模型质量的同时,自适应地从推理输入中选择模态。我们介绍了MOSEL,一个面向多模态机器学习模型的自动化推理服务系统,它根据用户定义的性能和精度要求,为每个请求仔细选择输入模态。MOSEL广泛利用模态配置,在保证精度的情况下将系统吞吐量提高了3.6倍,并将作业完成时间缩短了11倍。
英文摘要:
Rapid advancements over the years have helped machine learning models reach previously hard-to-achieve goals, sometimes even exceeding human capabilities. However, to attain the desired accuracy, the model sizes and in turn their computational requirements have increased drastically. Thus, serving predictions from these models to meet any target latency and cost requirements of applications remains a key challenge, despite recent work in building inference-serving systems as well as algorithmic approaches that dynamically adapt models based on inputs. In this paper, we introduce a form of dynamism, modality selection, where we adaptively choose modalities from inference inputs while maintaining the model quality. We introduce MOSEL, an automated inference serving system for multi-modal ML models that carefully picks input modalities per request based on user-defined performance and accuracy requirements. MOSEL exploits modality configurations extensively, improving system throughput by 3.6$\times$ with an accuracy guarantee and shortening job completion times by 11$\times$.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。