arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.19213eess.SYcs.AIcs.LGcs.NIcs.SY
收藏

EdgeSight:在边缘端实现无模式且成本高效的推理

EdgeSight: Enabling Modeless and Cost-Efficient Inference at the Edge

Harvard University(哈佛大学);Microsoft(微软公司);University of Texas at Austin(德克萨斯大学奥斯汀分校)

ChonLam Lao, Jiaqi Gao, Ganesh Ananthanarayanan, Aditya Akella, Minlan Yu

AI总结:

针对边缘端内存有限、网络不稳、功耗受限的约束,本文提出采用边缘-数据中心架构、置信度缩放及有损推理的EdgeSight系统,实现成本高效的无模式推理,在P99延迟和功耗上均优于现有方案。

摘要:

传统机器学习推理正朝着无模式推理演进,这种模式将模型选择的复杂性对用户屏蔽,允许系统根据准确率和资源需求自动为每个请求选择最合适的模型。已有研究聚焦于数据中心内的无模式推理,本文则解决了边缘端对成本高效的无模式推理的迫切需求——尤其是在边缘设备内存有限、网络状况不稳定以及功耗受限的独特约束下。\n 为克服这些挑战,我们提出了EdgeSight,这是一个在边缘端为各类深度神经网络(DNN)提供成本高效的无模式推理服务的系统。EdgeSight采用边缘-数据中心(edge-DC)架构,利用置信度缩放减少模型选项数量,同时满足多样化的准确率要求。此外,它还支持在不稳定网络环境下的有损推理。我们的实验结果表明,在无模式服务的P99延迟方面,EdgeSight的性能比现有系统最高提升1.6倍。此外,我们的FPGA原型在特定准确率水平下表现出相近的性能,功耗最高降低3.34倍。

英文摘要:

Traditional ML inference is evolving toward modeless inference, which abstracts the complexity of model selection from users, allowing the system to automatically choose the most appropriate model for each request based on accuracy and resource requirements. While prior studies have focused on modeless inference within data centers, this paper tackles the pressing need for cost-efficient modeless inference at the edge -- particularly within its unique constraints of limited device memory, volatile network conditions, and restricted power consumption. To overcome these challenges, we propose EdgeSight, a system that provides cost-efficient EdgeSight serving for diverse DNNs at the edge. EdgeSight employs an edge-data center (edge-DC) architecture, utilizing confidence scaling to reduce the number of model options while meeting diverse accuracy requirements. Additionally, it supports lossy inference in volatile network environments. Our experimental results show that EdgeSight outperforms existing systems by up to 1.6x in P99 latency for modeless services. Furthermore, our FPGA prototype demonstrates similar performance at certain accuracy levels, with a power consumption reduction of up to 3.34x.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑