Along with the rapid development in the field of artificial intelligence, especially deep learning, deep neural network applications are becoming more and more popular in reality. To be able to withstand the heavy load from mainstream users, deployment techniques are essential in bringing neural network models from research to production. Among the two popular computing topologies for deploying neural network models in production are cloud-computing and edge-computing. Recent advances in communication technologies, along with the great increase in the number of mobile devices, has made edge-computing gradually become an inevitable trend. In this paper, we propose an architecture to solve deploying and processing deep neural networks on edge-devices by leveraging their synergy with the cloud and the access-control mechanisms of the database. Adopting this architecture allows low-latency DNN model updates on devices. At the same time, with only one model deployed, we can easily make different versions of it by setting access permissions on the model weights. This method allows for dynamic model licensing, which benefits commercial applications.
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment
Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所);FAIR(FAIR(人工智能基础研究团队));Allen Institute for Artificial Intelligence(艾伦人工智能研究所)
Microservice Deployment in Space Computing Power Networks via Robust Reinforcement Learning
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);EPFL(洛桑联邦理工学院);Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
Towards Tensor Network Models for Low-Latency Jet Tagging on FPGAs
Dipartimento di Fisica e Astronomia "Galileo Galilei", University of Padua(伽利略·伽利莱物理与天文学系,帕多瓦大学);PlanQC GmbH(PlanQC公司);Institute for Complex Quantum Systems, Ulm University(复杂量子系统研究所,乌尔姆大学);European Organization for Nuclear Research (CERN)(欧洲核子研究中心);Padua Quantum Technologies Research Center, University of Padua(帕多瓦量子技术研究中心,帕多瓦大学)
AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environments
Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));Shanghai AI Laboratory(上海人工智能实验室);Shanghai University(上海大学);Microsoft Research(微软研究院);AsiaInfo Technologies (China), Inc.(亚信科技(中国)有限公司)
Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与技术学院);Department of Computer Science and Engineering, Chinese University of Hong Kong(香港中文大学计算机科学与工程系);School of Integrated Circuits, Sun Yat-sen University(中山大学集成电路学院)
本文提出 BMT 二值化 Transformer、BAT 加速器及设计空间联合优化方法,在精度、延迟和鲁棒性间取得平衡,并显著提升边缘部署的吞吐量与能效。