Deep Neural Networks (DNNs) have become an essential component in many application domains including web-based services. A variety of these services require high throughput and (close to) real-time features, for instance, to respond or react to users' requests or to process a stream of incoming data on time. However, the trend in DNN design is toward larger models with many layers and parameters to achieve more accurate results. Although these models are often pre-trained, the computational complexity in such large models can still be relatively significant, hindering low inference latency. Implementing a caching mechanism is a typical systems engineering solution for speeding up a service response time. However, traditional caching is often not suitable for DNN-based services. In this paper, we propose an end-to-end automated solution to improve the performance of DNN-based services in terms of their computational complexity and inference latency. Our caching method adopts the ideas of self-distillation of DNN models and early exits. The proposed solution is an automated online layer caching mechanism that allows early exiting of a large model during inference time if the cache model in one of the early exits is confident enough for final prediction. One of the main contributions of this paper is that we have implemented the idea as an online caching, meaning that the cache models do not need access to training data and perform solely based on the incoming data at run-time, making it suitable for applications using pre-trained models. Our experiments results on two downstream tasks (face and object classification) show that, on average, caching can reduce the computational complexity of those services up to 58\% (in terms of FLOPs count) and improve their inference latency up to 46\% with low to zero reduction in accuracy.
PA-Cache: Evolving Learning-Based Popularity-Aware Content Caching in Edge Networks
Chongqing University(重庆大学);Binghamton University, State University of New York(纽约州立大学宾汉姆顿大学);Swinburne University of Technology(斯威本科技大学);Harbin Institute of Technology(哈尔滨工业大学)
随着无处不在和个性化服务的蓬勃发展,大量移动设备在网络上产生了越来越多的流量。因此,内容缓存逐渐延伸到网络边缘,以提供低延迟服务,提高服务质量,并减少冗余数据流量。与传统的内容分发网络相比,边缘网络中具有较小尺寸的高速缓存通常必须容纳更多突发请求。在本文中,我们提出了一个不断发展的学习为基础的内容缓存策略,命名为PA缓存在边缘网络。它自适应地学习随时间变化的内容流行度,并确定当缓存已满时应该替换哪些内容。与传统的深度神经网络(DNN)不同,DNN使用具有高计算复杂性的整个训练数据集来学习微调但可能过时或有偏见的预测模型,PA-Cache对大量内容特征集进行加权,并在随着时间的推移有更多请求到达时从浅到深地训练多层递归神经网络。我们广泛评估我们提出的PA缓存的性能从一个大型的在线视频点播服务提供商在现实世界中的痕迹。\rb{The results show that PA-Cache outperforms existing popular caching algorithms and approximates the optimal algorithm with only a 3.8\% performance gap when the cache percentage is 1.0\%}。与传统的基于DNN的方法相比,PA-Cache还显着降低了计算成本。
Automated machine learning for borehole resistivity measurements
Software Competence Center Hagenberg GmbH (SCCH)(哈根伯格软件能力中心有限公司);University of the Basque Country (UPV/EHU)(巴斯克大学);Ikerbasque (Basque Foundation for Sciences)(伊克尔巴斯克基金会(巴斯克科学基金会))
ViT-CAT: Parallel Vision Transformers with Cross Attention Fusion for Popularity Prediction in MEC Networks
Concordia University(康考迪亚大学);Concordia Institute of Information Systems Engineering (CIISE)(康考迪亚信息系统工程研究院(CIISE));Defence Research and Development Canada (DRDC)(加拿大国防研究与发展部(DRDC));Yazd University(亚兹德大学);University of Toronto(多伦多大学)
Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
Aristotle University of Thessaloniki(亚里士多德大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);Southern Illinois University Carbondale(南伊利诺伊大学卡本代尔分校);University of Patras(帕特雷大学)
ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction
Massachusetts Institute of Technology(麻省理工学院);University of California, San Diego(加州大学圣地亚哥分校);MIT-IBM Watson AI Lab(麻省理工-IBM沃森人工智能实验室);IBM Research(IBM研究院);Thomas J. Watson Research Center(托马斯·J·沃森研究中心)
Rethinking the Potential of Layer Freezing for Efficient DNN Training
University of Georgia(佐治亚大学);Northeastern University(东北大学);University of Pittsburgh(匹兹堡大学);The University of Arizona(亚利桑那大学);Stevens Institute of Technology(斯蒂文斯理工学院);The University of North Carolina at Charlotte(北卡罗来纳大学教堂山分校);University of Virginia(弗吉尼亚大学)
DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
Nanyang Technological University(南洋理工大学);Harbin Institute of Technology(哈尔滨工业大学);Carnegie Mellon University(卡内基梅隆大学);University of Illinois at Urbana–Champaign(伊利诺伊大学厄巴纳-香槟分校);Kyushu University(九州大学);NVIDIA AI Technology Center(英伟达人工智能技术中心)