Deep neural networks have become larger over the years with increasing demand of computational resources for inference; incurring exacerbate costs and leaving little room for deployment on devices with limited battery and other resources for real-time applications. The multi-exit architectures are type of deep neural network that are interleaved with several output (or exit) layers at varying depths of the model. They provide a sound approach for improving computational time and energy utilization of running a model through producing predictions from early exits. In this work, we present a novel and architecture-agnostic approach for robust training of multi-exit architectures termed consistent exit training. The crux of the method lies in a consistency-based objective to enforce prediction invariance over clean and perturbed inputs. We leverage weak supervision to align model output with consistency training and jointly optimize dual-losses in a multi-task learning fashion over the exits in a network. Our technique enables exit layers to generalize better when confronted with increasing uncertainty, hence, resulting in superior quality-efficiency trade-offs. We demonstrate through extensive evaluation on challenging learning tasks involving sensor data that our approach allows examples to exit earlier with better detection rate and without executing all the layers in a deep model.
Temporal Decisions: Leveraging Temporal Correlation for Efficient Decisions in Early Exit Neural Networks
Infineon Technologies Dresden GmbH & Co. KG（英飞凌德累斯顿有限两合公司）；Technical University of Munich（慕尼黑工业大学）；Infineon Technologies AG（英飞凌股份公司）；Technical University of Dresden（德累斯顿工业大学）
Super Low Resolution RF Powered Accelerometers for Alerting on Hospitalized Patient Bed Exits
Auto-ID Lab, School of Computer Science, The University of Adelaide（阿德莱德大学计算机科学学院Auto-ID实验室）；The Queen Elizabeth Hospital（伊丽莎白女王医院）；School of Electrical and Electronic Engineering, The University of Adelaide（阿德莱德大学电气与电子工程学院）；School of Electrical Engineering and Computer Science, University of Michigan（密歇根大学电气与计算机科学学院）
Graph-Aware Contrasting for Multivariate Time-Series Classification
Institute for Infocomm Research, A*STAR（新加坡A*STAR资讯通信研究院）；Centre for Frontier AI Research, A*STAR（新加坡A*STAR前沿人工智能研究中心）；Nanyang Technological University（南洋理工大学）
Early Exiting Predictive Coding Neural Networks for Edge AI
TICLab, International University of Rabat, Morocco（TICLab，拉巴特国际大学，摩洛哥）；College of Computing, University Mohammed VI Polytechnic, Morocco（计算机学院，穆罕默德六世理工大学，摩洛哥）；School of Electronic and Electrical Engineering, University of Leeds, UK（电子与电气工程学院，利兹大学，英国）
Temporal Patience: Efficient Adaptive Deep Learning for Embedded Radar Data Processing
Infineon Technologies Dresden GmbH & Co. KG（英飞凌科技德累斯顿有限两合公司）；Technical University of Dresden（德累斯顿工业大学）；Infineon Technologies AG（英飞凌科技股份公司）；Technical University of Munich（慕尼黑工业大学）
Cross-Validation and Uncertainty Determination for Randomized Neural Networks with Applications to Mobile Sensors
Institute of Statistics, RWTH Aachen University（亚琛工业大学统计研究所）；Forschungszentrum Jülich, Institut für Energie- und Klimaforschung, IEK-5（于利希研究中心能源与气候研究所 IEK-5）
