Early-exit networks reduce inference cost by allowing ``easy'' inputs to stop early, but practical deployment hinges on knowing \emph{when} early exit is safe. We introduce SAFE-KD, a universal multi-exit wrapper for modern vision backbones that couples hierarchical distillation with \emph{conformal risk control}. SAFE-KD attaches lightweight exit heads at intermediate depths, distills a strong teacher into all exits via Decoupled Knowledge Distillation (DKD), and enforces deep-to-shallow consistency between exits. At inference, we calibrate per-exit stopping thresholds on a held-out set using conformal risk control (CRC) to guarantee a user-specified \emph{selective} misclassification risk (among the samples that exit early) under exchangeability. Across multiple datasets and architectures, SAFE-KD yields improved accuracy compute trade-offs, stronger calibration, and robust performance under corruption while providing finite-sample risk guarantees.
Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits
The Hebrew University of Jerusalem, Israel(特拉维夫大学);Bar Ilan University, Israel(巴伊兰大学);Ben-Gurion University of the Negev, Israel(贝纳亚克大学);University of California, Riverside, USA(加州大学河滨分校)
Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
TICLab, International University of Rabat, Morocco(拉巴特国际大学TIC实验室,摩洛哥);College of Computing, University Mohammed VI Polytechnic, Morocco(摩洛哥穆莱·伊斯梅尔Polytechnic大学计算机学院);School of Electronic and Electrical Engineering, University of Leeds, UK(利兹大学电子与电气工程学院)
How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategies
Jagiellonian University(雅盖隆大学);University of Warsaw(华沙大学);Warsaw University of Technology(华沙技术大学);IDEAS Research Institute(IDEAS研究所);Department of Information Engineering, Electronics, and Telecommunications (DIET) at Sapienza, University of Rome, Italy(罗马大学萨皮恩扎信息工程、电子与电信系);Wroclaw University of Science and Technology(波兹南科学与技术大学)
Safe Self-Refinement for Transformer-based Domain Adaptation
Stony Brook University(石溪大学);XPeng Motors(小鹏汽车)
无监督的领域自适应(UDA)旨在利用标签丰富的源域来解决相关未标记目标域上的任务。这是一个具有挑战性的问题,特别是当一个大的域之间的差距位于源和目标域。本文提出了一种新的解决方案SSRT(Safe Self-Refinement for Transformer-based领域自适应),从两个方面进行了改进。首先,由于视觉Transformer在各种视觉任务中的成功,我们为SSRT配备了一个Transformer骨干。我们发现,视觉Transformer与简单的对抗性适应的组合超过了最好的报告卷积神经网络(CNN)的结果,基于具有挑战性的DomainNet基准,显示其强大的可转移特征表示。其次,为了降低模型崩溃的风险,提高知识转移的有效性与大差距的领域之间,我们提出了一个安全的自我完善策略。具体而言,SSRT利用扰动的目标域数据的预测来细化模型。由于视觉Transformer的模型容量很大,并且在这种具有挑战性的任务中的预测可能是有噪声的,因此设计了一种安全的训练机制来自适应地调整学习配置。在几个广泛测试的UDA基准上进行了广泛的评估,SSRT始终获得最佳性能,包括在Bull-Home上的85.43%,在VisDA-2017上的88.76%和在DomainNet上的45.2%。
Department of Computer Science, Control and Management Engineering, Sapienza University of Rome, Rome, Italy(计算机科学与控制管理工程系,罗马萨皮恩扎大学,意大利罗马);Institute of Information Science and Technologies "Alessandro Faedo" - ISTI-CNR, Pisa, Italy(信息科学与技术研究所 "Alessandro Faedo" - ISTI-CNR,意大利比萨);Information Engineering Department, University of Pisa, Pisa, Italy(信息工程系,比萨大学,意大利比萨);CNRS, i3S, Inria(国家科学研究中心 CNRS、i3S、Inria)