基于桥模式连接的上下文自适应深度神经网络
Context-Adaptive Deep Neural Networks via Bridge-Mode Connectivity
摘要:
机器学习模型在安全关键应用中的部署伴随着这样的模型将在一系列上下文(例如,用于对街道标志进行分类的视觉模型应当在变化的照明/天气条件下在农村、城市和高速公路设置中工作)。然而,这些通用模型通常针对平均情况性能进行了优化,鼓励它们在标称条件下实现高性能,但在具有挑战性或罕见的环境中暴露于意外行为。为了解决这个问题,我们开发了一种新的方法来训练上下文相关的模型。我们扩展了桥接模式连接(BMC)(Garipov等人,2018)来训练在连续的上下文测量上的模型的无限集合,使得我们可以对专门调整到相应的评估上下文的模型参数进行采样。我们通过多个镜头探索图像分类任务中上下文的定义,包括风险特征的变化、长尾图像统计/外观和上下文相关的分布偏移。我们开发了新的扩展BMC优化这些情况下,我们的实验表明,模型的性能可以成功地调整到每个场景中的上下文。
英文摘要:
The deployment of machine learning models in safety-critical applications comes with the expectation that such models will perform well over a range of contexts (e.g., a vision model for classifying street signs should work in rural, city, and highway settings under varying lighting/weather conditions). However, these one-size-fits-all models are typically optimized for average case performance, encouraging them to achieve high performance in nominal conditions but exposing them to unexpected behavior in challenging or rare contexts. To address this concern, we develop a new method for training context-dependent models. We extend Bridge-Mode Connectivity (BMC) (Garipov et al., 2018) to train an infinite ensemble of models over a continuous measure of context such that we can sample model parameters specifically tuned to the corresponding evaluation context. We explore the definition of context in image classification tasks through multiple lenses including changes in the risk profile, long-tail image statistics/appearance, and context-dependent distribution shift. We develop novel extensions of the BMC optimization for each of these cases and our experiments demonstrate that model performance can be successfully tuned to context in each scenario.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。