跨边缘设备表征和驯服模型不稳定性
Characterizing and Taming Model Instability Across Edge Devices
摘要:
在不同边缘设备上运行的相同机器学习模型可能会在几乎相同的输入上产生高度发散的输出。差异的可能原因包括器械传感器、器械信号处理硬件和软件以及操作系统和处理器的差异。本文提出了第一个方法表征的变化模型预测在现实世界中的移动设备。我们证明,准确性是不是一个有用的指标来表征预测的分歧,并引入一个新的度量,不稳定性,它捕捉这种变化。我们表征了不稳定的不同来源,并表明压缩格式和图像信号处理的差异占对象分类模型的显着不稳定性。值得注意的是,在我们的实验中,14-17%的图像在一个或多个手机型号上产生了不同的分类。我们评估三种不同的技术,以减少不稳定性。特别是,我们调整了先前的工作,使模型对噪声具有鲁棒性,以便微调模型,使其对边缘设备的变化具有鲁棒性。我们证明了我们的微调技术减少了75%的不稳定性。
英文摘要:
The same machine learning model running on different edge devices may produce highly-divergent outputs on a nearly-identical input. Possible reasons for the divergence include differences in the device sensors, the device's signal processing hardware and software, and its operating system and processors. This paper presents the first methodical characterization of the variations in model prediction across real-world mobile devices. We demonstrate that accuracy is not a useful metric to characterize prediction divergence, and introduce a new metric, instability, which captures this variation. We characterize different sources for instability, and show that differences in compression formats and image signal processing account for significant instability in object classification models. Notably, in our experiments, 14-17% of images produced divergent classifications across one or more phone models. We evaluate three different techniques for reducing instability. In particular, we adapt prior work on making models robust to noise in order to fine-tune models to be robust to variations across edge devices. We demonstrate our fine-tuning techniques reduce instability by 75%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。