arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.08355cs.LG
收藏

面向可靠科学传感量化机器学习模型的损失景观分析

Loss Landscape Analysis for Reliable Quantized ML Models for Scientific Sensing

Scuola Superiore Sant’Anna(圣安娜高等学校);Fermi National Accelerator Laboratory(费米国家加速器实验室);University of California San Diego(加利福尼亚大学圣迭戈分校);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室)

Tommaso Baldi, Javier Campos, Olivia Weng, Caleb Geniesse, Nhan Tran, Ryan Kastner, Alessandro Biondi

AI总结:

本文提出针对科学传感量化ML模型的损失景观实证分析方法,可评估量化精度和正则化技术下模型对噪声扰动的鲁棒性,发现平缓景观与鲁棒性的强相关性,无需多模型训练即可探索权衡,助力帕累托优化构建可靠系统。

摘要:

本文提出一种对机器学习(ML)模型损失景观进行实证分析的方法。该方法应用于两个科学传感用ML模型,这些模型需经量化才能部署,且会因实验条件受噪声和扰动影响。我们的方法可评估ML模型对上述影响的鲁棒性,将其作为量化精度的函数,并在不同正则化技术下分析——这是两个迄今仍未被充分探索的关键问题。通过损失景观分析研究性能、效率与鲁棒性的相互作用,我们既证实了平缓形状的景观与对输入和权重扰动的鲁棒性之间存在强相关性,也观察到其他有趣且非显而易见的现象。我们的方法可预先系统探索此类权衡,即无需训练和测试多个模型,从而实现更高效的开发流程。本工作还强调了将鲁棒性纳入ML模型帕累托优化的重要性,助力构建更可靠、自适应的科学传感系统。

英文摘要:

In this paper, we propose a method to perform empirical analysis of the loss landscape of machine learning (ML) models. The method is applied to two ML models for scientific sensing, which necessitates quantization to be deployed and are subject to noise and perturbations due to experimental conditions. Our method allows assessing the robustness of ML models to such effects as a function of quantization precision and under different regularization techniques -- two crucial concerns that remained underexplored so far. By investigating the interplay between performance, efficiency, and robustness by means of loss landscape analysis, we both established a strong correlation between gently-shaped landscapes and robustness to input and weight perturbations and observed other intriguing and non-obvious phenomena. Our method allows a systematic exploration of such trade-offs a priori, i.e., without training and testing multiple models, leading to more efficient development workflows. This work also highlights the importance of incorporating robustness into the Pareto optimization of ML models, enabling more reliable and adaptive scientific sensing systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑