Monitoring the performance of classification models in production is critical yet challenging due to strict labeling budgets, one-shot batch acquisition of labels and extremely low error rates. We propose a general framework based on Stratified Importance Sampling (SIS) that directly addresses these constraints in model monitoring. While SIS has previously been applied in specialized domains, our theoretical analysis establishes its broad applicability to the monitoring of classification models. Under mild conditions, SIS yields unbiased estimators with strict finite-sample mean squared error (MSE) improvements over both importance sampling (IS) and stratified random sampling (SRS). The framework does not rely on optimally defined proposal distributions or strata: even with noisy proxies and sub-optimal stratification, SIS can improve estimator efficiency compared to IS or SRS individually, though extreme proposal mismatch may limit these gains. Experiments across binary and multiclass tasks demonstrate consistent efficiency improvements under fixed label budgets, underscoring SIS as a principled, label-efficient, and operationally lightweight methodology for post-deployment model monitoring.
Stratified Transfer Learning for Cross-domain Activity Recognition
Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所移动计算与普适设备北京市重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Information Technology, Hebei University of Economics and Business（河北经贸大学信息技术学院）；Department of Computer Science, University of Illinois at Chicago（伊利诺伊大学芝加哥分校计算机科学系）
Incremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention
Siemens AG（西门子公司）；Goethe University Frankfurt（法兰克福歌德大学）；LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；German Cancer Consortium (DKTK)（德国癌症联盟）
Monitoring Object Detection Abnormalities via Data-Label and Post-Algorithm Abstractions
Technology Center of Software Engineering, ISCAS, China（中国科学院软件研究所软件工程技术研究中心）；University of Chinese Academy of Sciences, China（中国科学院大学）；State Key Laboratory of Computer Science, ISCAS, China（中国科学院软件研究所计算机科学国家重点实验室）；DENSO AUTOMOTIVE Deutschland GmbH, Germany（电装汽车德国有限公司）
Federated Learning with Heterogeneous Labels and Models for Mobile Activity Monitoring
Global AI Accelerator, Ericsson（爱立信全球人工智能加速器）；Ericsson Research（爱立信研究院）
各种医疗保健应用，如辅助生活、跌倒检测等，需要通过人类活动识别（HAR）对用户行为进行建模。这样的应用需要使用机器学习技术来表征来自多个资源受限的用户设备的见解，以用于有效的个性化活动监视。基于设备的联邦学习被证明是分布式协作机器学习的一种有效方法。然而，在解决用户之间的统计（非IID数据）和模型异质性方面存在各种挑战。此外，在本文中，我们探索了一个新的挑战-在联邦学习过程中处理用户之间标签（活动）的异质性。为此，我们提出了一个框架，联邦标签为基础的聚合，利用重叠的信息增益跨活动使用模型蒸馏更新。我们还建议，模型分数的联邦传输是足够的，而不是从设备到服务器的模型权重传输。在Raspberry Pi 2上使用Heteroimmune Human Activity Recognition（HHAR）数据集（有四个活动用于有效阐明结果）进行的实证评估表明，平均确定性准确度至少提高了~ 11.01%，从而证明了我们提出的框架的设备上功能。
Active transfer learning for structural health monitoring
Dynamics Research Group, School of Mechanical, Aerospace and Civil Engineering, University of Sheffield（谢菲尔德大学机械、航空航天与土木工程学院动力研究组）；Department of Civil and Environmental Engineering, University of Perugia（佩鲁吉亚大学土木与环境工程学院）
Uncertainty-Guided Label Rebalancing for CPS Safety Monitoring
Lero Research Ireland Centre for Software Research, University of Limerick, Castletroy, Limerick, Ireland（勒罗爱尔兰软件研究中心，利默里克大学，卡斯莱特里，利默里克，爱尔兰）；University of Ottawa, Canada（渥太华大学，加拿大）；Lero Research Ireland Centre for Software Research, University of Limerick, Ireland（勒罗爱尔兰软件研究中心，利默里克大学，爱尔兰）
Real-time Adaptation for Condition Monitoring Signal Prediction using Label-aware Neural Processes
Department of Systems & Information Engineering, University of Virginia（系统与信息工程系，弗吉尼亚大学）；Department of Industrial & Operations Engineering, University of Michigan（工业与运营工程系，密歇根大学）
Monitoring Urban Forests from Auto-Generated Segmentation Maps
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Technical University of Munich（慕尼黑工业大学）；TJ Watson Research Center, IBM Research（IBM研究院托马斯·J·沃森研究中心）
Label-free Monitoring of Self-Supervised Learning Progress
Faculty of Computer Science Dalhousie University Halifax, Canada；Faculty of Computer Science Dalhousie University Halifax, Canada 0000-0002-5237-3867；Faculty of Computer Science Dalhousie University Halifax, Canada 0000-0002-6144-8963
Continuous Determination of Respiratory Rate in Hospitalized Patients using Machine Learning Applied to Electrocardiogram Telemetry
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Wellman Center for Photomedicine（韦尔曼光医学中心）；Healthcare Transformation Lab（医疗转型实验室）；Center for Systems Biology（系统生物学中心）
