arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01437cs.LGstat.ML
收藏

对部分观测数据的度量一致性正则化的理论分析

Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data

H. Milton Stewart School of Industrial & Systems Engineering at Georgia Institute of Technology(佐治亚理工学院H.Milton Stewart工业与系统工程学院);Department of Mechanical and Industrial Engineering at Northeastern University(东北大学机械与工业工程系)

Yinsong Wang, Shahin Shahrampour

AI总结:

本文通过理论分析探讨了度量一致性正则化在部分观测数据中的作用机制,并提出了一种新的训练协议以提升模型泛化能力。

摘要:

损坏的数据、缺失的特征或缺失的模态继续困扰着现代机器学习领域。为解决这个问题,一种强制拟合数据和完全观测数据之间一致性的正则化方法已成为提高模型泛化能力的有希望的方法,特别是在部分观测设置中。我们将这类方法称为度量一致性正则化(MCR)。尽管在各种应用中,如图像修复、数据填补和半监督学习中,MCR在经验上取得了成功,但对其理论基础的理解仍然有限。本文通过神经网络距离的视角,填补了这一空白,探讨了为什么、何时以及如何在部分可观察性下MCR提高填补质量。我们的理论分析确定了MCR泛化优势的关键术语,并扩展到不完美的训练环境,证明这种优势并不总是保证的。受这些见解的启发,我们提出了一种新的训练协议,通过监控对偶间隙来确定一个早期停止点,以保持泛化优势。然后,我们提供了详细的实证证据来支持我们的理论主张,并展示我们所提出停止条件的有效性和准确性。我们进一步提供了一组现实数据模拟,以展示MCR在不同模型架构下的通用性,这些架构适用于不同的数据源。

英文摘要:

The problem of corrupted data, missing features, or missing modalities continues to plague the modern machine learning landscape. To address this issue, a class of regularization methods that enforce consistency between imputed and fully observed data has emerged as a promising approach for improving model generalization, particularly in partially observed settings. We refer to this class of methods as Measure Consistency Regularization (MCR). Despite its empirical success in various applications, such as image inpainting, data imputation and semi-supervised learning, a fundamental understanding of the theoretical underpinnings of MCR remains limited. This paper bridges this gap by offering theoretical insights into when MCR yields a more favorable finite-sample estimation-error upper bound, viewed through the lens of neural network distance. Under ideal interpolation and compatibility conditions, we show that the MCR estimation-error upper bound is no larger than vanilla supervised training and becomes strictly smaller when the mixed-sample alternative is favorable. We then extend the analysis to the non-ideal regime, where optimization and compatibility residuals can potentially offset this finite-sample advantage. Guided by these insights, we propose a novel practical diagnostic that leverages the duality gap and a calibrated drift estimator to infer the potential benefit of MCR training. We present detailed empirical evidence to support our theoretical claims and to show the effectiveness and accuracy of our practical diagnostic. We further provide simulations on real-world datasets to show the versatility of MCR under different model architectures designed for different data sources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑