监督学习问题的腐败:类型与缓解方法
Corruptions of Supervised Learning Problems: Typology and Mitigations
AI总结:
本文提出了一种通用的腐败理论,通过马尔可夫核分析底层概率分布的变化,统一了不同类型的腐败模型,并探讨了针对各种腐败类型的缓解方法。
摘要:
腐败在数据收集中普遍存在。尽管已有大量研究,现有文献主要集中在特定设置和学习场景,缺乏对腐败建模和缓解的统一视角。本文开发了一种通用的腐败理论,涵盖监督学习问题的所有修改,包括模型类和损失的变化。通过分析底层概率分布的变化,我们的方法带来了三个新机会:首先,构建了一个新型且可证明的腐败框架,区分不同类型的腐败;其次,通过比较清洁和受污染场景下的贝叶斯风险,系统分析了腐败对学习任务的影响;第三,基于这些结果,我们研究了各种腐败类型的缓解方法。我们扩展了现有的标签腐败损失修正方法以处理依赖性腐败类型。我们的发现强调了将经典腐败修正学习框架推广到更宽松的范式以涵盖更多腐败类型的必要性。我们提供了这种范式以及属性和联合腐败情况下的损失修正公式。
英文摘要:
Corruption is notoriously widespread in data collection. Despite extensive research, the existing literature predominantly focuses on specific settings and learning scenarios, lacking a unified view of corruption modelization and mitigation. In this work, we develop a general theory of corruption, which incorporates all modifications to a supervised learning problem, including changes in model class and loss. Focusing on changes to the underlying probability distributions via Markov kernels, our approach leads to three novel opportunities. First, it enables the construction of a novel, provably exhaustive corruption framework, distinguishing among different corruption types. This serves to unify existing models and establish a consistent nomenclature. Second, it facilitates a systematic analysis of corruption's consequences on learning tasks, by comparing Bayes risks in the clean and corrupted scenarios. Notably, while label corruptions affect only the loss function, attribute corruptions additionally influence the hypothesis class. Third, building upon these results, we investigate mitigations for various corruption types. We expand existing loss-correction methods for label corruption to handle dependent corruption types. Our findings highlight the necessity to generalize this classical corruption-corrected learning framework to a new paradigm with weaker requirements to encompass more corruption types. We provide such a paradigm as well as loss correction formulas in the attribute and joint corruption cases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。