面向分布外预测的因果语义表示学习
Learning Causal Semantic Representation for Out-of-Distribution Prediction
摘要：
传统的监督学习方法，特别是深度学习方法，被发现对分布外（OOD）示例敏感，这主要是因为学习的表示将语义因子与变异因子混合在一起，因为它们具有特定领域的相关性，而只有语义因子会导致输出。为了解决这个问题，我们提出了一个因果语义生成模型（CSG）的基础上的因果推理，使这两个因素分别建模，并开发方法OOD预测从一个单一的训练域，这是常见的和具有挑战性的。该方法是基于因果不变性原理，在变分贝叶斯的有效学习和易于预测的新设计。从理论上证明了在一定条件下，CSG可以通过拟合训练数据来识别语义因子，这种语义识别保证了OOD泛化误差的有界性和自适应的成功性。实证研究表明，OOD的业绩比现行基准有所改善。
英文摘要：
Conventional supervised learning methods, especially deep ones, are found to be sensitive to out-of-distribution (OOD) examples, largely because the learned representation mixes the semantic factor with the variation factor due to their domain-specific correlation, while only the semantic factor causes the output. To address the problem, we propose a Causal Semantic Generative model (CSG) based on a causal reasoning so that the two factors are modeled separately, and develop methods for OOD prediction from a single training domain, which is common and challenging. The methods are based on the causal invariance principle, with a novel design in variational Bayes for both efficient learning and easy prediction. Theoretically, we prove that under certain conditions, CSG can identify the semantic factor by fitting training data, and this semantic-identification guarantees the boundedness of OOD generalization error and the success of adaptation. Empirical study shows improved OOD performance over prevailing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
