arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.00563stat.MLcs.LG
收藏

关于通用领域泛化方法的局限性

On the Limitations of General Purpose Domain Generalisation Methods

School of Informatics(信息学院);University of Edinburgh(爱丁堡大学);Samsung AI Centre, Cambridge(三星剑桥人工智能中心)

Henry Gouk, Ondrej Bohdal, Da Li, Timothy Hospedales

AI总结:

本文探讨了领域泛化中学习算法的性能极限,证明了在多种设置下ERM难以被显著超越,并提出了优化ERM的策略,通过实验验证了理论假设。

摘要:

我们研究了在多种领域泛化(DG)设置中学习算法的基本性能局限性。鉴于先前提出的方法在可靠地超越经验风险最小化(ERM)方面的困难,我们推导了ERM超额风险的上界,以及极小极大超额风险的下界。我们的发现表明,在我们考虑的所有DG设置中,不可能显著优于ERM。我们的结论不仅限于标准的协变量偏移设置,还适用于另外两种对领域差异方式有额外限制的设置。第一种设置限制所有领域之间的成对距离具有非平凡上界,该距离由一类广泛的积分概率度量来衡量。第二种替代设置考虑一类受限的DG问题,其中所有领域具有相同的底层支持。我们的分析还提出了在这些DG设置中可用于优化ERM性能的不同策略。我们还通过实验探索了由我们的理论分析提出的假设。

英文摘要:

We investigate the fundamental performance limitations of learning algorithms in several Domain Generalisation (DG) settings. Motivated by the difficulty with which previously proposed methods have in reliably outperforming Empirical Risk Minimisation (ERM), we derive upper bounds on the excess risk of ERM, and lower bounds on the minimax excess risk. Our findings show that in all the DG settings we consider, it is not possible to significantly outperform ERM. Our conclusions are limited not only to the standard covariate shift setting, but also two other settings with additional restrictions on how domains can differ. The first constrains all domains to have a non-trivial bound on pairwise distances, as measured by a broad class of integral probability metrics. The second alternate setting considers a restricted class of DG problems where all domains have the same underlying support. Our analysis also suggests how different strategies can be used to optimise the performance of ERM in each of these DG setting. We also experimentally explore hypotheses suggested by our theoretical analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑