面向分布偏移学习的高效差异测试
Efficient Discrepancy Testing for Learning with Distribution Shift
AI总结:
本文针对分布偏移下的差异距离测试难题,提出可证明高效的局部化差异测试算法,并据此推广 TDS 学习,取得近最优错误率、常数深度电路的指数级改进及测试时完全多项式运行。
摘要:
在领域适应领域,训练分布与测试分布之间距离的一个基本概念是差异距离。尽管该距离通常难以计算,但本文首次给出一组可证明高效的算法,用于测试局部化差异距离,即相对于一个固定输出分类器来计算差异。这些结果意味着,在 Klivans 等人(2023)近期提出的“分布偏移下可测试学习”(TDS learning)模型中,存在一系列广泛的新型高效学习算法。我们的方法推广并改进了此前关于 TDS 学习的所有工作:(1)获得了能够同时对大类测试分布取得成功的通用学习器;(2)实现了接近最优的错误率;(3)对常数深度电路给出了指数级改进。我们的方法还扩展到半参数设置,并给出了低维凸集的首批正面结果。此外,我们将学习阶段与测试阶段分离,得到了在测试时以完全多项式时间运行的算法。
英文摘要:
A fundamental notion of distance between train and test distributions from the field of domain adaptation is discrepancy distance. While in general hard to compute, here we provide the first set of provably efficient algorithms for testing localized discrepancy distance, where discrepancy is computed with respect to a fixed output classifier. These results imply a broad set of new, efficient learning algorithms in the recently introduced model of Testable Learning with Distribution Shift (TDS learning) due to Klivans et al. (2023). Our approach generalizes and improves all prior work on TDS learning: (1) we obtain universal learners that succeed simultaneously for large classes of test distributions, (2) achieve near-optimal error rates, and (3) give exponential improvements for constant depth circuits. Our methods further extend to semi-parametric settings and imply the first positive results for low-dimensional convex sets. Additionally, we separate learning and testing phases and obtain algorithms that run in fully polynomial time at test time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。