Computational-Statistical Trade-off in Kernel Two-Sample Testing with Random Fourier Features
Department of Statistics and Data Sciences, The University of Texas at Austin(德克萨斯大学奥斯汀分校统计与数据科学系);Department of Mathematical Sciences, KAIST(韩国科学技术院数学科学系)
Recent years have seen a surge in methods for two-sample testing, among which the Maximum Mean Discrepancy (MMD) test has emerged as an effective tool for handling complex and high-dimensional data. Despite its success and widespread adoption, the primary limitation of the MMD test has been its quadratic-time complexity, which poses challenges for large-scale analysis. While various approaches have been proposed to expedite the procedure, it has been unclear whether it is possible to attain the same power guarantee as the MMD test at sub-quadratic time cost. To fill this gap, we revisit the approximated MMD test using random Fourier features, and investigate its computational-statistical trade-off. We start by revealing that the approximated MMD test is pointwise consistent in power only when the number of random features approaches infinity. We then consider the uniform power of the test and study the time-power trade-off under the minimax testing framework. Our result shows that, by carefully choosing the number of random features, it is possible to attain the same minimax separation rates as the MMD test within sub-quadratic time. We demonstrate this point under different distributional assumptions such as densities in a Sobolev ball. Our theoretical findings are corroborated by simulation studies.
From Fourier Amplitudes to Persistence Images for Impact-Echo Defect Detection
Institute of Mathematics for Industry, Kyushu University(九州大学工业数学研究所);Department of Mathematics, Ajou University(亚洲大学数学系);Graduate School of Science, Kyoto University(京都大学理学研究科)
Towards A Unified Analysis of Random Fourier Features
University of Oxford(牛津大学);King’s College London(伦敦国王学院)
随机傅立叶特征是一种广泛使用的简单有效的核方法扩展技术。然而,现有的理论分析的方法,仍然集中在特定的学习任务,并通常给出悲观的界限,这是不一致的实证结果。我们解决了这些问题,并提供了第一个统一的风险分析学习与随机傅立叶特征使用平方误差和Lipschitz连续损失函数。在我们的边界,计算成本和预期的风险收敛速度之间的权衡是特定于问题的,并表示在正则化参数和\emph{number of effective degrees of freedom}。我们研究了标准的随机傅立叶特征的方法,我们提高了现有的界限,以保证相应的极小极大风险的核岭回归收敛速度所需的功能的数量,以及一个数据相关的修改,采样功能成比例的\emph{ridge leverage scores},进一步减少了所需的功能数量。由于岭杠杆分数是昂贵的计算,我们设计了一个简单的近似方案,可证明降低了计算成本,而不损失统计效率。