基于Metropolis抽样的自适应随机傅立叶特征
Adaptive random Fourier features with Metropolis sampling
摘要：
作为随机傅立叶特征算法，研究了确定具有一个隐层的神经网络逼近$\mathbb{R}^d\ni x\mapsto\sum_{k=1}^K\hatβ_k e^{\mathrm{i}ω_k\cdot x}$的监督学习问题。傅立叶特征，即，使用自适应Metropolis采样器对频率$ω_k\in\mathbb{R}^d$进行采样。Metropolis测试接受建议频率$ω_k'$，具有相应的幅度$\hatβ_k'$，对于某个正参数$γ$，概率为$\min\big\{1, (|\hatβ_k'|/|\hatβ_k|)^γ\big\}$，通过最小化给定计算工作的近似误差来确定。这种自适应的非参数随机方法渐近地导致，如$K\to\infty$，等分布的振幅$|\hatβ_k|$，类似于微分方程的确定性自适应算法。在随机傅立叶特征方法中，等分布的振幅渐近对应于独立样本的最佳密度。数值证据，以证明所提出的算法的近似性能和效率。该算法的合成数据和现实世界的高维基准测试。
英文摘要：
The supervised learning problem to determine a neural network approximation $\mathbb{R}^d\ni x\mapsto\sum_{k=1}^K\hatβ_k e^{\mathrm{i}ω_k\cdot x}$ with one hidden layer is studied as a random Fourier features algorithm. The Fourier features, i.e., the frequencies $ω_k\in\mathbb{R}^d$, are sampled using an adaptive Metropolis sampler. The Metropolis test accepts proposal frequencies $ω_k'$, having corresponding amplitudes $\hatβ_k'$, with the probability $\min\big\{1, (|\hatβ_k'|/|\hatβ_k|)^γ\big\}$, for a certain positive parameter $γ$, determined by minimizing the approximation error for given computational work. This adaptive, non-parametric stochastic method leads asymptotically, as $K\to\infty$, to equidistributed amplitudes $|\hatβ_k|$, analogous to deterministic adaptive algorithms for differential equations. The equidistributed amplitudes are shown to asymptotically correspond to the optimal density for independent samples in random Fourier features methods. Numerical evidence is provided in order to demonstrate the approximation properties and efficiency of the proposed algorithm. The algorithm is tested both on synthetic data and a real-world high-dimensional benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
