Kernel interpolation with continuous volume sampling

Interpolation and Learning with Scale Dependent Kernels

We study the learning properties of nonparametric ridge-less least squares. In particular, we consider the common case of estimators defined by scale dependent kernels, and focus on the role of the scale. These estimators interpolate the data and the scale can be shown to control their stability through the condition number. Our analysis shows that are different regimes depending on the interplay between the sample size, its dimensions, and the smoothness of the problem. Indeed, when the sample size is less than exponential in the data dimension, then the scale can be chosen so that the learni…

大维度下核插值的相图

The phase diagram of kernel interpolation in large dimensions

本文研究大维度下核插值的泛化能力，通过精确刻画方差和偏差的阶数，得到了 $(s,\gamma)$-相图，划分了核插值最优、次优和不一致的区域，有助于理解良性过拟合现象。

Product Kernel Interpolation for Scalable Gaussian Processes

Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs). Structured Kernel Interpolation (SKI) exploits these techniques by deriving approximate kernels with very fast MVMs. Unfortunately, such strategies suffer badly from the curse of dimensionality. We develop a new technique for MVM based learning that exploits product kernel structure. We demonstrate that this technique is broadly applicable, resulting in linear rather than exponential runtime with dimension for SKI, as well as state…

稀疏网格核插值

Kernel Interpolation with Sparse Grids

针对结构化核插值（SKI）因密集网格随维度指数增长而难以扩展的问题，本文提出在SKI中使用稀疏网格，并设计了近线性时间矩阵-向量乘法算法及基于单纯形的插值方案，使SKI能在保持精度的同时扩展到更高维度。

使用软核插值的高维高斯过程回归

High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation

提出 SoftKI 方法，结合 SKI 与变分诱导点方法，通过 softmax 插值实现高维数据集上的可扩展高斯过程回归，在适中维度下表现具竞争力。

SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes

State-of-the-art methods for scalable Gaussian processes use iterative algorithms, requiring fast matrix vector multiplies (MVMs) with the covariance kernel. The Structured Kernel Interpolation (SKI) framework accelerates these MVMs by performing efficient MVMs on a grid and interpolating back to the original space. In this work, we develop a connection between SKI and the permutohedral lattice used for high-dimensional fast bilateral filtering. Using a sparse simplicial grid instead of a dense rectangular one, we can perform GP inference exponentially faster in the dimension than SKI. Our app…

Kernel Interpolation for Scalable Online Gaussian Processes

Gaussian processes (GPs) provide a gold standard for performance in online settings, such as sample-efficient control and black box optimization, where we need to update a posterior distribution as we acquire data in a sequential fashion. However, updating a GP posterior to accommodate even a single new observation after having observed $n$ points incurs at least $O(n)$ computations in the exact setting. We show how to use structured kernel interpolation to efficiently recycle computations for constant-time $O(1)$ online updates with respect to the number of points $n$, while retaining exact i…

线性时间的代价：结构化核插值的误差分析

The Price of Linear Time: Error Analysis of Structured Kernel Interpolation

本文对结构化核插值（SKI）进行严格误差分析，给出Gram矩阵误差界及诱导点数量选择指南，并揭示维度对线性时间与精度权衡的支配规律。

基于完整导数观测的高斯过程回归扩展

Scaling Gaussian Process Regression with Full Derivative Observations

DSoftKI通过局部温度向量改进插值方案，实现可扩展的高斯过程回归，支持一阶和二阶导数的近似，适用于高维分子力场预测。