通过核平方和逼近带约束的优化问题

Optimality Implies Kernel Sum Classifiers are Statistically Efficient

We propose a novel combination of optimization tools with learning theory bounds in order to analyze the sample complexity of optimal kernel sum classifiers. This contrasts the typical learning theoretic results which hold for all (potentially suboptimal) classifiers. Our work also justifies assumptions made in prior work on multiple kernel learning. As a byproduct of our analysis, we also provide a new form of Rademacher complexity for hypothesis classes containing only optimal classifiers.

The Statistical Cost of Robust Kernel Hyperparameter Tuning

This paper studies the statistical complexity of kernel hyperparameter tuning in the setting of active regression under adversarial noise. We consider the problem of finding the best interpolant from a class of kernels with unknown hyperparameters, assuming only that the noise is square-integrable. We provide finite-sample guarantees for the problem, characterizing how increasing the complexity of the kernel class increases the complexity of learning kernel hyperparameters. For common kernel classes (e.g. squared-exponential kernels with unknown lengthscale), our results show that hyperparamet…

基于递归最小二乘和递归核方法的在线多任务学习

Online Multi-Task Learning with Recursive Least Squares and Recursive Kernel Methods

本文提出两种基于递归最小二乘和核方法的在线多任务学习算法，实现二次复杂度递归，并在风速预测中显著提升性能。

Wasserstein Exponential Kernels

In the context of kernel methods, the similarity between data points is encoded by the kernel function which is often defined thanks to the Euclidean distance, a common example being the squared exponential kernel. Recently, other distances relying on optimal transport theory - such as the Wasserstein distance between probability distributions - have shown their practical relevance for different machine learning techniques. In this paper, we study the use of exponential kernels defined thanks to the regularized Wasserstein distance and discuss their positive definiteness. More specifically, we…

Near-optimal estimation of smooth transport maps with kernel sums-of-squares

It was recently shown that under smoothness conditions, the squared Wasserstein distance between two distributions could be efficiently computed with appealing statistical error upper bounds. However, rather than the distance itself, the object of interest for applications such as generative modeling is the underlying optimal transport map. Hence, computational and statistical guarantees need to be obtained for the estimated maps themselves. In this paper, we propose the first tractable algorithm for which the statistical $L^2$ error on the maps nearly matches the existing minimax lower-bounds…

基于Flash Attention的快速高斯和计算

Fast Gauss Sums via Flash Attention

该研究提出通过两次输入增强，利用Flash Attention计算带任意符号权重的高斯核和，无需自定义GPU代码，在fp16、D>8时性能优于PyTorch与PyKeOps，内存扩展呈线性。

Diagonally Square Root Integrable Kernels in System Identification

In recent years, the reproducing kernel Hilbert space (RKHS) theory has played a crucial role in linear system identification. The core of a RKHS is the associated kernel characterizing its properties. Accordingly, this work studies the class of diagonally square root integrable (DSRI) kernels. We demonstrate that various well-known stable kernels introduced in system identification belong to this category. Moreover, it is shown that any DSRI kernel is also stable and integrable. We look into certain topological features of the RKHSs associated with DSRI kernels, particularly the continuity of…

超立方体的平方多项式逼近核：改进的误差界及其对Lasserre层级的影响

Squared polynomial approximation kernels for the hypercube: improved error bounds and implications for Lasserre hierarchies

本文提出一类新的多项式逼近核，用于逼近超立方体上的非负多项式，将系数ℓ1范数的误差界从已知的O(1/r)改进为O(log^3 r/r^2)，并由此改进了超立方体上多项式优化的Lasserre层级的收敛速率。

多项式优化的平方和层级与Christoffel-Darboux核

Sum-of-squares hierarchies for polynomial optimization and the Christoffel-Darboux kernel

本文证明在单位球和标准单纯形上，基于Schmüdgen型证书的Lasserre平方和层级以O(1/r^2)速率收敛到多项式全局最小值，并通过Christoffel-Darboux核建立证明。

On Robustness of Kernel-Based Regularized System Identification

This paper presents a novel feature of the kernel-based system identification method. We prove that the regularized kernel-based approach for the estimation of a finite impulse response is equivalent to a robust least-squares problem with a particular uncertainty set defined in terms of the kernel matrix, and thus, it is called kernel-based uncertainty set. We provide a theoretical foundation for the robustness of the kernel-based approach to input disturbances. Based on robust and regularized least-squares methods, different formulations of system identification are considered, where the kern…

Best $\ell_1$-approximation of nonnegative polynomials by sums of squares

Given a nonnegative polynomial f, we provide an explicit expression for its best $\ell_1$-norm approximation by a sum of squares of given degree.

Sums of squares over totally real fields are rational sums of squares

Let $K$ be a totally real number field with Galois closure $L$. We prove that if $f \in \mathbb Q[x_1,...,x_n]$ is a sum of $m$ squares in $K[x_1,...,x_n]$, then $f$ is a sum of \[4m \cdot 2^{[L: \mathbb Q]+1} {[L: \mathbb Q] +1 \choose 2}\] squares in $\mathbb Q[x_1,...,x_n]$. Moreover, our argument is constructive and generalizes to the case of commutative $K$-algebras. This result gives a partial resolution to a question of Sturmfels on the algebraic degree of certain semidefinite programing problems.