向量值正则化最小二乘算法的最优 Sobolev 范数速率研究

图上概率测度的可扩展 Sobolev IPM

Scalable Sobolev IPM for Probability Measures on a Graph

本文针对图上概率测度的 Sobolev IPM 计算难题，利用 Sobolev 范数与加权 $L^p$ 范数的关系提出可闭式快速计算的正则化方法，并据此构造正定核用于文档分类和拓扑数据分析。

扩散模型中的Sobolev正则化得分差估计

Sobolev Regularized Score Difference Estimation in Diffusion Models

本文针对扩散模型得分差估计存在的统计一致性或可扩展性问题，提出Sobolev正则化的一致可扩展估计器，在小样本场景稳定性提升，真实任务性能优于非正则化方法。

正则化随机傅里叶特征与有限元重建在Sobolev空间中的算子学习

Regularized Random Fourier Features and Finite Element Reconstruction for Operator Learning in Sobolev Space

本文提出正则化随机傅里叶特征与有限元重建方法，用于在噪声数据中学习算子，通过多变量t分布随机特征和频率加权Tikhonov正则化抑制高频噪声，实现稳健性和高效性。

BitNet: Bit-Regularized Deep Neural Networks

We present a novel optimization strategy for training neural networks which we call "BitNet". The parameters of neural networks are usually unconstrained and have a dynamic range dispersed over all real values. Our key idea is to limit the expressive power of the network by dynamically controlling the range and set of values that the parameters can take. We formulate this idea using a novel end-to-end approach that circumvents the discrete parameter space by optimizing a relaxed continuous and differentiable upper bound of the typical classification loss function. The approach can be interpret…

Optimal rates for the regularized learning algorithms under general source condition

We consider the learning algorithms under general source condition with the polynomial decay of the eigenvalues of the integral operator in vector-valued function setting. We discuss the upper convergence rates of Tikhonov regularizer under general source condition corresponding to increasing monotone index function. The convergence issues are studied for general regularization schemes by using the concept of operator monotone index functions in minimax setting. Further we also address the minimum possible error for any learning algorithm.

可扩展正则化向量乘法误差模型用于正值金融时间序列

Scalable Regularized Vector Multiplicative Error Models for Positive-valued Financial Time Series

本文提出针对正值金融时间序列的log-vMEM模型正则化估计方法，结合分层滞后结构与多种惩罚，采用块坐标下降算法，并利用GPU加速积分提升计算可扩展性。

Vector-valued Reproducing Kernel Banach Spaces with Applications to Multi-task Learning

Motivated by multi-task machine learning with Banach spaces, we propose the notion of vector-valued reproducing kernel Banach spaces (RKBS). Basic properties of the spaces and the associated reproducing kernels are investigated. We also present feature map constructions and several concrete examples of vector-valued RKBS. The theory is then applied to multi-task machine learning. Especially, the representer theorem and characterization equations for the minimizer of regularized learning schemes in vector-valued RKBS are established.

Convergence Rates of Smooth Message Passing with Rounding in Entropy-Regularized MAP Inference

Maximum a posteriori (MAP) inference is a fundamental computational paradigm for statistical inference. In the setting of graphical models, MAP inference entails solving a combinatorial optimization problem to find the most likely configuration of the discrete-valued model. Linear programming (LP) relaxations in the Sherali-Adams hierarchy are widely used to attempt to solve this problem, and smooth message passing algorithms have been proposed to solve regularized versions of these LPs with great success. This paper leverages recent work in entropy-regularized LPs to analyze convergence rates…

重尾噪声下的正则化最小二乘学习是极小极大最优的

Regularized least squares learning with heavy-tailed noise is minimax optimal

本文证明在噪声仅具有有限高阶矩的重尾条件下，再生核希尔伯特空间中的岭回归仍能达到此前仅在次指数噪声假设下成立的极小极大最优收敛速率，从而揭示了正则化最小二乘对重尾噪声的渐近鲁棒性。