arXiv 2104.081912021-04stat.MLcs.LG

PAC-Bayesian Matrix Completion with a Spectral Scaled Student Prior

Norwegian University of Science and Technology（挪威科技大学）

The Tien Mai

英文摘要：

We study the problem of matrix completion in this paper. A spectral scaled Student prior is exploited to favour the underlying low-rank structure of the data matrix. We provide a thorough theoretical investigation for our approach through PAC-Bayesian bounds. More precisely, our PAC-Bayesian approach enjoys a minimax-optimal oracle inequality which guarantees that our method works well under model misspecification and under general sampling distribution. Interestingly, we also provide efficient gradient-based sampling implementations for our approach by using Langevin Monte Carlo. More specifically, we show that our algorithms are significantly faster than Gibbs sampler in this problem. To illustrate the attractive features of our inference strategy, some numerical simulations are conducted and an application to image inpainting is demonstrated.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

1-bit Matrix Completion: PAC-Bayesian Analysis of a Variational Approximation

Due to challenging applications such as collaborative filtering, the matrix completion problem has been widely studied in the past few years. Different approaches rely on different structure assumptions on the matrix in hand. Here, we focus on the completion of a (possibly) low-rank matrix with binary entries, the so-called 1-bit matrix completion problem. Our approach relies on tools from machine learning theory: empirical risk minimization and its convex relaxations. We propose an algorithm to compute a variational approximation of the pseudo-posterior. Thanks to the convex relaxation, the c…

2016-04-14stat.ML

1比特矩阵补全中分数后验的集中性质

Concentration properties of fractional posterior in 1-bit matrix completion

Norwegian University of Science and Technology（挪威科技大学）

本文针对1比特矩阵补全问题，提出分数后验的集中性质理论，使用两种先验分布恢复参数矩阵，无需先验秩信息，且假设条件少于频率派方法。

2024-04-13cs.LGstat.ML

Calibrated Elastic Regularization in Matrix Completion

This paper concerns the problem of matrix completion, which is to estimate a matrix from observations in a small subset of indices. We propose a calibrated spectrum elastic net method with a sum of the nuclear and Frobenius penalties and develop an iterative algorithm to solve the convex minimization problem. The iterative algorithm alternates between imputing the missing entries in the incomplete matrix by the current guess and estimating the matrix by a scaled soft-thresholding singular value decomposition of the imputed matrix until the resulting matrix converges. A calibration step follows…

2012-11-09math.STstat.MLstat.TH

通过缩放梯度下降法保证非凸低秩张量估计

Guaranteed Nonconvex Low-Rank Tensor Estimation via Scaled Gradient Descent

Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）

本文提出ScaledGD算法，通过缩放梯度下降法在t-SVD框架下实现低秩张量估计的线性收敛，克服了传统方法在条件数依赖上的限制。

2026-02-14cs.ITcs.LGmath.IT

Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion

University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）

The matrix completion problem seeks to recover a $d\times d$ ground truth matrix of low rank $r\ll d$ from observations of its individual elements. Real-world matrix completion is often a huge-scale optimization problem, with $d$ so large that even the simplest full-dimension vector operations with $O(d)$ time complexity become prohibitively expensive. Stochastic gradient descent (SGD) is one of the few algorithms capable of solving matrix completion on a huge scale, and can also naturally handle streaming data over an evolving ground truth. Unfortunately, SGD experiences a dramatic slow-down …

2022-10-23cs.LGmath.OCstat.ML

Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent

Carnegie Mellon University（卡内基梅隆大学）；UC Berkeley（加州大学伯克利分校）

Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to factorize the matrix into two compact low-rank factors, and then optimize these factors directly via simple iterative methods such as gradient descent and alternating minimization. Despite nonconvexity, recent literatures have shown that these simple heuristics in fact achieve linear convergence when initialized properly for a growing number of problems of interest. However, upon closer examination, existing approach…

2021-06-14cs.ITcs.LGeess.SP

更多相关论文收起更多论文

Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares

The matrix-completion problem has attracted a lot of attention, largely as a result of the celebrated Netflix competition. Two popular approaches for solving the problem are nuclear-norm-regularized matrix approximation (Candes and Tao, 2009, Mazumder, Hastie and Tibshirani, 2010), and maximum-margin matrix factorization (Srebro, Rennie and Jaakkola, 2005). These two procedures are in some cases solving equivalent problems, but with quite different algorithms. In this article we bring the two approaches together, leading to an efficient algorithm for large matrix factorization and completion t…

2014-10-09stat.MEstat.ML

缩放核范数最小化用于低秩张量补全

Scaled Nuclear Norm Minimization for Low-Rank Tensor Completion

Columbia University（哥伦比亚大学）

本文提出通过最小化张量展开的加权核范数之和来恢复低TT秩张量，证明该方法相比传统核范数求和方法更高效，需更少样本即可恢复原始张量。

2017-07-25cs.NAmath.NAstat.ML

N²：一种统一的Python包和测试平台，用于基于最近邻的矩阵补全

N$^2$: A Unified Python Package and Test Bench for Nearest Neighbor-Based Matrix Completion

Cornell University（康奈尔大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）

本文提出N²，一种统一的Python包和测试平台，用于基于最近邻的矩阵补全，展示了新的NN变体和现实世界数据集的基准测试，证明NN方法在现实应用中的优越性。

2026-09-15cs.LGstat.COstat.ML

高维分位数预测的稀疏PAC-Bayes方法

A sparse PAC-Bayesian approach for high-dimensional quantile prediction

Norwegian University of Science and Technology（挪威科技大学）

本文提出一种面向高维分位数预测的稀疏PAC-Bayes方法，结合尺度化Student-t先验与Langevin Monte Carlo，通过非渐近oracle不等式证明其极小极大最优性与稀疏适应性，并在模拟和真实数据中验证了竞争力。

2024-09-03cs.LGmath.STstat.ML

On the properties of variational approximations of Gibbs posteriors

The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow for big datasets. We consider instead variational approximations of the Gibbs posterior, which are fast to compute. We undertake a general study of the properties of such approximations. Our main finding is that such a variational approximation has often the same rate of converg…

2015-06-15math.STstat.MLstat.TH