基于全连接神经网络非光滑正则化的矩阵补全
Matrix Completion via Nonsmooth Regularization of Fully Connected Neural Networks
AI总结：
本文针对全连接神经网络过拟合问题，提出带非光滑正则化项的深度神经网络（DNN-NSR）算法，通过逐步加入ℓ₁范数和核范数正则化项，提升了矩阵补全性能。
摘要：
传统矩阵补全方法通过假设矩阵为低秩来近似缺失值，这会导致缺失值的线性近似。已有研究表明，使用深度神经网络等非线性估计器可获得更优性能。深度全连接神经网络（FCNN）是最适合矩阵补全的架构之一，但因其容量较高易过拟合，导致泛化能力较低。本文通过对FCNN模型的中间表示的ℓ₁范数和权重矩阵的核范数进行正则化来控制过拟合，由此得到的正则化目标函数是非光滑且非凸的，即现有基于梯度的方法无法应用于该模型。本文提出了一种近端梯度法的变体，并研究其收敛到临界点的情况。在FCNN训练的初始轮次中，正则化项被忽略，随着轮次推进，正则化项的影响逐渐增大。逐步加入非光滑正则化项是带非光滑正则化项的深度神经网络（DNN-NSR）算法性能更优的主要原因。仿真结果表明，与现有线性和非线性算法相比，所提算法具有优越性。
英文摘要：
Conventional matrix completion methods approximate the missing values by assuming the matrix to be low-rank, which leads to a linear approximation of missing values. It has been shown that enhanced performance could be attained by using nonlinear estimators such as deep neural networks. Deep fully connected neural networks (FCNNs), one of the most suitable architectures for matrix completion, suffer from over-fitting due to their high capacity, which leads to low generalizability. In this paper, we control over-fitting by regularizing the FCNN model in terms of the $\ell_{1}$ norm of intermediate representations and nuclear norm of weight matrices. As such, the resulting regularized objective function becomes nonsmooth and nonconvex, i.e., existing gradient-based methods cannot be applied to our model. We propose a variant of the proximal gradient method and investigate its convergence to a critical point. In the initial epochs of FCNN training, the regularization terms are ignored, and through epochs, the effect of that increases. The gradual addition of nonsmooth regularization terms is the main reason for the better performance of the deep neural network with nonsmooth regularization terms (DNN-NSR) algorithm. Our simulations indicate the superiority of the proposed algorithm in comparison with existing linear and nonlinear algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
