ReLU Code Space: A Basis for Rating Network Quality Besides Accuracy

Nonlinear Inductive Matrix Completion based on One-layer Neural Networks

The goal of a recommendation system is to predict the interest of a user in a given item by exploiting the existing set of ratings as well as certain user/item features. A standard approach to modeling this problem is Inductive Matrix Completion where the predicted rating is modeled as an inner product of the user and the item features projected onto a latent space. In order to learn the parameters effectively from a small number of observed ratings, the latent space is constrained to be low-dimensional which implies that the parameter matrix is constrained to be low-rank. However, such biline…

具有内部宽度为一的残差神经网络通用逼近的最小块宽度

Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One

研究残差神经网络通用逼近性质，针对特定激活函数建立块宽度上下界，证明内部宽度为1时在紧致域上实现\(L^p\)逼近的最小块宽度，还给出特定块宽度网络能实现一致逼近及宽度小于某值不能逼近所有目标函数的结论。

面向非凸学习的TUSLA算法的非渐近估计及其在带ReLU激活函数的神经网络中的应用

Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function

该研究针对带超线性增长不连续随机梯度的非凸优化问题，推导TUSLA算法的Wasserstein距离非渐近误差界与超额风险估计，通过理论、合成实验和真实数据集验证其在ReLU神经网络等场景下优于主流优化器。