We explore the potential for using a nonsmooth loss function based on the max-norm in the training of an artificial neural network. We hypothesise that this may lead to superior classification results in some special cases where the training data is either very small or unbalanced.
Our numerical experiments performed on a simple artificial neural network with no hidden layers (a setting immediately amenable to standard nonsmooth optimisation techniques) appear to confirm our hypothesis that uniform approximation based approaches may be more suitable for the datasets with reliable training data that either is limited size or biased in terms of relative cluster sizes.
A Unified Analysis on the Subgradient Upper Bounds for the Subgradient Methods Minimizing Composite Nonconvex, Nonsmooth and Non-Lipschitz Functions
Antai College of Economics and Management, Shanghai Jiao Tong University(上海交通大学安泰经济管理学院);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Institute of Translational Medicine and National Center for Translational Medicine, Shanghai Jiao Tong University(上海交通大学转化医学研究院和转化医学中心);Department of Industrial and Systems Engineering, University of Minnesota(明尼苏达大学工业与系统工程系)
Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
Toulouse School of Economics(图卢兹经济学院);Université Toulouse 1 Capitole(图卢兹第一大学);IRIT(图卢兹计算机研究所);Université de Toulouse(图卢兹大学);IRT Saint Exupery(圣埃克苏佩里技术研究院)
Learning to Control: The iUzawa-Net for Nonsmooth Optimal Control of Linear PDEs
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University(南洋理工大学数学科学系);Department of Mathematics, The University of Hong Kong(香港大学数学系);School of Mathematical Sciences, Nankai University(南开大学数学学院)
Over-fitting Control in Deep Nonlinear Matrix Completion via Gradual Nonsmooth Regularization
Sharif University of Technology(谢里夫理工大学);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);University of Antwerp(安特卫普大学);University of Luxembourg(卢森堡大学)