This paper aims to overcome a fundamental problem in the theory and application of deep neural networks (DNNs). We propose a method to solve the local minimum problem in training DNNs directly. Our method is based on the cross-entropy loss criterion's convexification by transforming the cross-entropy loss into a risk averting error (RAE) criterion. To alleviate numerical difficulties, a normalized RAE (NRAE) is employed. The convexity region of the cross-entropy loss expands as its risk sensitivity index (RSI) increases. Making the best use of the convexity region, our method starts training with an extensive RSI, gradually reduces it, and switches to the RAE as soon as the RAE is numerically feasible. After training converges, the resultant deep learning machine is expected to be inside the attraction basin of a global minimum of the cross-entropy loss. Numerical results are provided to show the effectiveness of the proposed method.
Adaptively trained Physics-informed Radial Basis Function Neural Networks for Solving Multi-asset Option Pricing Problems
College of Mathematics and Statistics, Northwest Normal University（西北师范大学数学与统计学院）；Department of Mathematics, City University of Hong Kong（香港城市大学数学系）；Department of Information Technology, Uppsala University（乌普萨拉大学信息科技系）
Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods
ShanghaiTech University（上海科技大学）；Shenzhen University（深圳大学）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）
Department of Statistics University of Michigan（密歇根大学统计系）；Department of EECS University of Michigan（密歇根大学电气与计算机科学系）
我们提出了一个新的主题建模过程，利用的事实，即潜在的狄利克雷分配（LDA）的日志似然函数是渐近等价的对数的主题单形的体积。这使得主题建模可以重新表述为找到概率单纯形，使其体积最小化，并包含表示为单词分布的文档。提出了一种最小体积主题模型优化的凸松弛方法，并证明了在Arora et al.（2013）和Huang et al.（2016）引入的可分性假设和充分分散假设下，松弛后的问题与原问题具有相同的全局最小值.提出了一种局部收敛的交替方向乘子法（ADMM）求解松弛最小体积问题。数值实验表明，我们的方法的计算时间和主题恢复性能方面的好处。
