Data sparsity is a common issue to train machine learning tools such as neural networks for engineering and scientific applications, where experiments and simulations are expensive. Recently physics-constrained neural networks (PCNNs) were developed to reduce the required amount of training data. However, the weights of different losses from data and physical constraints are adjusted empirically in PCNNs. In this paper, a new physics-constrained neural network with the minimax architecture (PCNN-MM) is proposed so that the weights of different losses can be adjusted systematically. The training of the PCNN-MM is searching the high-order saddle points of the objective function. A novel saddle point search algorithm called Dual-Dimer method is developed. It is demonstrated that the Dual-Dimer method is computationally more efficient than the gradient descent ascent method for nonconvex-nonconcave functions and provides additional eigenvalue information to verify search results. A heat transfer example also shows that the convergence of PCNN-MMs is faster than that of traditional PCNNs.
Minimax Regret of Switching-Constrained Online Convex Optimization: No Phase Transition
Yale University（耶鲁大学）；Simons Institute for the Theory of Computing（西蒙斯理论计算研究所）；Massachusetts Institute of Technology（麻省理工学院）；University of Southern California（南加利福尼亚大学）
我们研究的问题切换约束在线凸优化（OCO），玩家有有限的机会改变她的行动。虽然这个在线学习任务的离散模拟已被广泛研究，以前的工作在连续设置既没有建立极大极小率，也没有算法实现it.In本文中，我们表明，$ T $轮开关约束OCO少于$ K $开关有极大极小遗憾$ Θ(\frac{T}{\sqrt{K}}) $。特别地，对于一维，其至少为$ \frac{T}{\sqrt{2K}} $，对于更高维度，其至少为$ \frac{T}{\sqrt{K}} $。在高维的下界是通过一个正交子空间的论点。在一维中，一种新的对抗策略产生了$O(\frac{T}{\sqrt{K}})$的下界，但更涉及到包含常数的精确极大极小分析。为了建立更严格的一维结果，我们引入了\emph{fugal game}松弛，其极小极大遗憾的下限开关约束OCO。我们证明了逃逸对策的最小最大遗憾至少是$ \frac{T}{\sqrt{2K}} $，从而建立了一维最优最小最大遗憾下界。为了建立与维数无关的上界，我们接下来证明了一个minimax算法提供了一个$ O(\frac{T}{\sqrt{K}}) $上界，因此得出结论，对于任何$K$，切换约束OCO的极小极大遗憾是$ Θ(\frac{T}{\sqrt{K}}) $。这是在鲜明的对比，它的离散对应，开关约束预测专家的问题，表现出一个相变之间的低开关和高开关制度的最小最大遗…
A Geometric Approach to Constrained Online Learning
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur（计算机科学与工程系，印度理工学院Kharagpur分校）；School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, India（技术与计算机科学学院，印度基础研究科学研究院，孟买，印度）
