A deep learning approach for the computation of curvature in the level-set method

Error-Correcting Neural Networks for Two-Dimensional Curvature Computation in the Level-Set Method

We present an error-neural-modeling-based strategy for approximating two-dimensional curvature in the level-set method. Our main contribution is a redesigned hybrid solver [Larios-Cárdenas and Gibou, J. Comput. Phys. (May 2022), 10.1016/j.jcp.2022.111291] that relies on numerical schemes to enable machine-learning operations on demand. In particular, our routine features double predicting to harness curvature symmetry invariance in favor of precision and stability. The core of this solver is a multilayer perceptron trained on circular- and sinusoidal-interface samples. Its role is to quantify …

A Loss Curvature Perspective on Training Instability in Deep Learning

In this work, we study the evolution of the loss Hessian across many classification tasks in order to understand the effect the curvature of the loss has on the training dynamics. Whereas prior work has focused on how different learning rates affect the loss Hessian observed during training, we also analyze the effects of model initialization, architectural choices, and common training heuristics such as gradient clipping and learning rate warmup. Our results demonstrate that successful model and hyperparameter choices allow the early optimization trajectory to either avoid -- or navigate out …

基于平面曲率与梯度方向的手写识别MLP基线

An MLP Baseline for Handwriting Recognition Using Planar Curvature and Gradient Orientation

本研究验证了仅用平面曲率和梯度方向等二阶几何特征即可驱动MLP实现手写字符识别，在MNIST和EMNIST上分别达到97%和89%的准确率，证明可解释手工特征也能发挥深度学习优势。

Efficiently Training Low-Curvature Neural Networks

The highly non-linear nature of deep neural networks causes them to be susceptible to adversarial examples and have unstable gradients which hinders interpretability. However, existing methods to solve these issues, such as adversarial training, are expensive and often sacrifice predictive accuracy. In this work, we consider curvature, which is a mathematical quantity which encodes the degree of non-linearity. Using this, we demonstrate low-curvature neural networks (LCNNs) that obtain drastically lower curvature than standard models while exhibiting similar predictive performance, which leads…

A Tikhonov approach to level set curvature computation

In numerical simulations of two-phase flows, the computation of the curvature of the interface is a crucial ingredient. Using a finite element and level set discretization, the discrete interface is typically the level set of a low order polynomial, which often results in a poor approximation of the interface curvature. We present an approach to curvature computation using an approximate inversion of the $L^2$ projection operator from the Sobolev space $H^2$ or $H^3$. For finite element computation of the approximate inverse, the resulting higher order equation is reformulated as a system of s…

Machine learning algorithms for three-dimensional mean-curvature computation in the level-set method

We propose a data-driven mean-curvature solver for the level-set method. This work is the natural extension to $\mathbb{R}^3$ of our two-dimensional strategy in [DOI: 10.1007/s10915-022-01952-2][1] and the hybrid inference system of [DOI: 10.1016/j.jcp.2022.111291][2]. However, in contrast to [1,2], which built resolution-dependent neural-network dictionaries, here we develop a pair of models in $\mathbb{R}^3$, regardless of the mesh size. Our feedforward networks ingest transformed level-set, gradient, and curvature data to fix numerical mean-curvature approximations selectively for interface…