Large sparse symmetric linear systems appear in several branches of science and engineering thanks to the widespread use of the finite element method (FEM). The fastest sparse linear solvers available implement hybrid iterative methods. These methods are based on heuristic algorithms to permute rows and columns or find a preconditioner matrix. In addition, they are inherently sequential, making them unable to leverage the GPU processing power entirely. We propose neural sparse linear solvers, a deep learning framework to learn approximate solvers for sparse symmetric linear systems. Our method relies on representing a sparse symmetric linear system as an undirected weighted graph. Such graph representation is inherently permutation-equivariant and scale-invariant, and it can become the input to a graph neural network trained to regress the solution. We test neural sparse linear solvers on static linear analysis problems from structural engineering. Our method is less accurate than classic algorithms, but it is hardware-independent, fast on GPUs, and applicable to generic sparse symmetric systems without any additional hypothesis. Although many limitations remain, this study shows a general approach to tackle problems involving sparse symmetric matrices using graph neural networks.
Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs
University of Utah(犹他大学);University of California, Los Angeles(加州大学洛杉矶分校);Illinois Institute of Technology(伊利诺伊理工学院);University of California, Irvine(加州大学欧文分校);Scientific Computing and Imaging (SCI) Institute(科学计算与成像研究所)
Speeding Up Mixed-Integer Programming Solvers with Sparse Learning for Branching
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology(佐治亚理工学院 H. Milton Stewart 工业与系统工程系);School of Chemical & Biomolecular Engineering, Georgia Institute of Technology(佐治亚理工学院化学与生物分子工程学院)
IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks
School of EIC, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);Shanghai Jiao Tong University(上海交通大学);Renmin University of China(中国人民大学);Huawei(华为);School of Science, Wuhan University of Technology(武汉理工大学理学院)
Semi-Implicit Neural Solver for Time-dependent Partial Differential Equations
Technical University Munich(慕尼黑工业大学);École normale supérieure, Paris(巴黎高等师范学院);deepc GmbH, Munich(慕尼黑deepc有限公司);Helmholtz Zentrum München(慕尼黑亥姆霍兹研究中心);University of Zurich(苏黎世大学)
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
School of Science, Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学深圳校区科学学院);Department of Theoretical Physics(理论物理系);Applied Mathematics, University of Cambridge, Cambridge, United Kingdom(应用数学系,剑桥大学,英国)
RINS: Residual-Image Neural Subspace Solvers for Large Sparse Linear Systems
Shanghai Innovation Institute(上海创新研究院);Harbin Institute of Technology(哈尔滨工业大学);Shanghai Jiao Tong University(上海交通大学);Renmin University of China(中国人民大学)
PIS: A Generalized Physical Inversion Solver for Arbitrary Sparse Observations via Set Conditioned Flow Matching
School of Information, University of California, Berkeley, USA(信息学院,加州大学伯克利分校);College of Civil Engineering, Tongji University, Shanghai, China(土木学院,同济大学);Institute of Mechanical Sciences(机械科学研究院)
Leveraging Differentiable PDE Solvers for Semi-Neural Spatial Reconstruction From Sparse Measurements
Department of Mathematics, Technion, Haifa, Israel(数学系,技术离子学院,海法,以色列);Department of Civil and Environmental Engineering, Technion, Haifa, Israel(土木与环境工程系,技术离子学院,海法,以色列)
Department of Mathematics, Columbia University(哥伦比亚大学数学系);HKU Musketeers Foundation Institute of Data Science(香港大学 Musketeers 基金会数据科学研究所);HKU-Shanghai Intelligent Computing Research Center(香港大学-上海智能计算研究中心)
NewPINNs: Physics-Informing Neural Networks Using Conventional Solvers for Partial Differential Equations
Department of Mathematics, University of California Riverside(加州大学河滨分校数学系);Department of Mechanical Engineering, University of California Riverside(加州大学河滨分校机械工程系);German Innovation Center, Procter & Gamble Service GmbH(普罗cter与 gamble 服务德国创新中心)
Lie Point Symmetry Data Augmentation for Neural PDE Solvers
University of Amsterdam(阿姆斯特丹大学);Johannes Kepler University Linz(约翰内斯·开普勒林茨大学);Microsoft Research(微软研究院);Qualcomm AI Research(高通人工智能研究院);Deepmind(深度思维)
Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术学院);Massachusetts Institute of Technology(麻省理工学院);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Artificial Intelligence Research Institute(人工智能研究所)
A Solver-Free Framework for Scalable Learning in Neural ILP Architectures
Indian Institute of Technology Delhi(印度理工学院德里分校)
最近的一个焦点是设计在神经模型(本文中称为神经ILP)中具有非线性规划(ILP)层的架构。神经ILP架构适用于需要数据驱动约束学习的纯推理任务或需要感知(神经)和推理(ILP)的任务。最近一种用于神经ILP端到端训练的SOTA方法通过ILP黑盒明确定义了梯度(Paulus et al. 2021)-由于对minibatch中每个训练数据点的底层ILP求解器的调用,这种训练非常缓慢。作为回应,我们提出了一种替代的训练策略,是无求解器的,即,在训练时根本不调用ILP求解程序。神经ILP有一组可训练的超平面(用于ILP中的成本和约束),它们共同代表一个多面体。我们的关键思想是,训练损失应该强制最终多面体通过软余量公式将正面(满足所有约束)与负面(至少一个违反约束或次优成本值)分开。虽然正面例子作为训练数据的一部分提供,但我们设计了用于生成负面样本的新技术。我们的解决方案是灵活的,足以处理平等以及不平等的约束。在几个问题上的实验,无论是感知的还是符号的,都需要学习ILP的约束条件,表明我们的方法具有优越的性能,与纯神经基线和其他需要基于求解器的训练的最先进的模型相比,其规模要好得多。特别是,我们能够在9 x 9符号和视觉数独中获得出色的性能,而其他Neural ILP求解器无法扩展。
King Abdullah University of Science and Technology (KAUST)(沙特阿拉伯国王 Abdullah 科技大学(KAUST)(注:此处“King Abdullah University of Science and Technology”的中文常见译名为“沙特国王 Abdullah 科技大学”,但根据原文本括号内缩写“KAUST”,保留“国王 Abdullah 科技大学(KAUST)”或更简洁的“沙特国王 Abdullah 科技大学”均可,此处按原文括号内缩写整合为“沙特阿拉伯国王 Abdullah 科技大学(KAUST)”以确保准确对应))
Rethinking Constraint Awareness for Efficient State Embedding of Neural Routing Solver
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);School of Automation and Intelligent Manufacturing, Southern University of Science and Technology(南方科技大学自动化与智能制造学院);Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, Southern University of Science and Technology(广东省全驱动系统控制理论与技术重点实验室,南方科技大学);Pengcheng Laboratory(鹏城实验室)
LocalEscaper: A Weakly-supervised Framework with Regional Reconstruction for Scalable Neural TSP Solvers
School of Computer Science, Huazhong University of Science and Technology, China(华中科技大学计算机学院);Department of Computer Science, Cornell University, United States(康奈尔大学计算机系)
Provably Convergent Working Set Algorithm for Non-Convex Regularized Regression
Univ. de Rouen(鲁昂大学);Criteo AI Lab(Criteo人工智能实验室);Univ Côte d’Azur(蔚蓝海岸大学);CNRS(法国国家科学研究中心);OCA Lagrange(拉格朗日蔚蓝海岸天文台);INSA de Rouen(鲁昂国立应用科学学院);IMAG(蒙彼利埃亚历山大·格罗滕迪克研究所);Université de Montpellier(蒙彼利埃大学)