arXiv 1806.021362018-06-06cs.MScs.LGcs.PLcs.SCstat.ML

Efficient Differentiable Programming in a Functional Array-Processing Language

EPFL（瑞士联邦理工学院洛桑分校）；Microsoft HoloLens（微软全息透镜）；Microsoft Research（微软研究院）

Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, Simon Peyton Jones, Christoph Koch

英文摘要：

We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demonstrate how for some real-world machine learning and computer vision benchmarks, the system outperforms the state-of-the-art automatic differentiation tools.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

$\nabla$SD: Differentiable Programming for Sparse Tensors

University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）

Sparse tensors are prevalent in many data-intensive applications, yet existing differentiable programming frameworks are tailored towards dense tensors. This presents a significant challenge for efficiently computing gradients through sparse tensor operations, as their irregular sparsity patterns can result in substantial memory and computational overheads. In this work, we introduce a novel framework that enables the efficient and automatic differentiation of sparse tensors, addressing this fundamental issue. Our experiments demonstrate the effectiveness of the proposed framework in terms of …

2023-03-13cs.LGcs.MScs.PL

通过可微编程学习优化

Learning to Optimize by Differentiable Programming

Nanyang Technological University（南洋理工大学）

本教程介绍利用可微编程学习设计一阶优化算法，通过端到端训练提升收敛性和解质量，并基于Fenchel-Rockafellar对偶性展示ADMM和PDHG等算法的学习与适应。

2026-09-01cs.LGcs.MSmath.OC

Efficient and Sound Differentiable Programming in a Functional Array-Processing Language

University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）；ETH Zurich（苏黎世联邦理工学院）；Graphcore；Microsoft Research（微软研究院）；Epic Games；DeepMind

Automatic differentiation (AD) is a technique for computing the derivative of a function represented by a program. This technique is considered as the de-facto standard for computing the differentiation in many machine learning and optimisation software tools. Despite the practicality of this technique, the performance of the differentiated programs, especially for functional languages and in the presence of vectors, is suboptimal. We present an AD system for a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both so…

2022-12-20cs.LGcs.MScs.PL

AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia

University of Basel（巴塞尔大学）；Pumas-AI Inc.（普马斯人工智能公司）；UNSW Canberra（新南威尔士大学堪培拉分校）；Invenia Labs（因维尼亚实验室）；Massachusetts Institute of Technology（麻省理工学院）；Julia Computing Inc.（朱莉娅计算公司）

No single Automatic Differentiation (AD) system is the optimal choice for all problems. This means informed selection of an AD system and combinations can be a problem-specific variable that can greatly impact performance. In the Julia programming language, the major AD systems target the same input and thus in theory can compose. Hitherto, switching between AD packages in the Julia Language required end-users to familiarize themselves with the user-facing API of the respective packages. Furthermore, implementing a new, usable AD package required AD package developers to write boilerplate code…

2022-02-04cs.LGcs.MScs.SE

通过模型转换实现灵活的可微优化

Flexible Differentiable Optimization via Model Transformations

Zuse Institute Berlin（柏林祖斯研究所）；PSR（PSR公司）；PUC-Rio（里约热内卢天主教大学）；Massachusetts Institute of Technology（麻省理工学院）；Columbia University（哥伦比亚大学）

本文介绍了Julia库DiffOpt.jl，通过模型转换实现对优化问题解的微分，支持前向和反向微分模式，将约束优化与端到端可微编程连接，适用于超参数优化、反向传播和敏感性分析等多种场景。

2023-07-31cs.LGcs.MSmath.OC

基于可微编程与机器学习的多物理反演

Learned multiphysics inversion with differentiable programming and machine learning

Georgia Institute of Technology（佐治亚理工学院）；Rice University（莱斯大学）；University Medical Center Utrecht（乌得勒支大学医学中心）；Microsoft（微软）；SINTEF Digital（挪威科技工业研究院数字研究所）；Imperial College London（帝国理工学院）

本文提出SLIM开源软件框架，通过集成多层抽象与可微编程，支持波动方程反问题、学习先验正则化及多相流神经代理，并构建可扩展原型验证了其优势。

2023-04-12cs.DCcs.LGcs.MS

更多相关论文收起更多论文

跨越 PDE 与机器学习壁垒的可微编程

Differentiable programming across the PDE and Machine Learning barrier

I-X Centre for AI In Science（I-X科学人工智能中心）；Imperial College London（伦敦帝国学院）

本文提出通用可微编程抽象，通过代码生成自动耦合任意 PDE 系统与机器学习模型，已集成于 Firedrake 并支持 PyTorch、JAX。

2024-09-09cs.LGcs.MScs.NA

Array Programming with NumPy

Array programming provides a powerful, compact, expressive syntax for accessing, manipulating, and operating on data in vectors, matrices, and higher-dimensional arrays. NumPy is the primary array programming library for the Python language. It plays an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, material science, engineering, finance, and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves and the first imaging of a black ho…

2020-06-18cs.MSstat.CO

DiFVM：一种基于图的有限体积求解器，用于无结构网格上的可微CFD

DiFVM: A Vectorized Graph-Based Finite Volume Solver for Differentiable CFD on Unstructured Meshes

University of Notre Dame（圣母大学）；Cornell University（康奈尔大学）

DiFVM是首个在无结构多面体网格上原生运行的可微有限体积CFD求解器，通过将有限体积离散化与图神经网络消息传递结构建立结构同构，实现无结构网格连接性的高效处理。

2026-03-16cs.MS