计算机视觉和机器学习中若干问题的数学微分工具基准
A Benchmark of Selected Algorithmic Differentiation Tools on Some Problems in Computer Vision and Machine Learning
摘要:
数学微分(AD)允许仅给定目标函数的实现来精确计算导数。虽然有许多AD工具可用,但AD方法的正确和有效实现并不简单。现有的工具通常差异太大,无法支持通用的测试套件。在本文中,我们比较了15种计算导数的方法,包括11个自动微分工具,实现各种方法,并编写在各种语言(C++,F#,MATLAB,Julia和Python),两个符号微分工具,有限差分,和手派生计算。我们从计算机视觉和机器学习中研究三个目标函数。这些目标在很大程度上是简单的,因为不涉及迭代循环,并且条件语句被封装在诸如{ \tt abs}或{ \tt logsumexp}之类的函数中。然而,对于算法微分的成功来说,有效地处理这样的“简单”目标函数是很重要的,因为计算机视觉和机器学习中的许多问题都是这种形式。当然,我们的结果取决于程序员的技能和对工具的熟悉程度。然而,我们认为这篇论文提出了一个重要的数据点:一个熟练的程序员花了大约一个星期的时间来研究我们提出的每个工具。我们已经将我们的实现作为开源提供,以允许社区复制和更新这些基准。
英文摘要:
Algorithmic differentiation (AD) allows exact computation of derivatives given only an implementation of an objective function. Although many AD tools are available, a proper and efficient implementation of AD methods is not straightforward. The existing tools are often too different to allow for a general test suite. In this paper, we compare fifteen ways of computing derivatives including eleven automatic differentiation tools implementing various methods and written in various languages (C++, F#, MATLAB, Julia and Python), two symbolic differentiation tools, finite differences, and hand-derived computation. We look at three objective functions from computer vision and machine learning. These objectives are for the most part simple, in the sense that no iterative loops are involved, and conditional statements are encapsulated in functions such as {\tt abs} or {\tt logsumexp}. However, it is important for the success of algorithmic differentiation that such `simple' objective functions are handled efficiently, as so many problems in computer vision and machine learning are of this form. Of course, our results depend on programmer skill, and familiarity with the tools. However, we contend that this paper presents an important datapoint: a skilled programmer devoting roughly a week to each tool produced the timings we present. We have made our implementations available as open source to allow the community to replicate and update these benchmarks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。