用重载方法减小伴随微分法的随机存储器容量
Reduction of the Random Access Memory Size in Adjoint Algorithmic Differentiation by Overloading
摘要:
伴随算法微分算子和函数重载是基于有向无环图的解释,从数值模拟程序的评估。存储图形所需的计算机系统存储器的大小与底层程序执行的浮点运算的数量成比例地增长。它很快就超过了可用的内存资源。除了相对简单的数值模拟,朴素的伴随算法微分往往变得不可行。对与图相关联的数据的访问可以被分类为顺序的和随机的。后者指的是由图内顶点之间的邻接关系定义的存储器访问模式。顺序访问的数据可以分解成块。这些块可以跨系统存储层次结构进行流传输,从而将可用存储器的量扩展到例如硬盘。异步i/o可以帮助减轻由于访问较慢的内存而增加的成本。因此,可以解决更大的问题实例,而无需求助于技术上具有挑战性的用户干预,如检查点。随机访问的数据不应该被分解。它的逐块流式传输可能会由于跨块的数据访问而产生大量的计算开销。因此,伴随所需的随机访问内存的大小应保持最小,以消除分解的需要。我们提出了一个组合的专用内存的伴随$L$ -值与剩余带宽的开发作为一种可能的解决方案。测试结果表明,在保持整体计算效率的同时,显著节省了随机存取存储器的大小。
英文摘要:
Adjoint algorithmic differentiation by operator and function overloading is based on the interpretation of directed acyclic graphs resulting from evaluations of numerical simulation programs. The size of the computer system memory required to store the graph grows proportional to the number of floating-point operations executed by the underlying program. It quickly exceeds the available memory resources. Naive adjoint algorithmic differentiation often becomes infeasible except for relatively simple numerical simulations. Access to the data associated with the graph can be classified as sequential and random. The latter refers to memory access patterns defined by the adjacency relationship between vertices within the graph. Sequentially accessed data can be decomposed into blocks. The blocks can be streamed across the system memory hierarchy thus extending the amount of available memory, for example, to hard discs. Asynchronous i/o can help to mitigate the increased cost due to accesses to slower memory. Much larger problem instances can thus be solved without resorting to technically challenging user intervention such as checkpointing. Randomly accessed data should not have to be decomposed. Its block-wise streaming is likely to yield a substantial overhead in computational cost due to data accesses across blocks. Consequently, the size of the randomly accessed memory required by an adjoint should be kept minimal in order to eliminate the need for decomposition. We propose a combination of dedicated memory for adjoint $L$-values with the exploitation of remainder bandwidth as a possible solution. Test results indicate significant savings in random access memory size while preserving overall computational efficiency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。