基于延迟迹的可微因果计算
Differentiable Causal Computations via Delayed Trace
摘要:
我们调查因果计算的输入序列的输出序列的$n$ th输出依赖于第一个$n$输入只。我们通过一个笛卡尔范畴$C$到另一个范畴$St(C)$的一个新的跟踪类操作称为“延迟跟踪”,它错过了通常的跟踪猛拉和dinaturality公理的建设范畴理论模型。延迟的跟踪操作在$St(C)$中提供了一种具有隐式保护性保证的反馈机制。当$C$配备笛卡尔微分算子时,我们使用时间反向传播的抽象版本为$St(C)$构造微分算子,这是一种基于函数展开的机器学习技术。这获得了一系列通过时间反向传播的属性,包括链式法则和施瓦茨定理。我们的微分算子也能够计算有状态网络的导数,而不需要将网络展开。
英文摘要:
We investigate causal computations taking sequences of inputs to sequences of outputs where the $n$th output depends on the first $n$ inputs only. We model these in category theory via a construction taking a Cartesian category $C$ to another category $St(C)$ with a novel trace-like operation called "delayed trace", which misses yanking and dinaturality axioms of the usual trace. The delayed trace operation provides a feedback mechanism in $St(C)$ with an implicit guardedness guarantee. When $C$ is equipped with a Cartesian differential operator, we construct a differential operator for $St(C)$ using an abstract version of backpropagation through time, a technique from machine learning based on unrolling of functions. This obtains a swath of properties for backpropagation through time, including a chain rule and Schwartz theorem. Our differential operator is also able to compute the derivative of a stateful network without requiring the network to be unrolled.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。