发表机构
ENS de Lyon; CNRS; Inria（里昂高等师范学院; 法国国家科学研究中心; 法国国家信息与自动化研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文通过归纳法证明DAG ReLU网络路径提升Jacobian的秩，利用骨架矩阵提供无需反向传播的高效计算方法，并附Python模块验证计算增益。
AI 中文摘要
本文通过对网络隐藏节点数量进行归纳，提供了关于有向无环图（DAG）ReLU网络的路径提升（pathlifting）Jacobian的秩的自包含证明。实际上，该归纳是初等的，关键方法是考虑网络的骨架矩阵（skeleton matrix），即一个编码网络路径的稀疏矩阵，并将其一个隐藏神经元的表示转换为输出节点。该证明依赖于连接路径提升、其Jacobian、网络参数及其骨架矩阵的中间命题，这些命题除了有助于得出路径提升Jacobian的秩的结论外，还提供了一种无需反向传播即可计算该Jacobian的方法，且其计算成本在实践中相比通常的反向传播极为高效。本文附带一个Python模块，该模块实现了论文中针对前馈网络的各种命题，并用于实验量化使用所提出理论计算路径提升Jacobian的计算增益。
英文摘要
This paper provides a self-contained proof of the rank of the pathlifting Jacobian of a DAG ReLU network by performing an induction on the network's number of hidden nodes. In fact, the induction is elementary, and the key recipe is to consider the skeleton matrix of the network, a sparse matrix encoding the network paths, and transform the representation of one of its hidden neurons into an output node. The proof relies on intermediate propositions which link the pathlifting, its Jacobian, the network parameters, and its skeleton matrix, which, on top of permitting to conclude on the rank of the pathlifting Jacobian, also provide a way to compute it without backpropagation and whose computation cost is super efficient in practice compare to usual backpropagation. The paper is provided with a Python module that implements the different propositions of the paper for feed forward networks and is used to experimentally quantifies the computational gain of computing the pathlifting Jacobian with the proposed theory.
