可逆神经网络的泛逼近性质
Universal approximation property of invertible neural networks
摘要:
可逆神经网络(Invertible Neural Networks,INN)是一种通过设计而具有可逆性的神经网络结构。由于其可逆性和雅可比矩阵的易处理性,INN具有各种机器学习应用,例如概率建模,生成建模和表示学习。然而,它们吸引人的特性往往是以限制层设计为代价的,这就对它们的表示能力提出了一个问题:我们能否使用这些模型来近似足够多样化的函数?为了回答这个问题,我们已经开发了一个通用的理论框架来研究INN的表示能力,建立在微分几何的结构定理上。该框架简化了仿射的逼近问题,使我们能够显示INN的通用逼近性质。我们将该框架应用于两类具有代表性的INN,即基于耦合流的INN(CF-INN)和神经常微分方程(NODE),并阐明其高表示能力,尽管其架构上的限制。
英文摘要:
Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。