arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.12866stat.MLcs.LG
收藏

用于密度估计和近似的增强KRnet

Augmented KRnet for density estimation and approximation

Louisiana State University(路易斯安那州立大学);Peng Cheng Laboratory(鹏城实验室)

Xiaoliang Wan, Kejun Tang

摘要:

在这项工作中,我们提出了增强KRnets,包括离散和连续模型。基于流的生成式建模的一个困难是保持传输图的可逆性,这通常是有效性和鲁棒性之间的权衡。在实NVP中,使用特定的模式在两个分离的维度组之间交换信息,已经实现了精确的可逆性。KRnet的开发是为了通过将Knothe-Rosenblatt重排纳入运输图的结构来增强数据维度之间的信息交换。由于保持了精确的可逆性,在KRnet中,所有数据维度的完全非线性更新需要三次迭代。为了缓解这个问题,我们将添加增强的维度,作为数据维度之间通信的通道。在增强的KRnet中,完全非线性更新在两次迭代中实现。我们还表明,增广KRnet可以重新表述为神经ODE的离散化,其中保持精确可逆性,使得伴随方法可以相对于离散化ODE来制定,以获得精确的梯度。数值实验证明了模型的有效性。

英文摘要:

In this work, we have proposed augmented KRnets including both discrete and continuous models. One difficulty in flow-based generative modeling is to maintain the invertibility of the transport map, which is often a trade-off between effectiveness and robustness. The exact invertibility has been achieved in the real NVP using a specific pattern to exchange information between two separated groups of dimensions. KRnet has been developed to enhance the information exchange among data dimensions by incorporating the Knothe-Rosenblatt rearrangement into the structure of the transport map. Due to the maintenance of exact invertibility, a full nonlinear update of all data dimensions needs three iterations in KRnet. To alleviate this issue, we will add augmented dimensions that act as a channel for communications among the data dimensions. In the augmented KRnet, a fully nonlinear update is achieved in two iterations. We also show that the augmented KRnet can be reformulated as the discretization of a neural ODE, where the exact invertibility is kept such that the adjoint method can be formulated with respect to the discretized ODE to obtain the exact gradient. Numerical experiments have been implemented to demonstrate the effectiveness of our models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑