CKNet：一种基于Koopman算子的卷积神经网络，用于从像素建模潜在动力学
CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels
AI总结：
本文提出CKNet，利用Koopman算子理论从原始像素中学习线性、时不变且可解释的潜在动力学，并通过DCKNet和VCKNet在多个连续控制任务中验证了其建模与图像生成能力。
摘要：
随着基于深度学习的端到端控制的发展，研究新的系统建模技术以实现高维输入的动力学建模具有重要意义。本文提出了一种新颖的基于Koopman的深度卷积网络，称为CKNet，用于从原始像素中识别潜在动力学。CKNet学习一个编码器和一个解码器，分别扮演Koopman特征函数和Koopman模态的角色。Koopman特征值可以通过学习到的状态转移矩阵的特征值来近似。提出了确定性卷积Koopman网络（DCKNet）和变分卷积Koopman网络（VCKNet），分别用于张成某个子空间以近似Koopman算子。由于CKNet是在Koopman理论的约束下训练的，识别出的潜在动力学具有线性形式，并具有良好的可解释性。此外，状态转移矩阵和控制矩阵作为可训练张量进行训练，因此识别出的动力学也是时不变的。我们还设计了一个辅助权重项，用于降低多步线性化误差和预测损失。实验分别在Gym和Mujoco环境中对两个离线训练和四个在线训练的具有连续动作空间的非线性受迫动力学系统进行，结果表明识别出的动力学足以近似潜在动力学并生成清晰的图像。特别是对于离线训练的情况，这项工作从一个新颖的角度证实了CKNet：我们基于相同的回合，分别用DCKNet和VCKNet可视化每个任务的潜在状态和Koopman特征函数的演化过程，结果表明不同的方法学习到了形状上相似的特征。
英文摘要：
With the development of end-to-end control based on deep learning, it is important to study new system modeling techniques to realize dynamics modeling with high-dimensional inputs. In this paper, a novel Koopman-based deep convolutional network, called CKNet, is proposed to identify latent dynamics from raw pixels. CKNet learns an encoder and decoder to play the role of the Koopman eigenfunctions and modes, respectively. The Koopman eigenvalues can be approximated by eigenvalues of the learned state transition matrix. The deterministic convolutional Koopman network (DCKNet) and the variational convolutional Koopman network (VCKNet) are proposed to span some subspace for approximating the Koopman operator respectively. Because CKNet is trained under the constraints of the Koopman theory, the identified latent dynamics is in a linear form and has good interpretability. Besides, the state transition and control matrices are trained as trainable tensors so that the identified dynamics is also time-invariant. We also design an auxiliary weight term for reducing multi-step linearity and prediction losses. Experiments were conducted on two offline trained and four online trained nonlinear forced dynamical systems with continuous action spaces in Gym and Mujoco environment respectively, and the results show that identified dynamics are adequate for approximating the latent dynamics and generating clear images. Especially for offline trained cases, this work confirms CKNet from a novel perspective that we visualize the evolutionary processes of the latent states and the Koopman eigenfunctions with DCKNet and VCKNet separately to each task based on the same episode and results demonstrate that different approaches learn similar features in shapes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
