谱蒸馏：从非线性动力学到线性状态空间模型

中文摘要 AI 辅助

能否通过紧凑的线性状态空间表示学习非线性动力学系统，而无需直接求解非凸的系统辨识问题？我们提供了一种可证明的流程来实现这一点。从未知非线性动力学系统的观测结果出发，我们首先使用观测谱滤波（OSF）学习隐式谱预测器，这是一种凸方法，其性能可与该系统最优的线性观测器相媲美。随后我们应用谱到线性动力系统（LDS）的蒸馏技术，将该预测器转换为显式的循环线性动力学系统。我们的主定理表明，蒸馏得到的LDS的平均预测误差可分解为指数级小的蒸馏项和由最优观测器的Luenberger复杂度决定的OSF学习项。该保证是无维度的：它依赖于观测器复杂度，而非表示非线性系统所需的隐维度。据我们所知，这是首个端到端可证明的方法，通过凸学习后接可证明蒸馏，提取非线性动力学的事后最优LDS表示。在LDS线性基准和MuJoCo行为克隆任务上的实验表明，先训练后蒸馏的流程能生成紧凑的LDS预测器，其性能与直接训练的基线相当或更优。

英文摘要

Can nonlinear dynamical systems be learned through a compact linear state-space representation, without directly solving a non-convex system-identification problem? We give a provable pipeline for doing so. Starting from observations of an unknown nonlinear dynamical system, we first learn an implicit spectral predictor using Observation Spectral Filtering (OSF), a convex method that competes with the best linear observer for the system. We then apply spectral-to-LDS distillation to convert this predictor into an explicit recurrent linear dynamical system. Our main theorem shows that the average prediction error of the distilled LDS decomposes into an exponentially-small distillation term and the OSF learning term governed by the Luenberger complexity of the best observer. The guarantee is dimension-free: it depends on observer complexity rather than on the latent dimension needed to represent the nonlinear system. To our knowledge, this yields the first end-to-end provable method for extracting a best-in-hindsight LDS representation of nonlinear dynamics through convex learning followed by provable distillation. Experiments on linear LDS benchmarks and MuJoCo behavior cloning show that the train-then-distill pipeline produces compact LDS predictors that match or outperform directly trained baselines.

发表机构

• Princeton University（普林斯顿大学）

机构由 AI 辅助整理，请以论文原文为准。