Geometric Neural Operators via Lie Group-Constrained Latent Dynamics
School of Information and Software Engineering, University of Electronic Science and Technology of China(信息与软件工程学院,电子科学与技术大学);Computer Science and Engineering, University of Electronic Science and Technology of China(计算机科学与工程,电子科学与技术大学);Institute of Software Chinese Academy of Sciences, Beijing, China(软件研究所,中国科学院);School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, China(机器人与先进制造学院,哈尔滨工业大学(深圳));Department of Computer Science, University of Oxford, Oxford, United Kingdom(计算机科学系,牛津大学)
Jiaquan Zhang, Fachrina Dewi Puspitasari, Songbo Zhang, Yibei Liu, Kuien Liu, Caiyan Qin, Fan Mo, Peng Wang, Yang Yang, Chaoning Zhang
Neural operators offer an effective framework for learning solutions of partial differential equations for many physical systems in a resolution-invariant and data-driven manner. Existing neural operators, however, often suffer from instability in multi-layer iteration and long-horizon rollout, which stems from the unconstrained Euclidean latent space updates that violate the geometric and conservation laws. To address this challenge, we propose to constrain manifolds with low-rank Lie algebra parameterization that performs group action updates on the latent representation. Our method, termed Manifold Constraining based on Lie group (MCL), acts as an efficient \emph{plug-and-play} module that enforces geometric inductive bias to existing neural operators. Extensive experiments on various partial differential equations, such as 1-D Burgers and 2-D Navier-Stokes, over a wide range of parameters and steps demonstrate that our method effectively lowers the relative prediction error by 30-50\% at the cost of 2.26\% of parameter increase. The results show that our approach provides a scalable solution for improving long-term prediction fidelity by addressing the principled geometric constraints absent in the neural operator updates.
Latent-Constrained Conditional VAEs for Augmenting Large-Scale Climate Ensembles
Hong Kong Polytechnic University(香港理工大学);Georgia Institute of Technology(佐治亚理工学院);Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);TomTom North America Inc.(TomTom北美公司)
Time-Dependent PDE-Constrained Optimization via Weak-Form Latent Dynamics
Department of Applied Mathematics, University of Colorado(应用数学系,科罗拉多大学);Center for Applied Scientific Computing, Lawrence Livermore National Laboratory(应用科学计算中心,劳伦斯利弗莫尔国家实验室)
Beyond the EM Algorithm: Constrained Optimization Methods for Latent Class Model
Precima(普雷西马公司)
隐类模型(Latent class model,LCM)是统计学和机器学习领域中应用最广泛的模型之一,它是不同类别分布的有限混合。由于其非连续性和形状的灵活性,市场营销和社会科学等实践领域的研究人员也经常使用LCM从他们的数据中获得见解。一种基于似然的方法,期望最大化(EM)算法,经常被用来获得模型估计。然而,期望最大化算法是众所周知的,其收敛速度慢。在这项研究中,我们探索替代的可能性为基础的方法,可以潜在的补救缓慢收敛的期望最大化算法。更具体地说,我们把基于似然的方法作为一个约束非线性优化问题,并应用拟牛顿型方法来解决它们。我们研究了两种不同的约束优化方法,以最大限度地提高对数似然函数。我们目前的仿真研究结果表明,所提出的方法不仅收敛在更少的迭代比期望最大化算法,但也产生更准确的模型估计。
Manifold-Constrained Energy-Based Transition Models for Offline Reinforcement Learning
Department of Electrical and Computer Engineering, George Washington University(乔治华盛顿大学电气与计算机工程系);Department of Electrical and Computer Engineering, Northeastern University(东北大学电气与计算机工程系)
Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination
College of Intelligence Science and Technology, National University of Defense Technology(国防科技大学智能科学与技术学院);School of Engineering, Newcastle University(纽卡斯尔大学工程学院)
Latent Iterative Refinement Flow: A Geometric Constrained Approach for Few-Shot Generation
School of Mathematics(数学学院);Statistics, Huazhong University of Science(统计学,华中科技大学);School of Electronic Information(电子信息学院);Communications, Huazhong University of Science(通信学院,华中科技大学);Center for Mathematical Science, Huazhong University of Science(数学科学中心,华中科技大学);Huawei(华为)
Escaping from Collapsing Modes in a Constrained Space
National Tsing Hua University(国立清华大学);Google AI(谷歌人工智能);Google Cloud AI(谷歌云人工智能)
生成对抗网络(GANs)在训练过程中经常遭受不可预测的模式崩溃。研究了边界平衡生成对抗网络(BEGAN)的模式坍塌问题。尽管BEGAN具有生成高质量图像的潜力,但我们发现,经过一段时间的训练后,BEGAN往往会在某些模式下崩溃。我们提出了一个新的模型,称为\emph{BEGAN with a Constrained Space}(BEGAN-CS),其中包括潜在的空间约束的损失函数。我们表明,BEGAN-CS可以显着提高训练的稳定性和抑制模式坍塌,既不增加模型的复杂性或降低图像质量。此外,我们可视化的潜在向量的分布,阐明潜在空间约束的效果。实验结果表明,我们的方法具有额外的优势,能够在小数据集上进行训练,并生成与给定的真实图像相似的图像,但具有指定属性的变化。