发表机构
Université Paris-Saclay; CEA; CEA, List（巴黎-萨克雷大学; 法国原子能和替代能源委员会; 法国原子能和替代能源委员会 电子与信息技术实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出LaGSplat框架，可从单目视频推断物理控制的交互式模拟，能对刚性/可变形物体施加训练外的力并实时渲染响应，解决了无约束预测器发散问题。
AI 中文摘要
我们提出了LaGSplat（潜拉格朗日高斯溅射，Latent Lagrangian Gaussian Splatting），这是一种可从单个或少量单目视频中推断交互式、物理控制动力学的框架。在推理时，它允许用户对拍摄的物体（无论是刚性还是可变形的）施加在训练期间从未测量、标注或见过的外力。这之所以可行，是因为低维潜态向量$\boldsymbol{q} \in \mathbb{R}^d$同时扮演两个角色：它是学习到的耗散拉格朗日的广义坐标，也是高斯溅射解码器的条件变量。该解码器的归纳偏差在于其基元是随物体移动的显式点$\boldsymbol{\mu}_i(\boldsymbol{q})$，这使得在图像中施加的力$f$可以拉回为潜广义力$J(\boldsymbol{q})^\top f$并纳入运动方程，而基于像素空间（CNN）或神经场（NeRF）的解码器无法做到这一点。我们结合单目视频和传感器测量，在难度递增的测试用例上验证了LaGSplat，涵盖从刚性到可变形、从自主到受迫真实系统的场景。我们进一步展示了其交互式使用：可随时对重建物体施加任意大小和方向的力，其响应能以2D或3D形式实时渲染。假设在几个广义坐标上存在耗散欧拉-拉格朗日方程，这在一定程度上以通用性为代价，换取对未见过的力的有界、合理响应，而无约束预测器会出现发散问题。
英文摘要
We present LaGSplat (Latent Lagrangian Gaussian Splatting), a framework that infers interactive, physics-governed dynamics from one or a few monocular videos. At inference it lets a user push on the filmed object, rigid or deformable, with an external force that was never measured, annotated, or seen during training. This is possible because a low-dimensional latent state $\mathbf{q} \in \mathbb{R}^d$ plays two roles at once: it is the generalised coordinate of a learned dissipative Lagrangian and the conditioning variable of a Gaussian Splatting decoder. The inductive bias of this decoder, whose primitives are explicit points $μ_i(\mathbf{q})$ that move with the object, is what lets a force $f$ applied in the image pull back into a latent generalised force $J(\mathbf{q})^\top f$ and enter the equations of motion, which pixel-space (CNN) or neural-field (NeRF) decoders cannot do. We validate LaGSplat on test cases of increasing difficulty, from rigid to deformable and from autonomous to forced real systems, combining monocular video and sensor measurements. We further demonstrate interactive use: forces of arbitrary magnitude and direction can be applied to the reconstructed object at any time, its response rendered in real time, in 2D or 3D. Assuming a dissipative Euler-Lagrange equation over a few generalised coordinates trades generality for a bounded, plausible response to unseen forces, where an unconstrained predictor diverges.
Comments25 pages, 11 figures, 4 tables. Project page with interactive demo: https://louenpottier.github.io/lagsplat.html
