RealSimLoop：基于视觉反馈的可微降阶仿真的在线真实到仿真自适应
RealSimLoop: Online Real-to-Sim Adaptation via Differentiable Reduced-Order Simulation with Vision Feedback
浏览论文内容
中文总结 AI 辅助
RealSimLoop提出可微降阶仿真结合视觉反馈，实现准实时在线真实到仿真自适应，跟踪时变材料属性，优于离线方法，支持外力预测与3D应力场重建。
中文摘要 AI 辅助
真实世界中可变形物体的观测往往是稀疏的或仅停留在表面层面，而下游任务则需要隐藏的物理量，如内部变形、应力场和相互作用力。基于物理的仿真可以恢复这些量，但在线真实到仿真自适应由于全空间优化的高成本、有限的反馈以及随时间变化的材料属性而仍然具有挑战性。为解决这些问题，我们提出了RealSimLoop，一个利用视觉数据作为物理反馈的在线真实到仿真自适应的可微框架。我们的方法通过在降阶神经子空间内执行可微仿真，实现了准实时性能，极大地加速了优化循环。我们将这一高效动力学模型与可微渲染相结合，实现直接梯度反向传播，利用高保真像素数据来优化材料刚度等物理参数。此外，通过采用滑动窗口目标函数，RealSimLoop实现了鲁棒的在线自适应，使系统能够跟踪随时间变化的材料属性，并有效弥合因模型降阶或未建模动力学而产生的真实到仿真差距。大量实验表明，我们的方法优于传统的离线方法，并且我们验证了该框架在下游应用中的多功能性，包括外力预测和带新颖视图合成的3D应力场重建。
英文摘要
Real-world observations of deformable objects are often sparse or surface-level, while downstream tasks require hidden physical quantities such as internal deformation, stress fields, and interaction forces. Physics-based simulation can recover these quantities, but online real-to-sim adaptation remains challenging due to costly full-space optimization, limited feedback, and time-varying material properties. To address these challenges, we propose RealSimLoop, a differentiable framework for online real-to-sim adaptation using vision data as physical feedback. Our approach achieves quasi-real-time performance by executing differentiable simulation within a reduced-order neural subspace, drastically accelerating the optimization loop. We couple this efficient dynamics model with differentiable rendering, enabling direct gradient backpropagation that leverages high-fidelity pixel data to refine physical parameters such as material stiffness. Furthermore, by employing a sliding-window objective function, RealSimLoop enables robust online adaptation, allowing the system to track time-varying material properties and effectively bridge the real-to-sim gap arising from model reduction or unmodeled dynamics. Extensive experiments demonstrate that our method outperforms conventional offline methods, and we validate the framework's versatility in downstream applications, including external force prediction and 3D stress field reconstruction with novel view synthesis.
发表机构
- South China University of Technology（华南理工大学）
- The Chinese University of Hong Kong（香港中文大学）
机构由 AI 辅助整理，请以论文原文为准。
