Vid2Sid: 视频可以帮助缩小仿真到现实的差距
Vid2Sid: Videos Can Help Close the Sim2Real Gap
AI总结：
Vid2Sid通过视频驱动的系统识别方法，结合基础模型感知与VLM循环优化器，实现仿真到现实的物理参数校准，提供可解释的推理并提升校准精度。
摘要：
对机器人仿真器的物理参数（摩擦力、阻尼力、材料刚度）进行校准，通常通过手动调整或使用黑盒优化器来减少误差，但无法解释导致误差的具体物理差异。当感知仅限于外部摄像头时，问题进一步复杂化，因为存在感知噪声和缺乏直接的力或状态测量。我们提出了Vid2Sid，一个基于视频的系统识别流水线，它将基础模型感知与一个在循环中使用VLM的优化器相结合，分析配对的仿真-现实视频，诊断具体的不匹配，并提出物理参数更新的自然语言理由。我们在腱状驱动的手指（MuJoCo中的刚体动力学）和可变形连续帐篷（PyElastica中的软体动力学）上评估了我们的方法。在仿真到现实的测试中，未在训练中见过的控制设置上，Vid2Sid在所有设置中实现了最佳的平均排名，匹配或超过了黑盒优化器，同时在每次迭代中提供可解释的推理。仿真到仿真的验证确认Vid2Sid最准确地恢复了真实参数（平均相对误差低于13% vs. 28-98%），消融分析揭示了三个校准阶段。VLM引导的优化在感知清晰且仿真具有表现力时表现最佳，而模型类别限制则在更具有挑战性的设置中限制性能。
英文摘要：
Calibrating a robot simulator's physics parameters (friction, damping, material stiffness) to match real hardware is often done by hand or with black-box optimizers that reduce error but cannot explain which physical discrepancies drive the error. When sensing is limited to external cameras, the problem is further compounded by perception noise and the absence of direct force or state measurements. We present Vid2Sid, a video-driven system identification pipeline that couples foundation-model perception with a VLM-in-the-loop optimizer that analyzes paired sim-real videos, diagnoses concrete mismatches, and proposes physics parameter updates with natural language rationales. We evaluate our approach on a tendon-actuated finger (rigid-body dynamics in MuJoCo) and a deformable continuum tentacle (soft-body dynamics in PyElastica). On sim2real holdout controls unseen during training, Vid2Sid achieves the best average rank across all settings, matching or exceeding black-box optimizers while uniquely providing interpretable reasoning at each iteration. Sim2sim validation confirms that Vid2Sid recovers ground-truth parameters most accurately (mean relative error under 13\% vs. 28--98\%), and ablation analysis reveals three calibration regimes. VLM-guided optimization excels when perception is clean and the simulator is expressive, while model-class limitations bound performance in more challenging settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
