AI 中文总结
该研究提出Cosmos-H-Dreams系统，结合生成模型与蒸馏技术实现实时外科机器人仿真，支持多控制器驱动，为外科教育等提供开放基础。
AI 中文摘要
外科机器人的生成式仿真仍缺乏实时交互能力。涉及动物或尸体实验室的物理机器人实验耗时久、成本高且难以复现，而经典模拟器难以捕捉照片级真实感外观和可变形组织动力学。我们通过Cosmos-H-Dreams解决这一缺口，这是一个集成的实时外科世界模型系统，结合了动作条件生成模型、教师到学生的蒸馏方案，以及基于NVIDIA FlashDreams流推理库构建的部署栈。从Cosmos-H-Surgical-Simulator（一个在大规模Open-H-Embodiment语料库上微调的多体动作条件外科视频世界模型）出发，我们在特定体和特定手术流程数据上对该检查点进行后训练。通过使用自强迫（Self Forcing）将所得的双向教师蒸馏为因果、少步长的学生模型，我们将被动视频生成器转变为可控的外科模拟器，在单个NVIDIA RTX PRO 6000 Blackwell工作站GPU上以约160推理FPS的速度流式运行。至关重要的是，Cosmos-H-Dreams与控制器无关：任何发出机器人运动学流的接口都可驱动它。我们演示了通过WebRTC的浏览器键盘、通过WebXR的Meta Quest头显、CMR Surgical的Versius等商业外科机器人控制台，以及闭环运行的学习策略实现的实时控制。据我们所知，这是首个支持实时人类和策略控制的交互式外科世界模型。人类操作员和策略均可在合成世界中行动并实时观察后果。我们将Cosmos-H-Dreams作为开放外科仿真系统发布，为外科教育、可扩展合成数据生成以及未来术中决策支持提供共同基础。
英文摘要
Generative simulation for surgical robotics still lacks real-time interaction. Physical-robot experiments, often involving animal or cadaver labs, are time-consuming, costly, and difficult to reproduce, while classical simulators struggle to capture photorealistic appearance and deformable-tissue dynamics. We address this gap with Cosmos-H-Dreams, an integrated real-time surgical world-model system combining an action-conditioned generative model, a teacher-to-student distillation recipe, and a deployment stack built on the NVIDIA FlashDreams streaming-inference library. Starting from Cosmos-H-Surgical-Simulator, a multi-embodiment action-conditioned surgical video world model fine-tuned on the large-scale Open-H-Embodiment corpus, we post-train this checkpoint on embodiment- and procedure-specific data. By distilling the resulting bidirectional teacher into a causal, few-step student with Self Forcing, we turn a passive video generator into a controllable surgical simulator that streams at $\sim$160 inference FPS on a single NVIDIA RTX PRO 6000 Blackwell workstation GPU. Crucially, Cosmos-H-Dreams is controller-agnostic: any interface that emits a stream of robot kinematics can drive it. We demonstrate live control through a browser keyboard over WebRTC, a Meta Quest headset over WebXR, a commercial surgical robot console such as CMR Surgical's Versius, and learned policies operating in closed loop. To our knowledge, this is the first interactive surgical world model supporting live human and policy control. Human operators and policies alike can act inside the synthesized world and observe the consequences in real time. We release Cosmos-H-Dreams as an open surgical simulation system, providing a common foundation for surgical education, scalable synthetic data generation, and future intraoperative decision support.
