WoMAP：面向具身开放词汇物体定位的世界模型
WoMAP: World Models For Embodied Open-Vocabulary Object Localization
AI总结：
本文提出WoMAP，通过高斯泼溅数据生成、开放词汇检测器奖励蒸馏和隐式世界模型，解决语言指令驱动的零样本物体定位问题，在仿真和真实机器人上显著超越VLM与扩散策略基线。
摘要：
语言指令驱动的主动物体定位是机器人面临的关键挑战，要求对部分可观测环境进行高效探索。然而，当前最先进的方法要么难以泛化到演示数据集之外（例如模仿学习方法），要么无法生成物理上可执行的动作（例如视觉语言模型VLM）。为解决这些局限，我们提出WoMAP（面向主动感知的世界模型）：一种训练开放词汇物体定位策略的方案，该方案：（i）使用基于高斯泼溅（Gaussian Splatting）的真实到仿真再到真实（real-to-sim-to-real）流程实现可扩展的数据生成，无需专家演示；（ii）从开放词汇物体检测器中蒸馏出密集奖励信号；（iii）利用隐式世界模型进行动力学和奖励预测，以在推理时将高层动作提议落实到物理层面。严格的仿真和硬件实验表明，WoMAP在广泛的零样本物体定位任务中表现出卓越性能，与VLM和扩散策略基线相比，成功率分别高出9倍以上和2倍以上。此外，我们证明WoMAP在TidyBot上实现了强大的泛化能力和从仿真到真实的迁移。
英文摘要：
Language-instructed active object localization is a critical challenge for robots, requiring efficient exploration of partially observable environments. However, state-of-the-art approaches either struggle to generalize beyond demonstration datasets (e.g., imitation learning methods) or fail to generate physically grounded actions (e.g., VLMs). To address these limitations, we introduce WoMAP (World Models for Active Perception): a recipe for training open-vocabulary object localization policies that: (i) uses a Gaussian Splatting-based real-to-sim-to-real pipeline for scalable data generation without the need for expert demonstrations, (ii) distills dense rewards signals from open-vocabulary object detectors, and (iii) leverages a latent world model for dynamics and rewards prediction to ground high-level action proposals at inference time. Rigorous simulation and hardware experiments demonstrate WoMAP's superior performance in a broad range of zero-shot object localization tasks, with more than 9x and 2x higher success rates compared to VLM and diffusion policy baselines, respectively. Further, we show that WoMAP achieves strong generalization and sim-to-real transfer on a TidyBot.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
