EmbodiedSplat：利用移动设备高斯溅射实现个性化真实到仿真到真实导航
EmbodiedSplat: Personalized Real-to-Sim-to-Real Navigation with Gaussian Splats from a Mobile Device
AI总结：
本文提出EmbodiedSplat，利用手机捕获场景并通过3D高斯溅射重建网格，在Habitat-Sim中微调导航策略，显著提升真实世界导航成功率。
摘要：
具身人工智能领域主要依赖仿真进行训练和评估，通常使用缺乏真实感的完全合成环境，或使用昂贵硬件捕获的高保真真实世界重建。因此，仿真到真实的迁移仍然是一个重大挑战。在本文中，我们提出了EmbodiedSplat，这是一种新颖的方法，通过高效捕获部署环境并在重建场景中微调策略来实现个性化策略训练。我们的方法利用3D高斯溅射（GS）和Habitat-Sim模拟器来弥合真实场景捕获与有效训练环境之间的差距。使用iPhone捕获的部署场景，我们通过GS重建网格，从而在与真实世界条件高度近似的环境中进行训练。我们对训练策略、预训练数据集和网格重建技术进行了全面分析，评估它们对真实世界场景中仿真到真实可预测性的影响。实验结果表明，使用EmbodiedSplat微调的智能体优于在大规模真实世界数据集（HM3D）和合成生成数据集（HSSD）上预训练的零样本基线，在真实世界图像导航任务上分别实现了20%和40%的绝对成功率提升。此外，我们的方法在重建网格上产生了较高的仿真与真实相关性（0.87-0.97），凸显了其以最小工作量将策略适应多样化环境的有效性。项目页面：https://gchhablani.github.io/embodied-splat。
英文摘要：
The field of Embodied AI predominantly relies on simulation for training and evaluation, often using either fully synthetic environments that lack photorealism or high-fidelity real-world reconstructions captured with expensive hardware. As a result, sim-to-real transfer remains a major challenge. In this paper, we introduce EmbodiedSplat, a novel approach that personalizes policy training by efficiently capturing the deployment environment and fine-tuning policies within the reconstructed scenes. Our method leverages 3D Gaussian Splatting (GS) and the Habitat-Sim simulator to bridge the gap between realistic scene capture and effective training environments. Using iPhone-captured deployment scenes, we reconstruct meshes via GS, enabling training in settings that closely approximate real-world conditions. We conduct a comprehensive analysis of training strategies, pre-training datasets, and mesh reconstruction techniques, evaluating their impact on sim-to-real predictivity in real-world scenarios. Experimental results demonstrate that agents fine-tuned with EmbodiedSplat outperform both zero-shot baselines pre-trained on large-scale real-world datasets (HM3D) and synthetically generated datasets (HSSD), achieving absolute success rate improvements of 20% and 40% on real-world Image Navigation task. Moreover, our approach yields a high sim-vs-real correlation (0.87-0.97) for the reconstructed meshes, underscoring its effectiveness in adapting policies to diverse environments with minimal effort. Project page: https://gchhablani.github.io/embodied-splat.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
