EgoSim：用于具身交互生成的视角世界模拟器
EgoSim: Egocentric World Simulator for Embodied Interaction Generation
AI总结：
EgoSim通过建模可更新的世界状态，解决现有视角模拟器在3D grounding和动态更新上的不足，生成空间一致的交互视频并支持跨具身迁移。
摘要：
我们介绍了EgoSim，一个闭环的视角世界模拟器，能够生成空间一致的交互视频并持续更新底层3D场景状态。现有视角模拟器要么缺乏显式的3D定位，导致视角变化时出现结构漂移，要么将场景视为静态，无法在多阶段交互中更新世界状态。EgoSim通过将3D场景建模为可更新的世界状态来解决这两个限制。我们通过Geometry-action-aware Observation Simulation模型生成具身交互，并利用Interaction-aware State Updating模块实现空间一致性。为克服由于难以获取密集对齐的场景-交互训练对而带来的关键数据瓶颈，我们设计了一条可扩展的管道，从真实大规模单目视角视频中提取静态点云、相机轨迹和具身动作。我们进一步引入EgoCap，一种能够使用未校准智能手机低成本收集现实数据的采集系统。大量实验表明，EgoSim在视觉质量、空间一致性和复杂场景及真实灵活交互的泛化能力方面显著优于现有方法，同时支持跨具身迁移至机器人操作。代码和数据集将很快开放。项目页面位于egosimulator.github.io。
英文摘要：
We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation. Existing egocentric simulators either lack explicit 3D grounding, causing structural drift under viewpoint changes, or treat the scene as static, failing to update world states across multi-stage interactions. EgoSim addresses both limitations by modeling 3D scenes as updatable world states. We generate embodiment interactions via a Geometry-action-aware Observation Simulation model, with spatial consistency from an Interaction-aware State Updating module. To overcome the critical data bottleneck posed by the difficulty in acquiring densely aligned scene-interaction training pairs, we design a scalable pipeline that extracts static point clouds, camera trajectories, and embodiment actions from in-the-wild large-scale monocular egocentric videos. We further introduce EgoCap, a capture system that enables low-cost real-world data collection with uncalibrated smartphones. Extensive experiments demonstrate that EgoSim significantly outperforms existing methods in terms of visual quality, spatial consistency, and generalization to complex scenes and in-the-wild dexterous interactions, while supporting cross-embodiment transfer to robotic manipulation. Codes and datasets will be open soon. The project page is at egosimulator.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
