Fysiverse-3D-SimReady 技术报告：面向务实3D世界重建的智能体物理仿真

中文摘要 AI 辅助

智能体识别要求视觉感知超越静态场景理解，生成支持感知-推理-行动循环的结构化场景表示。然而，现有的单图像3D生成方法主要产生视觉上合理的物体资产，而非仿真就绪的场景状态。当独立生成的网格被组合到共享空间中时，它们可能无法与输入相机对齐，违反重力约束，与附近物体相互穿插，或在物理仿真下变得不稳定。我们提出Fysiverse-3D-SimReady，一个从单张RGB图像及实例和地面提示重建仿真就绪多物体场景的接地细化框架。该方法将生成的物体网格放置到共享的重力对齐场景中，通过可微渲染细化其相机空间姿态，并校正场景级支撑和接触以实现稳定的物理执行。随后，该场景被用于智能体仿真工作流，该工作流将场景特定的任务目标转换为从原始相机视图渲染的可执行物理滚动。实验表明，Fysiverse-3D-SimReady在输入视图对齐、接触合理性和物理稳定性方面优于现有的单图像重建和场景生成基线，同时支持从单图像进行目标条件化的物理交互。

英文摘要

Agentic recognition requires visual perception to move beyond static scene understanding and produce structured scene representations that support the perception--reasoning--action loop. Existing single-image 3D generation methods, however, mainly produce visually plausible object assets rather than simulation-ready scene states. When independently generated meshes are composed in a shared space, they may fail to align with the input camera, violate gravity, interpenetrate nearby objects, or become unstable under physics simulation. We present Fysiverse-3D-SimReady, a grounded refinement framework for reconstructing simulation-ready multi-object scenes from a single RGB image with instance and ground prompts. The method places generated object meshes into a shared gravity-aligned scene, refines their camera-space poses through differentiable rendering, and corrects scene-level supports and contacts for stable physical execution. The scene is then used by an agentic simulation workflow, which converts a scene-specific task goal into an executable physics rollout rendered from the original camera view. Experiments show that Fysiverse-3D-SimReady improves input-view alignment, contact plausibility, and physical stability over existing single-image reconstruction and scene generation baselines, while enabling goal-conditioned physical interactions from a single image.

补充信息

备注

Fyscis AI Technical Report