KitchenTwin: 基于语义和几何的3D厨房数字孪生
KitchenTwin: Semantically and Geometrically Grounded 3D Kitchen Digital Twins
AI总结：
本文提出一种基于语义和几何的3D厨房数字孪生框架，通过融合视觉引导的对象网格与Transformer预测的全局点云，实现几何一致的数字孪生构建。
摘要：
具身AI的训练和评估需要具有准确度量几何和语义基础的对象中心数字孪生环境。最近的基于Transformer的前馈重建方法可以高效地从稀疏单目视频预测全局点云，但这些几何结构存在固有的尺度模糊和不一致的坐标约定。这种不匹配阻碍了这些无量纲点云预测与局部重建对象网格的可靠融合。我们提出了一种新的尺度感知的3D融合框架，通过将视觉引导的对象网格与Transformer预测的全局点云进行对齐，以构建具有度量一致性的数字孪生。我们的方法引入了一种由视觉-语言模型（VLM）引导的几何锚机制，通过恢复准确的现实世界度量尺度来解决这一基本坐标不匹配问题。为了融合这些网络，我们提出了一种几何感知的对齐流程，通过重力对齐的垂直估计、曼哈顿世界结构约束和无碰撞的局部细化来显式强制物理合理性。在真实室内厨房环境中进行的实验表明，该方法在跨网络对象对齐和几何一致性方面提高了下游任务的性能，包括多原始形拟合和度量测量。我们还引入了一个开源的室内数字孪生数据集，包含具有度量缩放场景和语义基础和注册的对象中心网格注释。
英文摘要：
Embodied AI training and evaluation require object-centric digital twin environments with accurate metric geometry and semantic grounding. Recent transformer-based feedforward reconstruction methods can efficiently predict global point clouds from sparse monocular videos, yet these geometries suffer from inherent scale ambiguity and inconsistent coordinate conventions. This mismatch prevents the reliable fusion of these dimensionless point cloud predictions with locally reconstructed object meshes. We propose a novel scale-aware 3D fusion framework that registers visually grounded object meshes with transformer-predicted global point clouds to construct metrically consistent digital twins. Our method introduces a Vision-Language Model (VLM)-guided geometric anchor mechanism that resolves this fundamental coordinate mismatch by recovering an accurate real-world metric scale. To fuse these networks, we propose a geometry-aware registration pipeline that explicitly enforces physical plausibility through gravity-aligned vertical estimation, Manhattan-world structural constraints, and collision-free local refinement. Experiments on real indoor kitchen environments demonstrate improved cross-network object alignment and geometric consistency for downstream tasks, including multi-primitive fitting and metric measurement. We additionally introduce an open-source indoor digital twin dataset with metrically scaled scenes and semantically grounded and registered object-centric mesh annotations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
