FusionSense：融合常识、视觉与触觉以实现鲁棒的稀疏视角重建
FusionSense: Bridging Common Sense, Vision, and Touch for Robust Sparse-View Reconstruction
AI总结：
本文提出FusionSense框架，通过融合基础模型先验与视觉触觉稀疏观测，利用3D Gaussian Splatting和分层优化策略，解决了透明、反光等挑战性物体的鲁棒3D重建问题。
摘要：
人类能够毫不费力地将常识知识与视觉和触觉的感官输入相结合来理解周围环境。为了模拟这种能力，我们引入了FusionSense，这是一种新颖的3D重建框架，使机器人能够将基础模型的先验与来自视觉和触觉传感器的高度稀疏观测相融合。FusionSense解决了三个关键挑战：(i) 机器人如何高效获取周围场景和物体的鲁棒全局形状信息？(ii) 机器人如何利用几何和常识先验在物体上策略性地选择触觉点？(iii) 诸如触觉信号等局部观测如何改善物体的整体表示？我们的框架采用3D Gaussian Splatting作为核心表示，并引入了分层优化策略，涉及全局结构构建、物体视觉外壳修剪和局部几何约束。这一进展使得在包含透明、反光或深色等传统上具有挑战性物体的环境中实现快速且鲁棒的感知，从而支持更多的下游操作或导航任务。在真实世界数据上的实验表明，我们的框架优于此前最先进的稀疏视角方法。所有代码和数据均在项目网站上开源。
英文摘要：
Humans effortlessly integrate common-sense knowledge with sensory input from vision and touch to understand their surroundings. Emulating this capability, we introduce FusionSense, a novel 3D reconstruction framework that enables robots to fuse priors from foundation models with highly sparse observations from vision and tactile sensors. FusionSense addresses three key challenges: (i) How can robots efficiently acquire robust global shape information about the surrounding scene and objects? (ii) How can robots strategically select touch points on the object using geometric and common-sense priors? (iii) How can partial observations such as tactile signals improve the overall representation of the object? Our framework employs 3D Gaussian Splatting as a core representation and incorporates a hierarchical optimization strategy involving global structure construction, object visual hull pruning and local geometric constraints. This advancement results in fast and robust perception in environments with traditionally challenging objects that are transparent, reflective, or dark, enabling more downstream manipulation or navigation tasks. Experiments on real-world data suggest that our framework outperforms previously state-of-the-art sparse-view methods. All code and data are open-sourced on the project website.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
