iTACO：基于随手拍摄RGBD视频的可交互关节物体数字孪生
iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videos
AI总结：
针对关节物体数字孪生需精心采集数据的痛点，本文提出由粗到精的iTACO框架，可从随手拍RGBD视频推断关节参数、分割可动部件，构建的大规模数据集验证其性能优于现有方法。
摘要：
关节物体在日常生活中十分常见，这类物体的可交互数字孪生在具身智能与机器人领域有大量应用。遗憾的是，当前对真实世界关节物体进行数字化的方法需要精心采集的数据，阻碍了实用化、可扩展、泛化性强的获取流程。本文聚焦于通过手持相机随手拍摄的RGBD视频，完成关节物体的运动分析与部件级分割。用户与关节物体交互的随手拍摄视频可通过智能手机大规模获取，但该场景存在挑战：人与物体交互时，物体与相机同时运动，且存在严重遮挡。为应对这些挑战，我们提出iTACO：一个由粗到精的框架，可从动态RGBD视频中推断关节参数并分割物体的可动部件。为在该新场景下评估方法，我们构建了包含11个类别、284个物体、共784段视频的数据集，规模是已有工作的20倍。我们随后将所提方法与同样以视频为输入的现有方法进行对比，实验表明，在合成数据与真实随手拍摄RGBD视频上，iTACO的表现均优于现有关节物体数字孪生方法。
英文摘要：
Articulated objects are prevalent in daily life. Interactable digital twins of such objects have numerous applications in embodied AI and robotics. Unfortunately, current methods to digitize articulated real-world objects require carefully captured data, preventing practical, scalable, and generalizable acquisition. We focus on motion analysis and part-level segmentation of an articulated object from a casually captured RGBD video shot with a hand-held camera. A casually captured video of an interaction with an articulated object is easy to obtain at scale using smartphones. However, this setting is challenging due to simultaneous object and camera motion and significant occlusions as the person interacts with the object. To tackle these challenges, we introduce iTACO: a coarse-to-fine framework that infers joint parameters and segments movable parts of the object from a dynamic RGBD video. To evaluate our method under this new setting, we build a dataset of 784 videos containing 284 objects across 11 categories that is 20$\times$ larger than available in prior work. We then compare our approach with existing methods that also take video as input. Our experiments show that iTACO outperforms existing articulated object digital twin methods on both synthetic and real casually captured RGBD videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
