DreamArt：从单张图像生成可交互的关节物体
DreamArt: Generating Interactable Articulated Objects from a Single Image
AI总结：
DreamArt提出三阶段框架，从单张图像生成高保真可交互关节物体，结合图像到3D重建、视频扩散先验和关节运动优化，实现准确部件形状与合理关节运动。
摘要：
生成关节物体（如笔记本电脑和微波炉）是一个关键且具有挑战性的任务，在具身智能和AR/VR中有着广泛的应用。当前的图像到3D方法主要关注表面几何和纹理，忽略了部件分解和关节建模。同时，神经重建方法（如NeRF或高斯泼溅）依赖于密集的多视图或交互数据，限制了其可扩展性。在本文中，我们介绍了DreamArt，一个新颖的框架，用于从单视图图像生成高保真、可交互的关节资产。DreamArt采用三阶段流程：首先，通过图像到3D生成、掩码提示的3D分割和部件非模态补全的组合，重建部件分割且完整的3D物体网格。其次，我们微调一个视频扩散模型以捕获部件级关节先验，利用可移动部件掩码作为提示和非模态图像来缓解遮挡引起的歧义。最后，DreamArt优化由双四元数表示的关节运动，并进行全局纹理细化和重绘，以确保所有部件具有连贯、高质量的纹理。实验结果表明，DreamArt能有效生成高质量的关节物体，具有准确的部件形状、高外观保真度和合理的关节运动，从而为关节资产生成提供了可扩展的解决方案。我们的项目页面可在https://dream-art-0.github.io/DreamArt/获取。
英文摘要：
Generating articulated objects, such as laptops and microwaves, is a crucial yet challenging task with extensive applications in Embodied AI and AR/VR. Current image-to-3D methods primarily focus on surface geometry and texture, neglecting part decomposition and articulation modeling. Meanwhile, neural reconstruction approaches (e.g., NeRF or Gaussian Splatting) rely on dense multi-view or interaction data, limiting their scalability. In this paper, we introduce DreamArt, a novel framework for generating high-fidelity, interactable articulated assets from single-view images. DreamArt employs a three-stage pipeline: firstly, it reconstructs part-segmented and complete 3D object meshes through a combination of image-to-3D generation, mask-prompted 3D segmentation, and part amodal completion. Second, we fine-tune a video diffusion model to capture part-level articulation priors, leveraging movable part masks as prompt and amodal images to mitigate ambiguities caused by occlusion. Finally, DreamArt optimizes the articulation motion, represented by a dual quaternion, and conducts global texture refinement and repainting to ensure coherent, high-quality textures across all parts. Experimental results demonstrate that DreamArt effectively generates high-quality articulated objects, possessing accurate part shape, high appearance fidelity, and plausible articulation, thereby providing a scalable solution for articulated asset generation. Our project page is available at https://dream-art-0.github.io/DreamArt/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
