TransDiff：一种使用单张 RGB-D 图像操作透明物体的基于扩散的方法
TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image
AI总结：
针对透明物体因反射、折射导致深度估计困难的问题，提出 TransDiff 框架，利用 DDPM 并融合 RGB 语义、边缘、法线及初始深度特征完成单视图 RGB-D 深度补全，在合成与真实基准上均优于基线。
摘要：
由于透明物体的反射和折射特性带来的复杂性，操作透明物体面临重大挑战，这些特性严重阻碍了对其三维形状的准确估计。为应对这些挑战，我们提出了一种基于单视图 RGB-D 的深度补全框架 TransDiff，该框架利用去噪扩散概率模型（Denoising Diffusion Probabilistic Models，DDPM），在桌面场景中实现与材质无关的物体抓取。具体而言，我们利用从 RGB 图像中提取的特征（包括语义分割、边缘图和法线图）来条件化深度图生成过程。我们的方法学习一种迭代去噪过程，在初始细化深度信息的引导下，将随机深度分布转换为深度图，从而确保在涉及透明物体的场景中获得更准确的深度估计。此外，我们提出了一种新颖的训练方法，以更好地对齐带噪深度特征与 RGB 图像特征，这些特征被用作条件来逐步细化深度估计。最后，我们采用改进的推理过程来加速去噪流程。通过全面的实验验证，我们证明本方法在合成基准和真实世界基准上均显著优于基线方法，同时推理时间可接受。本方法的演示可在 https://wang-haoxiao.github.io/TransDiff/ 查看。
英文摘要：
Manipulating transparent objects presents significant challenges due to the complexities introduced by their reflection and refraction properties, which considerably hinder the accurate estimation of their 3D shapes. To address these challenges, we propose a single-view RGB-D-based depth completion framework, TransDiff, that leverages the Denoising Diffusion Probabilistic Models(DDPM) to achieve material-agnostic object grasping in desktop. Specifically, we leverage features extracted from RGB images, including semantic segmentation, edge maps, and normal maps, to condition the depth map generation process. Our method learns an iterative denoising process that transforms a random depth distribution into a depth map, guided by initially refined depth information, ensuring more accurate depth estimation in scenarios involving transparent objects. Additionally, we propose a novel training method to better align the noisy depth and RGB image features, which are used as conditions to refine depth estimation step by step. Finally, we utilized an improved inference process to accelerate the denoising procedure. Through comprehensive experimental validation, we demonstrate that our method significantly outperforms the baselines in both synthetic and real-world benchmarks with acceptable inference time. The demo of our method can be found on https://wang-haoxiao.github.io/TransDiff/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
