面向已见与未见类别的单视图3D网格重建
Single-view 3D Mesh Reconstruction for Seen and Unseen Categories
AI总结：
本文提出端到端两阶段网络GenMesh，通过分解图像到网格映射、跨2D/3D局部特征采样及多视图轮廓损失，提升单视图3D网格重建在未见类别上的泛化能力。
摘要：
单视图3D物体重建是一项基础且具有挑战性的计算机视觉任务，旨在从单视图RGB图像中恢复3D形状。大多数现有基于深度学习的重建方法都在相同类别上进行训练和评估，在处理训练期间未见过的新类别物体时无法很好地工作。针对这一问题，本文研究单视图3D网格重建，以探究模型在未见类别上的泛化能力，并鼓励模型逐字（真实）重建物体。具体而言，我们提出了一个名为GenMesh的端到端两阶段网络，以打破重建中的类别边界。首先，我们将复杂的图像到网格映射分解为两个更简单的映射，即图像到点映射和点到网格映射，而后者主要是一个几何问题，对物体类别的依赖较小。其次，我们设计了一种2D和3D特征空间中的局部特征采样策略，以捕捉跨物体共享的局部几何结构，从而增强模型泛化能力。第三，除传统的点对点监督外，我们引入多视图轮廓损失来监督表面生成过程，这提供了额外的正则化，并进一步缓解了过拟合问题。实验结果表明，在不同场景和多种指标下，我们的方法在ShapeNet和Pix3D上显著优于现有工作，尤其是对于新物体。项目链接为https://github.com/Wi-sc/GenMesh。
英文摘要：
Single-view 3D object reconstruction is a fundamental and challenging computer vision task that aims at recovering 3D shapes from single-view RGB images. Most existing deep learning based reconstruction methods are trained and evaluated on the same categories, and they cannot work well when handling objects from novel categories that are not seen during training. Focusing on this issue, this paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories and encourage models to reconstruct objects literally. Specifically, we propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction. Firstly, we factorize the complicated image-to-mesh mapping into two simpler mappings, i.e., image-to-point mapping and point-to-mesh mapping, while the latter is mainly a geometric problem and less dependent on object categories. Secondly, we devise a local feature sampling strategy in 2D and 3D feature spaces to capture the local geometry shared across objects to enhance model generalization. Thirdly, apart from the traditional point-to-point supervision, we introduce a multi-view silhouette loss to supervise the surface generation process, which provides additional regularization and further relieves the overfitting problem. The experimental results show that our method significantly outperforms the existing works on the ShapeNet and Pix3D under different scenarios and various metrics, especially for novel objects. The project link is https://github.com/Wi-sc/GenMesh.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
