SSR：用于单视图2D到3D重建的半监督软光栅化器
SSR: Semi-supervised Soft Rasterizer for single-view 2D to 3D Reconstruction
摘要：
最近的工作在弱监督的学习对象网格方面取得了重大进展。软光栅化方法已经实现了从2D图像精确的3D重建与视点监督。在这项工作中，我们进一步减少了标记的努力，允许这样的三维重建方法利用未标记的图像。为了获得这些未标记图像的视点，我们建议使用一个暹罗网络，该网络将两个图像作为输入并输出它们是否对应于相同的视点。在训练过程中，我们最小化交叉熵损失，以最大化预测一对图像是否属于同一视点的概率。为了获得新图像的视点，我们将其与从训练样本中获得的不同视点进行比较，并选择具有最高匹配概率的视点。最后，我们用最有信心的预测视点标记未标记的图像，并训练具有可区分光栅化层的深度网络。我们的实验表明，即使只标记两个对象，在利用未标记的示例时，ShapeNet的IoU也会得到显着改善。代码可在https://github.com/IssamLaradji/SSR上获得。
英文摘要：
Recent work has made significant progress in learning object meshes with weak supervision. Soft Rasterization methods have achieved accurate 3D reconstruction from 2D images with viewpoint supervision only. In this work, we further reduce the labeling effort by allowing such 3D reconstruction methods leverage unlabeled images. In order to obtain the viewpoints for these unlabeled images, we propose to use a Siamese network that takes two images as input and outputs whether they correspond to the same viewpoint. During training, we minimize the cross entropy loss to maximize the probability of predicting whether a pair of images belong to the same viewpoint or not. To get the viewpoint of a new image, we compare it against different viewpoints obtained from the training samples and select the viewpoint with the highest matching probability. We finally label the unlabeled images with the most confident predicted viewpoint and train a deep network that has a differentiable rasterization layer. Our experiments show that even labeling only two objects yields significant improvement in IoU for ShapeNet when leveraging unlabeled examples. Code is available at https://github.com/IssamLaradji/SSR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
