单目图像中 3D 物体的姿态估计是计算机视觉中一个基础且长期存在的问题。现有的 6D 姿态估计深度学习方法通常依赖于 3D 物体模型和 6D 姿态标注的可用性假设。然而，在真实数据中精确标注 6D 姿态既复杂、耗时又难以扩展，而合成数据虽能很好地扩展却缺乏真实感。为避免这些问题，我们提出了一种基于重建的弱监督流程，名为 NeRF-Pose，它在训练期间仅需 2D 物体分割和已知的相对相机姿态。遵循先重建后回归的思想，我们首先以隐式神经表示的形式从多视角重建物体。然后，我们训练一个姿态回归网络，以预测图像与重建模型之间的逐像素 2D-3D 对应关系。在推理阶段，该方法仅需单张图像作为输入。使用一种基于 NeRF 的 PnP+RANSAC 算法从预测的对应关系中估计稳定且准确的姿态。在 LineMod 和 LineMod-Occlusion 上的实验表明，尽管仅使用弱标签进行训练，所提方法与最佳的 6D 姿态估计方法相比仍具有最先进的精度。此外，我们用更多真实训练图像扩展了 Homebrewed DB 数据集以支持弱监督任务，并在该数据集上取得了极具竞争力的结果。扩展的数据集和代码即将发布。
英文摘要：
Pose estimation of 3D objects in monocular images is a fundamental and long-standing problem in computer vision. Existing deep learning approaches for 6D pose estimation typically rely on the assumption of availability of 3D object models and 6D pose annotations. However, precise annotation of 6D poses in real data is intricate, time-consuming and not scalable, while synthetic data scales well but lacks realism. To avoid these problems, we present a weakly-supervised reconstruction-based pipeline, named NeRF-Pose, which needs only 2D object segmentation and known relative camera poses during training. Following the first-reconstruct-then-regress idea, we first reconstruct the objects from multiple views in the form of an implicit neural representation. Then, we train a pose regression network to predict pixel-wise 2D-3D correspondences between images and the reconstructed model. At inference, the approach only needs a single image as input. A NeRF-enabled PnP+RANSAC algorithm is used to estimate stable and accurate pose from the predicted correspondences. Experiments on LineMod and LineMod-Occlusion show that the proposed method has state-of-the-art accuracy in comparison to the best 6D pose estimation methods in spite of being trained only with weak labels. Besides, we extend the Homebrewed DB dataset with more real training images to support the weakly supervised task and achieve compelling results on this dataset. The extended dataset and code will be released soon.
RA-NeRF: Robust Neural Radiance Field Reconstruction with Accurate Camera Pose Estimation under Complex Trajectories
School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；Department of Computer Science and Technology, HIT (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术系）；XGRIDS；Department of Computer Science, HKBU（香港大学计算机科学系）；Department of Computer Science and Engineering, HKUST（香港理工大学计算机科学与工程系）；Data Science and Analytics Thrust, HKUST (Guangzhou)（香港理工大学数据科学与分析方向）
Multivariate Gaussian NeRF for Wide Field-of-View Ultrasound Reconstruction
Chair for Computer Aided Medical Procedures (CAMP), Technical University of Munich, Germany（计算机辅助医学程序系（CAMP），慕尼黑技术大学，德国）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心，慕尼黑，德国）；LUMA Vision, Beech Hill Road, Dublin, Ireland（LUMA视觉，Beech Hill Road，都柏林，爱尔兰）
Evaluating the point cloud of individual trees generated from images based on Neural Radiance fields (NeRF) method
Fuzhou University（福州大学）；National Engineering Research Center of Geospatial Information Technology（国家地理空间信息技术工程研究中心）；Key Lab of Spatial Data Mining and Information Sharing of Ministry of Education（空间数据挖掘与信息共享教育部重点实验室）；The Academy of Digital China (Fujian)（数字中国研究院（福建））
Multispectral-NeRF:a multispectral modeling approach based on neural radiance fields
State Key Laboratory of Climate System Prediction and Risk Management, Nanjing Normal University（南京师范大学气候系统预测与风险管理国家重点实验室）；Key Lab of Virtual Geographic Environment (MOE), Nanjing Normal University（南京师范大学虚拟地理环境教育部重点实验室）；State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province)（地理环境演化国家重点实验室培育基地（江苏省））；Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application（江苏省地理信息资源开发与利用协同创新中心）
University of Southern California（南加州大学）；Adobe Research（奥多比研究院）
像NeRF这样的体积神经渲染方法可以生成高质量的视图合成结果，但会针对每个场景进行优化，导致重建时间过长。另一方面，深度多视图立体视觉方法可以通过直接网络推理快速重建场景几何。Point-NeRF结合了这两种方法的优点，通过使用神经3D点云和相关的神经特征来建模辐射场。Point-NeRF可以通过聚集场景表面附近的神经点特征，在基于光线推进的渲染管道中有效地渲染。此外，Point-NeRF可以通过预训练的深度网络的直接推理来初始化，以产生神经点云;可以对该点云进行微调，以超过NeRF的视觉质量，训练时间快30倍。Point-NeRF可以与其他3D重建方法相结合，并通过一种新的修剪和生长机制来处理这些方法中的错误和离群值。在DTU、NeRF Synthetics、ScanNet和Tanks and Temples数据集上的实验表明，Point-NeRF可以超越现有的方法，达到最先进的结果。
OmniNeRF: Hybriding Omnidirectional Distance and Radiance fields for Neural Surface Reconstruction
The University of Sydney（悉尼大学）；Massachusetts Institute of Technology（麻省理工学院）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Beijing Institute of Technology（北京理工大学）；The University of New South Wales（新南威尔士大学）
NeRF-CA: Dynamic Reconstruction of X-ray Coronary Angiography with Extremely Sparse-views
Department of Mathematics & Computer Science, Eindhoven University of Technology（数学与计算机科学系，埃因霍温理工大学）；Philips Healthcare（飞利普医疗）；Department of Electrical Engineering, Eindhoven University of Technology（电子工程系，埃因霍温理工大学）；ASML（ASML公司）
