Reconstructing 3D objects from a single image is an intriguing but challenging problem. One promising solution is to utilize multi-view (MV) 3D reconstruction to fuse generated MV images into consistent 3D objects. However, the generated images usually suffer from inconsistent lighting, misaligned geometry, and sparse views, leading to poor reconstruction quality. To cope with these problems, we present a novel 3D reconstruction framework that leverages intrinsic decomposition guidance, transient-mono prior guidance, and view augmentation to cope with the three issues, respectively. Specifically, we first leverage to decouple the shading information from the generated images to reduce the impact of inconsistent lighting; then, we introduce mono prior with view-dependent transient encoding to enhance the reconstructed normal; and finally, we design a view augmentation fusion strategy that minimizes pixel-level loss in generated sparse views and semantic loss in augmented random views, resulting in view-consistent geometry and detailed textures. Our approach, therefore, enables the integration of a pre-trained MV image generator and a neural network-based volumetric signed distance function (SDF) representation for a single image to 3D object reconstruction. We evaluate our framework on various datasets and demonstrate its superior performance in both quantitative and qualitative assessments, signifying a significant advancement in 3D object reconstruction. Compared with the latest state-of-the-art method Syncdreamer~\cite{liu2023syncdreamer}, we reduce the Chamfer Distance error by about 36\% and improve PSNR by about 30\% .
SCALP: Semi-Supervised Statistical Shape Modeling from Imperfect 3D Photogrammetry via Landmark-Anchored Spectral Warp
Scientific Computing and Imaging Institute, University of Utah(犹他大学科学计算与成像研究所);Kahlert School of Computing, University of Utah(犹他大学卡勒特计算机学院);Division of Pediatric Plastic Surgery, UPMC Children’s Hospital of Pittsburgh(匹兹堡大学医学中心儿童医院小儿整形外科)
PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space
School of Journalism and Information Communication, Huazhong University of Science and Technology(华中科技大学新闻与信息传播学院);ByteDance Inc.(字节跳动公司);Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(科技研究局高性能计算研究所)
University of California at Berkeley(加州大学伯克利分校);Institute of Systems and Robotics, University of Coimbra(科英布拉大学系统与机器人研究所);Anthropics;University College London(伦敦大学学院)
Centerness-based Instance-aware Knowledge Distillation with Task-wise Mutual Lifting for Object Detection on Drone Imagery
Beihang University(北京航空航天大学);State Key Laboratory of Software Development Environment(软件开发环境国家重点实验室);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Hangzhou Innovation Institute, Beihang University(北京航空航天大学杭州创新研究院)
HFT: Lifting Perspective Representations via Hybrid Feature Transformation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);PhiGent Robotics(鉴智机器人)
自动驾驶需要准确而详细的鸟瞰图(BEV)语义分割进行决策,这是高级场景感知最具挑战性的任务之一。正视图到电动汽车的特征转换是电动汽车语义分割的关键技术。现有工程大致可分为两类,即:基于相机模型的特征变换(CBFT)和相机无模型特征变换(CFFT)。在本文中,我们实证分析CBFT和CFFT之间的重要差异。前者基于平世界假设对特征进行变换,这可能会导致位于地平面上方的区域失真。后者是有限的分割性能,由于缺乏几何先验知识和耗时的计算。为了获得CBFT和CFFT的优点并避免其缺点,我们提出了一个新的框架与混合特征变换模块(HFT)。具体来说,我们解耦HFT产生的特征图,用于估计BEV中室外场景的布局。此外,我们设计了一个相互学习的计划,以增加混合转换,通过应用特征模仿。值得注意的是,大量的实验表明,与现有的最佳方法相比,HFT在Argoverse数据集上实现了13.3%的相对改进,在KITTI 3D Object数据集上实现了16.8%的相对改进。这些代码可在https://github.com/JiayuZou2020/HFT上获得。
Toward Accurate Camera-based 3D Object Detection via Cascade Depth Estimation and Calibration
School of Data Science(数据科学学院);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)(香港中文大学(深圳));NIO(蔚来汽车)
Vision-based Lifting of 2D Object Detections for Automated Driving
Intel. Systems and Prod. Engineering(英特尔系统与生产工程);FZI Research Center for Inf. Technology(弗劳恩霍夫研究所信息技术研究中心);Institute of Meas. and Control Systems(测量与控制系统研究所);Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院)