Human can infer the 3D geometry of a scene from a sketch instead of a realistic image, which indicates that the spatial structure plays a fundamental role in understanding the depth of scenes. We are the first to explore the learning of a depth-specific structural representation, which captures the essential feature for depth estimation and ignores irrelevant style information. Our S2R-DepthNet (Synthetic to Real DepthNet) can be well generalized to unseen real-world data directly even though it is only trained on synthetic data. S2R-DepthNet consists of: a) a Structure Extraction (STE) module which extracts a domaininvariant structural representation from an image by disentangling the image into domain-invariant structure and domain-specific style components, b) a Depth-specific Attention (DSA) module, which learns task-specific knowledge to suppress depth-irrelevant structures for better depth estimation and generalization, and c) a depth prediction module (DP) to predict depth from the depth-specific representation. Without access of any real-world images, our method even outperforms the state-of-the-art unsupervised domain adaptation methods which use real-world images of the target domain for training. In addition, when using a small amount of labeled real-world data, we achieve the state-ofthe-art performance under the semi-supervised setting. The code and trained models are available at https://github.com/microsoft/S2R-DepthNet.
The Midas Touch for Metric Depth: Interpretable Metricization for Generalizable Depth Perception
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学上海智能自主系统研究院);College of Electronic & Information Engineering, Tongji University(同济大学电子与信息工程学院);The Chinese University of Hong Kong(香港中文大学);Shanghai Institute of Intelligent Science and Technology(上海智能科学技术研究院)
Depth-Guided Bundle Sampling for Efficient Generalizable Neural Radiance Field Reconstruction
Key Laboratory of Media Audio and Video (Communication University of China), Ministry of Education(媒体音频与视频重点实验室(中国传媒大学),教育部);School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院)
针对可泛化神经辐射场渲染计算量大、采样冗余的问题,提出深度引导的束采样策略,通过分组采样和自适应分配采样点,在DTU数据集上实现1.27 dB PSNR提升和47% FPS增加,并达到2倍渲染加速。
SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning
Shanghai Research Institute for Intelligent Autonomous Systems(上海智能自主系统研究院);Shanghai Institute of Intelligent Science and Technology(上海智能科学技术研究院);Frontiers Science Center for Intelligent Autonomous Systems, Tongji University(同济大学智能自主系统前沿科学中心)
Monocular and Generalizable Gaussian Talking Head Animation
South China University of Technology(华南理工大学);Technical University of Munich(慕尼黑工业大学);Pazhou Laboratory(琶洲实验室);Guangdong University of Technology(广东工业大学);The Hong Kong Polytechnic University(香港理工大学)
EndoGMDE: Generalizable Monocular Depth Estimation with Mixture of Low-Rank Experts for Diverse Endoscopic Scenes
Department of Electronic Engineering, The Chinese University of Hong Kong(香港中文大学电子工程系);College of Biomedical Engineering, Fudan University(复旦大学生物医学工程学院);Shanghai Key laboratory of Medical Image Computing and Computer Assisted Intervention(上海医学影像计算与计算机辅助手术重点实验室);ENT Institute and Department of Otolaryngology, Eye & ENT Hospital of Fudan University(复旦大学眼耳鼻喉科医院耳鼻喉科及耳鼻喉科研究所)
REArtGS++: Generalizable Articulation Reconstruction with Temporal Geometry Constraint via Planar Gaussian Splatting
Hefei Institutes of Physical Science, Chinese Academy of Sciences(合肥物理科学研究院,中国科学院);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学);RoboticsX, Tencent(腾讯机器人院);Shanghai Jiao Tong University(上海交通大学)
EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting
Function Hub, Hong Kong University of Science and Technology(香港科技大学(广州)功能中心);Faculty of Robot Science and Engineering, Northeastern University(东北大学机器人科学与工程学院)
Harnessing Foundation Models for Robust and Generalizable 6-DOF Bronchoscopy Localization
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);The First Affiliated Hospital, Sun Yat-sen University(中山大学第一附属医院);Centre for Artificial Intelligence and Robotics, Chinese Academy of Sciences, HK, China(中国科学院香港人工智能与机器人中心);School of Engineering and Imaging Sciences, King’s College London, UK(伦敦国王学院工程与影像科学学院)
ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Camera
Samsung R&D Institute China-Beijing(三星中国北京研发中心);CFCS, School of Computer Science, Peking University(计算机科学系,北京大学);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院)
SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images
The Chinese University of Hong Kong(香港中文大学);Hong Kong Center for Logistics Robotics(香港物流机器人中心);University of California, Berkeley(加州大学伯克利分校);Zhejiang University(浙江大学)