Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation
College of Computer Science, Sichuan University, Chengdu, China（四川大学计算机学院）；Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China（四川大学消化内科与肝病科）
Juan Yang, Yuyan Zhang, Han Jia, Bing Hu, Wanzhong Song
备注：
\c{opyright} 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
Monocular depth estimation (MDE) for colonoscopy is hampered by the domain gap between simulated and real-world images. Existing image-to-image translation methods, which use depth as a posterior constraint, often produce structural distortions and specular highlights by failing to balance realism with structure consistency. To address this, we propose a Structure-to-Image paradigm that transforms the depth map from a passive constraint into an active generative foundation. We are the first to introduce phase congruency to colonoscopic domain adaptation and design a cross-level structure constraint to co-optimize geometric structures and fine-grained details like vascular textures. In zero-shot evaluations conducted on a publicly available phantom dataset, the MDE model that was fine-tuned on our generated data achieved a maximum reduction of 44.18% in RMSE compared to competing methods. Our code is available at https://github.com/YyangJJuan/PC-S2I.git.
Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation
Department of Biomedical Engineering, National University of Singapore, Singapore（新加坡国立大学生物医学工程系）；Department of Radiotherapy, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, China（西安交通大学第一附属医院放疗科）
ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopy
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Vetex Lab（Vetex实验室）；Southeast University - Monash University Joint Graduate School（东南大学-墨尔本大学联合研究生院）；Monash Suzhou Research Institute（墨尔本大学苏州研究院）；Faculty of Information Technology, Monash University（墨尔本大学信息科技学院）
Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（武汉光电国家研究中心，华中科技大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Wuhan United Imaging Healthcare Surgical Technology Co., Ltd（武汉联影医疗手术技术有限公司）
ColonSplat: Reconstruction of Peristaltic Motion in Colonoscopy with Dynamic Gaussian Splatting
Jagiellonian University, Kraków, Poland（杰洛尼摩大学）；Warsaw University of Technology, Poland（华沙技术大学）；Sano Centre for Computational Medicine, Kraków, Poland（Sano计算医学中心）；University of Verona, Italy（威尼斯大学）；IDEAS Research Institute, Warsaw, Poland（IDEAS研究机构）
C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism
Department of Biomedical Engineering, Johns Hopkins University（生物医学工程系，约翰霍普金斯大学）；Division of Gastroenterology and Hepatology, Johns Hopkins Medicine（胃肠病学与肝病学分会，约翰霍普金斯医学）
Fudan University（复旦大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Huashan Hospital（华山医院）
Automatic colorectal polyp detection in colonoscopy video is a fundamental task, which has received a lot of attention. Manually annotating polyp region in a large scale video dataset is time-consuming and expensive, which limits the development of deep learning techniques. A compromise is to train the target model by using labeled images and infer on colonoscopy videos. However, there are several issues between the image-based training and video-based inference, including domain differences, lack of positive samples, and temporal smoothness. To address these issues, we propose an Image-video-…
RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applications
Department of Electronics, Information, and Bioengineering, Politecnico di Milano（米兰理工大学电子、信息与生物工程系）；School of Electronic and Electrical Engineering, University of Leeds（利兹大学电子与电气工程学院）；Division of Gastroenterology, Vanderbilt University Medical Center（范德比尔特大学医学中心胃肠科）；Department of Mechanical Engineering, Vanderbilt University（范德比尔特大学机械工程系）；Endoscopy Unit, Humanitas Clinical and Research Center IRCCS（人类itas临床与研究中心内窥镜单元）；Department of Biomedical Sciences, Humanitas University（人类itas大学生物医学科学系）
While data-driven approaches excel at many image analysis tasks, the performance of these approaches is often limited by a shortage of annotated data available for training. Recent work in semi-supervised learning has shown that meaningful representations of images can be obtained from training with large quantities of unlabeled data, and that these representations can improve the performance of supervised tasks. Here, we demonstrate that an unsupervised jigsaw learning task, in combination with supervised training, results in up to a 9.8% improvement in correctly classifying lesions in colono…
CASA-SDF: Curriculum-Aware Spatial Adaptation with Curvature-Guided Density for Neural Implicit Surface Reconstruction
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Automation, Nanjing University of Science and Technology（南京理工大学自动化学院）
University of Minnesota（明尼苏达大学）；Facebook Reality Labs（Facebook现实实验室）
Improvements in data-capture and face modeling techniques have enabled us to create high-fidelity realistic face models. However, driving these realistic face models requires special input data, e.g. 3D meshes and unwrapped textures. Also, these face models expect clean input data taken under controlled lab environments, which is very different from data collected in the wild. All these constraints make it challenging to use the high-fidelity models in tracking for commodity cameras. In this paper, we propose a self-supervised domain adaptation approach to enable the animation of high-fidelity…
Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints
University of Electronic Science and Technology of China（电子科技大学）；National Research Council, Institute of Methodologies for Environmental Analysis (CNR-IMAA)（国家研究委员会，环境分析方法研究所（CNR-IMAA））
3D video avatars can empower virtual communications by providing compression, privacy, entertainment, and a sense of presence in AR/VR. Best 3D photo-realistic AR/VR avatars driven by video, that can minimize uncanny effects, rely on person-specific models. However, existing person-specific photo-realistic 3D models are not robust to lighting, hence their results typically miss subtle facial behaviors and cause artifacts in the avatar. This is a major drawback for the scalability of these models in communication systems (e.g., Messenger, Skype, FaceTime) and AR/VR. This paper addresses previou…
In recent years, diffusion models have demonstrated remarkable success in high-fidelity image synthesis. However, fine-tuning these models for specialized domains, such as medical imaging, remains challenging due to limited domain-specific data and the high computational cost of full model adaptation. In this paper, we introduce Lite-Diff (Lightweight Diffusion Model Adaptation), a novel finetuning approach that integrates lightweight adaptation layers into a frozen diffusion U-Net while enhancing training with a latent morphological autoencoder (for domain-specific latent consistency) and a p…
In this work, we propose an ID-preserving talking head generation framework, which advances previous methods in two aspects. First, as opposed to interpolating from sparse flow, we claim that dense landmarks are crucial to achieving accurate geometry-aware flow fields. Second, inspired by face-swapping methods, we adaptively fuse the source identity during synthesis, so that the network better preserves the key characteristics of the image portrait. Although the proposed model surpasses prior generation fidelity on established benchmarks, to further make the talking head generation qualified f…
EEG-Driven Image Reconstruction with Saliency-Guided Diffusion Models
Ivannikov Institute for System Programming of the Russian Academy of Sciences（俄罗斯科学院伊万诺夫系统编程研究所）；Research Center for Trusted Artificial Intelligence（可信人工智能研究中心）；AI Talent Hub, ITMO University（ITMO大学人工智能人才中心）；Moscow, Russia（俄罗斯莫斯科）
National Key Lab for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Tencent Youtu Lab（腾讯云图实验室）
Recent deep-learning based Super-Resolution (SR) methods have achieved remarkable performance on images with known degradation. However, these methods always fail in real-world scene, since the Low-Resolution (LR) images after the ideal degradation (e.g., bicubic down-sampling) deviate from real source domain. The domain gap between the LR images and the real-world images can be observed clearly on frequency density, which inspires us to explictly narrow the undesired gap caused by incorrect degradation. From this point of view, we design a novel Frequency Consistent Adaptation (FCA) that ensu…
High-Fidelity Facial Albedo Estimation via Texture Quantization
University of Technology Sydney（悉尼科技大学）；Shanghai Jiao Tong University（上海交通大学）；DeepGlint（旷视科技（北京））；Insightface；Zhejiang University（浙江大学）；Imperial College London（伦敦帝国学院）
Thanks to the recent development of deep generative models, it is becoming possible to generate high-quality images with both fidelity and diversity. However, the training of such generative models requires a large dataset. To reduce the amount of data required, we propose a new method for transferring prior knowledge of the pre-trained generator, which is trained with a large dataset, to a small dataset in a different domain. Using such prior knowledge, the model can generate images leveraging some common sense that cannot be acquired from a small dataset. In this work, we propose a novel met…
SpiralDiff: Spiral Diffusion with LoRA for RGB-to-RAW Conversion Across Cameras
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；State key laboratory of Smart Power Distribution Equipment and System, Tianjin University（天津大学智能电力分配设备与系统国家重点实验室）
HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks
University of Electronic Science and Technology of China（电子科技大学）；Institute of Electronics and Information Industry Technology of Kashgar（喀什电子信息产业技术研究院）
Tell2Adapt: A Unified Framework for Source Free Unsupervised Domain Adaptation via Vision Foundation Model
Northeastern University（东北大学）；Engineering Research Center of Medical Imaging and Intelligent Analysis, Ministry of Education（教育部医学影像与智能分析工程研究中心）；Key Laboratory of Medical Image Computing, Ministry of Education（教育部医学影像计算重点实验室）
SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation
School of Software, Northwestern Polytechnical University（西北工业大学软件学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Software Engineering, USTC（USTC软件工程学院）；Australian Institute of Health Innovation, Macquarie University（麦考瑞大学健康创新研究所）；Faculty of Applied Science, Macao Polytechnic University（澳门理工学院应用科学学院）；School of Computer Software, Tianjin University（天津大学计算机软件学院）
