Semantic segmentation is an essential step for electron microscopy (EM) image analysis. Although supervised models have achieved significant progress, the need for labor intensive pixel-wise annotation is a major limitation. To complicate matters further, supervised learning models may not generalize well on a novel dataset due to domain shift. In this study, we introduce an adversarial-prediction guided multi-task network to learn the adaptation of a well-trained model for use on a novel unlabeled target domain. Since no label is available on target domain, we learn an encoding representation not only for the supervised segmentation on source domain but also for unsupervised reconstruction of the target data. To improve the discriminative ability with geometrical cues, we further guide the representation learning by multi-level adversarial learning in semantic prediction space. Comparisons and ablation study on public benchmark demonstrated state-of-the-art performance and effectiveness of our approach.
NELVT, School of EE & CS, Peking University（北京大学电子信息与计算机科学学院NELVT）；University of Chinese Academy of Sciences（中国科学院大学）；School of ECE, Peking University（北京大学电子信息工程学院）；Pengcheng Laboratory（鹏城实验室）；Beijing Institute of Technology（北京理工大学）；Nanyang Technological University（南洋理工大学）
Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness of object detection models. However, existing UDA methods largely ignore the instantaneous data distribution during model learning, which could deteriorate the feature representation given large domain shift. In this work, we propose a Self-Guided Adaptation (SGA) model, target at aligning feature representation and transferring object detection models across domains while considering the instantaneous alignment difficulty. The core of SGA is to calculate "hardness" factors for samp…
A Chebyshev Confidence Guided Source-Free Domain Adaptation Framework for Medical Image Segmentation
Harbin Institute of Technology at Shenzhen（哈尔滨工业大学（深圳））；The Peng Cheng Laboratory（鹏城实验室）；Guangdong Provincial Key Laboratory of Aerospace Communication and Networking Technology（广东省航空航天通信与网络技术重点实验室）
MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM
College of Computer Science, Sichuan University（四川大学计算机学院）；West China Hospital, Sichuan University（四川大学华西医院）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）
The University of Sydney（悉尼大学）；Tencent AI Lab（腾讯人工智能实验室）
Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. UDA is of particular significance since no extra effort is devoted to annotating target domain samples. However, the different data distributions in the two domains, or \emph{domain shift/discrepancy}, inevitably compromise the UDA performance. Although there has been a progress in matching the marginal distributions between two domains, the classifier favors the source domain features and makes incorrect predictions on the target domain due to category…
We address the problem of semantic nighttime image segmentation and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in th…
Most progress in semantic segmentation reports on daytime images taken under favorable illumination conditions. We instead address the problem of semantic segmentation of nighttime images and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night via labeled synthetic images and unlabeled real images, bo…
CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentation
Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；National Supercomputing Center in Shenzhen（深圳国家超算中心）；The Hong Kong University of Science and Technology（香港科技大学）；Beijing Institute of Technology（北京理工大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）
Oak Ridge National Lab (ORNL)（橡树岭国家实验室）；St. Jude Children’s Hospital（圣裘德儿童医院）
Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the $z$-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this work, we introduce a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue. The proposed method combines the 2D YOLO detection method with a m…
Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy constraints. We focus on source-free domain adaptation for semantic segmentation, wherein a source model must adapt itself to a new target domain given only unlabeled target data. We propose Augmentation Consistency-guided Self-training (AUGCO), a source-free adaptation algorithm that uses the model's pixel-level predictive consistency across diverse, automatically generated views of each target image along with mode…
AdaptFly: Prompt-Guided Adaptation of Foundation Models for Low-Altitude UAV Networks
Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（申明-明伍智能工程学院，华南理工大学）；Key Laboratory of Cognitive Radio and Information Processing, Ministry of Education (Guilin University of Electronic Technology)（认知无线电与信息处理重点实验室，教育部（桂林电子科技大学））；School of Information and Communication, Guilin University of Electronic Technology（信息与通信学院，桂林电子科技大学）
CGSA: Class-Guided Slot-Aware Adaptation for Source-Free Object Detection
School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；Department of Electrical & Computer Engineering, National University of Singapore（电气与计算机工程系，新加坡国立大学）；School of Information and Electronics, Beijing Institute of Technology（信息与电子学院，北京理工大学）
Pay Attention to the Atlas: Atlas-Guided Test-Time Adaptation Method for Robust 3D Medical Image Segmentation
School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Imperial College London（伦敦帝国学院）；HeartFlow, Inc.（哈特弗洛公司）；School of Medicine, Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院医学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Computer Science, University of Sheffield（谢菲尔德大学计算机科学系）
University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；University of Pittsburgh（匹兹堡大学）；Microsoft Corporation（微软公司）
Unsupervised domain adaptation (UDA) for nuclei instance segmentation is important for digital pathology, as it alleviates the burden of labor-intensive annotation and domain shift across datasets. In this work, we propose a Cycle Consistency Panoptic Domain Adaptive Mask R-CNN (CyC-PDAM) architecture for unsupervised nuclei segmentation in histopathology images, by learning from fluorescence microscopy images. More specifically, we first propose a nuclei inpainting mechanism to remove the auxiliary generated objects in the synthesized images. Secondly, a semantic branch with a domain discrimi…
Present domain adaptation methods usually perform explicit representation alignment by simultaneously accessing the source data and target data. However, the source data are not always available due to the privacy preserving consideration or bandwidth limitation. Source-free domain adaptation aims to solve the above problem by performing domain adaptation without accessing the source data. The adaptation paradigm is receiving more and more attention in recent years, and multiple works have been proposed for unsupervised source-free domain adaptation. However, without utilizing any supervised s…
Kyushu University（九州大学）；Kyoto University（京都大学）；The University of Tokyo（东京大学）
Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classification tasks. For robustness, we propose a weakly-supervised clustering pipeline to obtain high-purity clusters and utilize the clusters in representation learning for domain adaptation. The proposed …
Peking University（北京大学）；South China University of Technology（华南理工大学）；National University of Singapore（新加坡国立大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
This paper studies a new, practical but challenging problem, called Class-Incremental Unsupervised Domain Adaptation (CI-UDA), where the labeled source domain contains all classes, but the classes in the unlabeled target domain increase sequentially. This problem is challenging due to two difficulties. First, source and target label sets are inconsistent at each time step, which makes it difficult to conduct accurate domain alignment. Second, previous target classes are unavailable in the current step, resulting in the forgetting of previous knowledge. To address this problem, we propose a nov…
University of Jena（耶拿大学）；Leibniz Institute of Photonic Technology（莱布尼茨光子技术研究所）；Jena Center for Soft Matter（耶拿软物质中心）；Centre for Structural Systems Biology（结构系统生物学中心）；University of Hamburg（汉堡大学）；Biozentrum, University of Basel（巴塞尔大学生物中心）；Leibniz Institute for Natural Product Research and Infection Biology - Hans Knöll Institute（莱布尼茨天然产物研究与感染生物学研究所——汉斯·克诺尔研究所）
Correlative light and electron microscopy is a powerful tool to study the internal structure of cells. It combines the mutual benefit of correlating light (LM) and electron (EM) microscopy information. However, the classical approach of overlaying LM onto EM images to assign functional to structural information is hampered by the large discrepancy in structural detail visible in the LM images. This paper aims at investigating an optimized approach which we call EM-guided deconvolution. It attempts to automatically assign fluorescence-labelled structures to details visible in the EM image to br…
Deep neural networks have significantly improved appearance-based gaze estimation accuracy. However, it still suffers from unsatisfactory performance when generalizing the trained model to new domains, e.g., unseen environments or persons. In this paper, we propose a plug-and-play gaze adaptation framework (PnP-GA), which is an ensemble of networks that learn collaboratively with the guidance of outliers. Since our proposed framework does not require ground-truth labels in the target domain, the existing gaze estimation networks can be directly plugged into PnP-GA and generalize the algorithms…
Motion-guided sparse correction enables expert-quality point tracking across diverse microscopy regimes
Department of Computer Science, Old Dominion University（奥德赛大学计算机科学系）；Department of Biology, Massachusetts Institute of Technology（麻省理工学院生物学系）；The Picower Institute for Learning and Memory, Massachusetts Institute of Technology（麻省理工学院学习与记忆研究所）；Department of Physics and Technology, UiT--The Arctic University of Norway（挪威北极大学物理与技术系）；Department of Physics, University of Oslo（奥斯陆大学物理系）；School of Data Science, Old Dominion University（奥德赛大学数据科学学院）；Department of Physics, Old Dominion University（奥德赛大学物理系）
Nip Rumors in the Bud: Retrieval-Guided Topic-Level Adaptation for Test-Time Fake News Video Detection
University of Electronic Science and Technology of China（电子科技大学）；Aiwen Technology Co., Ltd.（爱文科技有限公司）；Intelligent Digital Media Technology Key Laboratory of Sichuan Province（四川省智能数字媒体技术重点实验室）
Domain adaptation (DA) or domain generalization (DG) for face presentation attack detection (PAD) has attracted attention recently with its robustness against unseen attack scenarios. Existing DA/DG-based PAD methods, however, have not yet fully explored the domain-specific style information that can provide knowledge regarding attack styles (e.g., materials, background, illumination and resolution). In this paper, we introduce a novel Style-Guided Domain Adaptation (SGDA) framework for inference-time adaptive PAD. Specifically, Style-Selective Normalization (SSN) is proposed to explore the do…
While fully-supervised deep learning yields good models for urban scene semantic segmentation, these models struggle to generalize to new environments with different lighting or weather conditions for instance. In addition, producing the extensive pixel-level annotations that the task requires comes at a great cost. Unsupervised domain adaptation (UDA) is one approach that tries to address these issues in order to make such systems more scalable. In particular, self-supervised learning (SSL) has recently become an effective strategy for UDA in semantic segmentation. At the core of such methods…
ORB-SfMLearner: ORB-Guided Self-supervised Visual Odometry with Selective Online Adaptation
College of Electrical Engineering, Sichuan University（四川大学电气工程学院）；Rice University（里士满大学）；Graduate Institute of Networking and Multimedia, National Taiwan University（台湾大学网络与多媒体研究所）；Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）
Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of British Columbia（不列颠哥伦比亚大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
In this study, we focus on the unsupervised domain adaptation problem where an approximate inference model is to be learned from a labeled data domain and expected to generalize well to an unlabeled data domain. The success of unsupervised domain adaptation largely relies on the cross-domain feature alignment. Previous work has attempted to directly align latent features by the classifier-induced discrepancies. Nevertheless, a common feature space cannot always be learned via this direct feature alignment especially when a large domain gap exists. To solve this problem, we introduce a Gaussian…
Computer Science Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校计算机科学系）；Mechanical and Aerospace Engineering Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校机械与航空航天工程系）；Bioengineering Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校生物工程系）；Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校乔纳森综合癌症中心）
Unsupervised domain adaptation for semantic segmentation has been intensively studied due to the low cost of the pixel-level annotation for synthetic data. The most common approaches try to generate images or features mimicking the distribution in the target domain while preserving the semantic contents in the source domain so that a model can be trained with annotations from the latter. However, such methods highly rely on an image translator or feature extractor trained in an elaborated mechanism including adversarial training, which brings in extra complexity and instability in the adaptati…
Deimos Space（德莫斯航天公司）；Univ. Autónoma de Madrid（马德里自治大学）
Spacecraft pose estimation is a key task to enable space missions in which two spacecrafts must navigate around each other. Current state-of-the-art algorithms for pose estimation employ data-driven techniques. However, there is an absence of real training data for spacecraft imaged in space conditions due to the costs and difficulties associated with the space environment. This has motivated the introduction of 3D data simulators, solving the issue of data availability but introducing a large gap between the training (source) and test (target) domains. We explore a method that incorporates 3D…
Rutgers University（罗格斯大学）；Bytedance Inc.（字节跳动公司）；Playform AI
Can a text-to-image diffusion model be used as a training objective for adapting a GAN generator to another domain? In this paper, we show that the classifier-free guidance can be leveraged as a critic and enable generators to distill knowledge from large-scale text-to-image diffusion models. Generators can be efficiently shifted into new domains indicated by text prompts without access to groundtruth samples from target domains. We demonstrate the effectiveness and controllability of our method through extensive experiments. Although not trained to minimize CLIP loss, our model achieves equal…
Ain Shams University（艾因夏姆斯大学）；The American University in Cairo（开罗美国大学）
Face Presentation Attack Detection (PAD) has drawn increasing attentions to secure the face recognition systems that are widely used in many applications. Conventional face anti-spoofing methods have been proposed, assuming that testing is from the same domain used for training, and so cannot generalize well on unseen attack scenarios. The trained models tend to overfit to the acquisition sensors and attack types available in the training data. In light of this, we propose an end-to-end learning framework based on Domain Adaptation (DA) to improve PAD generalization capability. Labeled source-…
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
When a pre-trained general auto-segmentation model is deployed at a new institution, a support framework in the proposed Prior-guided DDL network will learn the systematic difference between the model predictions and the final contours revised and approved by clinicians for an initial group of patients. The learned style feature differences are concatenated with the new patients (query) features and then decoded to get the style-adapted segmentations. The model is independent of practice styles and anatomical structures. It meta-learns with simulated style differences and does not need to be e…
Flatness Guided Test-Time Adaptation for Vision-Language Models
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络科学与技术学院）；National Engineering Laboratory for Brain-Inspired Intelligence Technology and Applications, University of Science and Technology of China（中国科学技术大学脑启发式智能技术与应用国家工程实验室）；Peng Cheng Laboratory, Shenzhen, China（深圳鹏城实验室）
Million-scale multimodal pollen microscopy with expert-guided foundation models
Department of Physics of Complex Systems, ELTE Eötvös Loránd University（ELTE罗兰大学复杂物理系）；The Palynological Laboratory at the Swedish Museum of Natural History（瑞典自然历史博物馆孢粉学实验室）；National Centre for Public Health and Pharmacy（国家公共卫生与药品中心）；INRAE, UR 546 BioSP, Site Agroparc（法国国家农业、食品与环境研究院，UR 546 BioSP，阿格罗帕克园区）；National Korányi Institute for Pulmonology（国家科拉尼肺病研究所）；Health Data Science and AI Knowledge Centre, Health Services Management Training Centre, Faculty of Health and Public Administration, Semmelweis University（塞梅维什大学健康与公共管理学院卫生服务管理培训中心健康数据科学与人工智能知识中心）；Department of Biological Physics, ELTE Eötvös Loránd University（ELTE罗兰大学生物物理系）
提出百万级多模态花粉显微镜数据集Pollen AI Atlas，结合专家引导的视觉-语言模型生成形态描述，实现跨区域、跨设置的高精度花粉识别与检索。
Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided
College of Automation, Nanjing University of Posts and Telecommunications（南京邮电大学自动化学院）；Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电气与计算机工程系）；the Affiliated Stomatological Hospital of Nanjing Medical University（南京医科大学附属口腔医院）
